SaltyCrane Blog — Notes on JavaScript and web development

How to use ast-grep with GraphQL

ast-grep is a code search + replace tool that uses the abstract syntax tree (AST). It has language support for TypeScript, Python, and others, but it doesn't support GraphQL out of the box. However, there is documentation for supporting other languages: Custom Language Support.

These are my notes using that Custom Language Support documentation along with the Multi-Language Documents documentation to set up GraphQL support for ast-grep on macOS.

Prepare Tree-sitter Tool and Parser

  • Install tree-sitter CLI

    brew install tree-sitter
    

    (Alternatively, npm install -g tree-sitter-cli)

  • Test tree-sitter is installed

    tree-sitter --version
    
    tree-sitter 0.23.0
    
  • Use a working directory

    I'll use /tmp as my working directory, but a different directory can be used

    cd /tmp
    
  • Get the tree-sitter GraphQL grammar

    git clone https://github.com/bkegley/tree-sitter-graphql.git
    

    (This is the repo used by the neovim tree-sitter library.)

Compile the Parser as a Dynamic Library

  • Compile the GraphQL parser as a dynamic library

    cd tree-sitter-graphql
    export TREE_SITTER_LIBDIR=/tmp
    tree-sitter test
    

    Note: I got the following error when I ran this. If anyone knows what is going on, please let me know.

    Error in query file "formatter.scm"
    
    Caused by:
        Query error at 5:31. Impossible pattern:
        (input_object_type_definition (input_value_definition) @field_definition)
    

    However, it still generated the graphql.dylib file in /tmp. Note: according to ChatGPT, .dylib is used on Mac, .so is used on Linux, and .dll is used on Windows.

Register Language in sgconfig.yml

  • Create a new ast-grep project

    cd /tmp
    sg new
    
    No sgconfig.yml found. Creating a new ast-grep project...
    > Where do you want to have your rules? rules
    > Do you want to create rule tests? No
    > Do you want to create folder for utility rules? No
    Your new ast-grep project has been created!
    
  • Edit sgconfig.yml to add GraphQL as a custom language

    /tmp/sgconfig.yml:

    ruleDirs:
    - ./rules
    customLanguages:
    graphql:
        libraryPath: graphql.dylib
        extensions: [graphql]
        expandoChar: $
    

Use It!

  • Search for all input types in /path/to/my/schema.graphql

    sg -p "input" -l graphql /path/to/my/schema.graphql
    
    /path/to/my/schema.graphql
    6│input AddOrUpdateResidualWorksheetInputType {
    120│input AutoAssignStyleCodeInputType {
    478│input BulkCreateVehicleReleaseInputType {
    483│input BulkDeleteVehicleReleaseInputType {
    487│input BulkEditVehicleReleaseInputType {
    492│input BulkImportStyleFromCppInputType {
    949│input CloneModelInputType {
    958│input CloneModelStyleInputType {
    973│input CloneStyleInputType {
    1749│input CppConfigurationInputType {
    
  • Search for all DateTime input fields (but not output fields) in /path/to/my/schema.graphql. Note: the tree-sitter-graphql grammer.js file contains different kinds that can be used for filtering.

    sg scan --inline-rules '
    id: datetimeinputs
    language: graphql
    rule:
      regex: "DateTime"
      kind: input_value_definition
    ' /path/to/my/schema.graphql
    
    help[datetimeinputs]:
         ┌─ /path/to/my/schema.graphql:2356:3
         │
    2356 │   configurationLastUpdatedAt: DateTime
         │   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    
    help[datetimeinputs]:
         ┌─ /path/to/my/schema.graphql:8331:69
         │
    8331 │   previousStyleSnapshot(styleSnapshotRowId: Guid, styleRowId: Long, versionDate: DateTime): StyleSnapshotTypeOrOperationErrorType
         │                                                                     ^^^^^^^^^^^^^^^^^^^^^
    
    help[datetimeinputs]:
         ┌─ /path/to/my/schema.graphql:8794:16
         │
    8794 │   whatsChanged(date: DateTime!, flags: [WhatsChangedFilterFlagsEnumType], includeCppOrEmbargoedData: CppEmbargoedDataFilterEnumType): WhatsChangedType
         │                ^^^^^^^^^^^^^^^
    
    help[datetimeinputs]:
          ┌─ /path/to/my/schema.graphql:11536:3
          │
    11536 │   configurationLastUpdatedAt: DateTime
          │   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    

Next.js App Router (RSC) projects w/ open source code

Here are some example projects with open source code using Next.js App Router and React Server Components (RSC).

2024-10-24 additions

((Tiling) window managers) for macOS

Related: focus follows mouse for macOS

See also

Next.js Relay GraphQL Pokemon example

Here is a quick and dirty Pokemon TCG web UI using Next.js, Relay, and the TCGdex GraphQL API. Initially this was supposed to be a proof-of-concept of the Next.js rewrites feature, but that doesn't work with the Next.js static export for the GitHub Pages deploy, so I removed it.

Source code on GitHub here: https://github.com/saltycrane/next-relay-graphql-pokemon-example

Deployed to GitHub Pages here: https://saltycrane.github.io/next-relay-graphql-pokemon-example/

Uses

Doesn't use

  • Next.js App Router or React Server Components
  • Next.js Server Side Rendering
  • Next.js Image Optimization
  • React Transitions
  • Relay Fragments

Example Node.js Passport.js SAML app using OneLogin

I put an example Express.js, Passport.js OneLogin SAML SSO authentication app on github here: https://github.com/saltycrane/express-passport-saml-example. I used the following:

OneLogin configuration

  • create OneLogin developer account here: https://developers.onelogin.com/
  • for example, use the domain your-domain
  • at https://your-domain-dev.onelogin.com/admin2/apps select "Add App" > "SAML Custom Connector (Advanced)"
  • on "Configuration" tab, set the following 5 fields:
    • "Audience (EntityID)" [1]: your-example-app
    • "Recipient": your-example-app
    • "ACS (Consumer) URL Validator*": http://localhost:3000/login/sso/callback
    • "ACS (Consumer) URL*": http://localhost:3000/login/sso/callback
    • "SAML signature event" [1]: "Both"

[1] required as of node-saml v4.0.0

Set environment variables

  • copy .env.example to .env and change the following:
    • SSO_ENTRYPOINT: "SSO" tab > "SAML 2.0 Endpoint (HTTP)"
    • SSO_CERT: "SSO" tab > "X.509 Certificate" > "View Details" > "X.509 Certificate" with "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----" and newlines removed
    • SSO_COOKIE_SESSION_SECRET: generate or make up a secret string

Note: SSO_ISSUER should be "Recipient" on the "Configuration" tab and SSO_CALLBACK_URL should be "ACS (Consumer) URL*" on the "Configuration" tab.

Example .env

SSO_ENTRYPOINT='https://your-domain-dev.onelogin.com/trust/saml2/http-post/sso/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'

SSO_ISSUER='your-example-app'

SSO_CALLBACK_URL='http://localhost:3000/login/sso/callback'

SSO_CERT='XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX='

SSO_COOKIE_SESSION_SECRET='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'

Run app and test

Sequence of requests

  1. GET http://localhost:3000/login/sso
  2. GET https://your-domain-dev.onelogin.com/trust/saml2/http-post/sso/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
  3. POST http://localhost:3000/login/sso/callback
  4. GET http://localhost:3000/ (with user session cookie)

Troubleshooting

"Access Denied You do not have access to this application. Please contact your administrator."
  • ensure your user is added to the default role for the app in the OneLogin admin UI.
Error: Invalid signature
  • in the OneLogin admin UI, in the "Configuration" tab, ensure that "SAML signature element" is set to "Both"
  • alternatively, as a less secure option, add the following configuration to the passport-saml Strategy: wantAssertionsSigned: false.
  • node-saml changed in v4.0.0 to require all assertions be signed. See https://github.com/node-saml/node-saml/pull/177
Error: SAML assertion AudienceRestriction has no Audience value
  • in node-saml, audience defaults to the value of issuer
  • in the OneLogin admin UI, in the "Configuration" tab, ensure that "Audience (EntityID)" is the same as issuer. (In this example it is the value of "Recipient", your-example-app)

CSS Subgrid demo

This is a demo I did on CSS Subgrid.

It is a Next.js React project using CSS Modules. (As mentioned in the video, I wanted to use vanilla HTML and CSS for the demo, but this was just easier for me since I'm familiar with it.)

The code in the repo is slightly different than the code in the YouTube demo because I converted from using PostgreSQL to SQLite so I could include the data in the repo.

Aphrodite to CSS Modules codemod

I wanted to convert our React project from Aphrodite to CSS Modules. The biggest impetus was that Aphrodite isn't supported by the new Next.js v13 app directory feature, which I'm excited to try. I like styled-components, but my co-worker likes CSS Modules and it's hard to go wrong with CSS Modules. CSS Modules also has built-in support in Next.js and it looks pretty good in this graphic from the State of CSS survey.

To ease the conversion, I wrote a jscodeshift codemod to automate most of the process. The codemod is on github here: aphrodite-to-css-modules-codemod. An example is below.

The codemod worked well for my 200 Aphrodite files. I did spend time manually converting JS constants into CSS variables. I also manually handled CSS precedence issues since Aphrodite handles precedence more nicely than CSS. But overall I was pretty happy with the results. (It was certainly more successful than my attempt at a reactstrap-to-react-bootstrap codemod which I never used.)

Before

./example/src/MyComponent.tsx:

import { css, StyleSheet } from "aphrodite";
import classNames from "classnames";
import React from "react";

import { colors } from "./constants";
import { hexToRgbA } from "./utils";

export default function MyComponent() {
  const isSomething = true;
  const isSomethingElse = false;
  return (
    <div
      className={css(
        isSomethingElse ? myStyles.containerGrid : myStyles.containerFlex,
      )}
      style={{}}
    >
      <div className={css(myStyles.header, myStyles.content)}>header</div>
      <div className={classNames(css(myStyles.content), "another-class")}>
        <div>Lorem ipsum</div>
      </div>
      <span className={css(isSomething && myStyles.warning)}></span>
    </div>
  );
}

// comment I
export const myStyles = StyleSheet.create({
  containerGrid: {
    backgroundColor: "white",
    // comment 1
    /* comment 2 */ display: "grid" /* comment 4 */, // comment 5
    gridTemplate: `
      "sourceselect .       reviewbutton" auto
      "pagination   filters filters     " auto
      "rowcount     filters filters     " 20px
      / 2fr         1fr     2fr
    `,
    width: 200,
  },
  containerFlex: {
    display: "flex",
  },
  content: {
    lineHeight: 1.5,
  },
  header: {
    backgroundColor: "#ccc",
    color: hexToRgbA(colors.danger, 0.8),
    display: "inline-block",
    ":hover": {
      color: colors.primary,
      borderColor: `${colors.info} !important`,
    },
  },
  // comment a
  warning: {
    fontWeight: 700,
    color: colors.warning,
    opacity: 0,
  } /* comment b */, // comment c
});

After

./example/src/MyComponent.tsx:

import myStyles from "./MyComponent.module.css";
import classNames from "classnames";
import React from "react";

import { colors } from "./constants";
import { hexToRgbA } from "./utils";

export default function MyComponent() {
  const isSomething = true;
  const isSomethingElse = false;
  return (
    <div
      className={
        isSomethingElse ? myStyles.containerGrid : myStyles.containerFlex
      }
      style={{}}
    >
      <div
        className={
          // TODO: check CSS precedence
          classNames(myStyles.header, myStyles.content)
        }
      >
        header
      </div>
      <div className={classNames(myStyles.content, "another-class")}>
        <div>Lorem ipsum</div>
      </div>
      <span className={classNames(isSomething && myStyles.warning)}></span>
    </div>
  );
}

export { myStyles };

./example/src/MyComponent.module.css:

/* comment I */
.containerGrid {
  background-color: white;
  /* comment 1 */
  /* comment 2 */
  display: grid; /* comment 4 */ /* comment 5 */
  grid-template: 
  "sourceselect .       reviewbutton" auto
  "pagination   filters filters     " auto
  "rowcount     filters filters     " 20px
  / 2fr         1fr     2fr
;
  width: 200px;
}

.containerFlex {
  display: flex;
}

.content {
  line-height: 1.5;
}

.header {
  background-color: #ccc;
  color: var(--bs-danger-alpha80);
  display: inline-block;
}

.header:hover {
  color: var(--bs-primary);
  border-color: var(--bs-info) !important;
}

/* comment a */
.warning {
  font-weight: 700;
  color: var(--bs-warning);
  opacity: 0;
} /* comment b */ /* comment c */

JS context file

The expressions in the styles object (e.g. colors.danger, hexToRgbA(colors.danger, 0.8), etc.) were evaluated using the following "context" file.

./context.example.js:

const colors = {
  danger: "var(--bs-danger)",
  info: "var(--bs-info)",
  primary: "var(--bs-primary)",
  warning: "var(--bs-warning)",
};

function hexToRgbA(hex, alpha) {
  return hex.replace(/\)$/, `-alpha${alpha * 100})`);
}

Simple codemod example with jscodeshift

jscodeshift codemods allow refactoring JavaScript or TypeScript code by manipulating the abstract syntax tree.

This is an example showing how to rename variables named foo to bar.

Install jscodeshift

npm install -g jscodeshift

Create an example file to modify

  • create a folder

    mkdir my-project
    cd my-project
    
  • create an example file, my-file-to-modify.js

    const foo = 1;
    console.log(foo);
    

Create a transform

create a file my-transform.js

module.exports = function transformer(fileInfo, api) {
  return api
    .jscodeshift(fileInfo.source)
    .find(api.jscodeshift.Identifier)
    .forEach(function (path) {
      if (path.value.name === "foo") {
        api.jscodeshift(path).replaceWith(api.jscodeshift.identifier("bar"));
      }
    })
    .toSource();
};

Run it

jscodeshift -t my-transform.js ./my-file-to-modify.js

The file my-file-to-modify.js now contains:

const bar = 1;
console.log(bar);

Another example

This example removes the React JSX element <MyHeader /> and removes the MyHeader import. I'm not sure why, but it added some extra parentheses. Prettier cleaned this up for me, but if you have an improvement, let me know.

// removeMyHeader.js
module.exports = function transformer(file, api) {
  const jscodeshift = api.jscodeshift;

  const withoutElement = jscodeshift(file.source)
    .find(jscodeshift.JSXElement)
    .forEach(function (path) {
      if (path.value.openingElement.name.name === "MyHeader") {
        path.prune();
      }
    })
    .toSource();

  const withoutImport = jscodeshift(withoutElement)
    .find(jscodeshift.ImportDefaultSpecifier)
    .forEach(function (path) {
      if (path.value.local.name === "MyHeader") {
        path.parentPath.parentPath.prune();
      }
    })
    .toSource();

  return withoutImport;
};

Here is a command to run it for a React TypeScript codebase:

jscodeshift --parser=tsx --extensions=tsx -t ./removeMyHeader.js ./src

AST Explorer

AST Explorer is a very helpful tool to experiment and learn the API with code completion. Go to https://astexplorer.net/ and select "jscodeshift" under the "Transform" menu.

lodash error

Error: Cannot find module 'lodash'

When running jscodeshift, I got the above error so I ran npm install -g lodash and this got rid of the error for me.

Buildtime vs runtime environment variables with Next.js and Docker

For a Next.js app, buildtime environment variables are variables that are used when the next build command runs. Runtime variables are variables used when the next start command runs.

Below are ways to set buildtime and rutime environment variables with Docker and ways to use buildtime and runtime environment variables with Next.js. Note the Dockerfile is written for simplicity to illustrate the examples. For a more optimized Next.js Docker build see my Docker multi-stage CI example.

Methods for setting environment variables with Docker

MethodAvailable at buildtimeAvailable at runtimeValue passed to docker buildValue passed to docker run
ARG
ENV
ARG + docker build --build-arg
ARG + ENV + docker build --build-arg
docker run --env

Methods for using environment variables in Next.js

MethodSet atAvailable in Next.js client side rendered code (browser)Available in Next.js server side rendered codeAvailable in Node.jsNotes
.env files?both?process.env cannot be destructured or accessed with dynamic properties
NEXT_PUBLIC_ prefixed vars in .env filesbuildtimeprocess.env cannot be destructured or accessed with dynamic properties
env in next.config.jsbuildtimeprocess.env cannot be destructured or accessed with dynamic properties
publicRuntimeConfigruntimeRequires page uses SSR
serverRuntimeConfigruntime
process.envruntime

Assume this package.json for the examples below

{
  "scripts": {
    "build": "next build",
    "dev": "next",
    "start": "next start"
  },
  "dependencies": {
    "next": "^10.0.9",
    "react": "^17.0.2",
    "react-dom": "^17.0.2"
  }
}

Setting static environment variables for buildtime and runtime

Environment variables can be specified with the ENV instruction in a Dockerfile. Below MY_VAR will be available to both next build and next start. For more information see https://docs.docker.com/engine/reference/builder/#env

Dockerfile

FROM node:14-alpine

ENV MY_VAR=cake

WORKDIR /app
COPY . ./
RUN npm install
RUN npm run build
EXPOSE 3000
CMD ["npm", "start"]

Docker build

docker build -t mytag .

Docker run

docker run mytag

Setting dynamic buildtime environment variables

Dynamic environment variables can be passed to the docker build command using --build-arg and used in the Dockerfile with the ARG statement. Below MY_VAR is an environment variable available to next build.

Note that MY_VAR is not available to next start. ARG statements act like ENV statements in that they are treated like environment variables during docker build, but they are not persisted in the image. To make them available during docker run (and next start) set the value using ENV (see the next example).

For more information see https://docs.docker.com/engine/reference/builder/#arg

Dockerfile

FROM node:14-alpine

ARG MY_VAR

WORKDIR /app
COPY . ./
RUN npm install
RUN npm run build
EXPOSE 3000
CMD ["npm", "start"]

Docker build

docker build --build-arg MY_VAR=cake -t mytag .

Docker run

docker run mytag

Setting dynamic buildtime environment variables that are available at runtime also

The variable in the previous example, set using ARG, is not persisted in the Docker image so it is not available at runtime. To make it available at runtime, copy the value from ARG to ENV.

Dockerfile

FROM node:14-alpine

ARG MY_VAR
ENV MY_VAR=$MYVAR

WORKDIR /app
COPY . ./
RUN npm install
RUN npm run build
EXPOSE 3000
CMD ["npm", "start"]

Docker build

docker build --build-arg MY_VAR=cake -t mytag .

Docker run

docker run mytag

Setting dynamic runtime environment variables

Dynamic environment variables can be passed to docker run using the --env flag. These will not be available to next build but they will be available to next start. For more information see https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file

Dockerfile

FROM node:14-alpine
WORKDIR /app
COPY . ./
RUN npm install
RUN npm run build
EXPOSE 3000
CMD ["npm", "start"]

Docker build

docker build -t mytag .

Docker run

docker run --env MY_VAR=cake mytag

Using buildtime environment variables

To use buildtime environment variables in Next.js code, set them using env in next.config.js. Then access them via process.env in your app code. NOTE: process.env cannot be destructured or used with dynamic property access. Next.js does a string substituion at build time using the webpack DefinePlugin. For more information see https://nextjs.org/docs/api-reference/next.config.js/environment-variables

next.config.js

module.exports = {
  env: {
    MY_VAR: process.env.MY_VAR
  }
}

my-app-file.js

console.log(process.env.MY_VAR)

Using runtime environment variables (client-side or server-side)

To use runtime environment variables (client-side or server-side), set them using publicRuntimeConfig in next.config.js. Then access them using getConfig from next/config. NOTE: this only works for Next.js pages where server-side rendering (SSR) is used. i.e. the page must use getServerSideProps or getInitialProps. For more information see https://nextjs.org/docs/api-reference/next.config.js/runtime-configuration

next.config.js

module.exports = {
  publicRuntimeConfig: {
    MY_VAR: process.env.MY_VAR
  }
}

my-app-file.js

import getConfig from "next/config";
const { publicRuntimeConfig } = getConfig();
console.log(publicRuntimeConfig.MY_VAR)

Using runtime environment variables (server-side only)

To use runtime environment variables (server-side only), set them using serverRuntimeConfig in next.config.js. Then access them using getConfig from next/config. For more information see https://nextjs.org/docs/api-reference/next.config.js/runtime-configuration

NOTE: this applies to to files Next.js "builds". Server run files not processed by Next.js can use process.env to access environment variables. See below.

next.config.js

module.exports = {
  serverRuntimeConfig: {
    MY_VAR: process.env.MY_VAR
  }
}

my-app-file.js

import getConfig from "next/config";
const { serverRuntimeConfig } = getConfig();
console.log(serverRuntimeConfig.MY_VAR)

Using runtime environment variables server-side (not processed by Next.js)

For files not processed by Next.js (next build) (e.g. a server.js file run by node), runtime environment variables can be accessed on the server via process.env. NOTE: "runtime" variables here means variables used when the Node.js process runs. For more information see https://nodejs.org/docs/latest-v14.x/api/process.html#process_process_env

server.js

console.log(process.env.MY_VAR)

Next.js assetPrefix

If the Next.js assetPrefix is set in next.config.js using an environment variable, the environment variable should be set at buildtime for Next.js static pages but set at runtime for server rendered pages.

next.config.js

module.exports = {
  assetPrefix: process.env.MY_ASSET_PREFIX
}

Next.js GitLab CI/CD Docker multi-stage example

This describes an example Next.js project with a GitLab CI/CD pipeline that does the following:

  • installs npm packages and builds static assets
  • runs ESLint, TypeScript, and Cypress
  • builds a Docker image for deployment
  • pushes the Docker image to the GitLab Container Registry

This example prepares a Docker image for deployment but doesn't actually deploy it. See an example CI/CD pipeline that deploys to Amazon ECS.

To increase speed and reduce image size, it uses Docker multi-stage builds.

Dockerfile

The Dockerfile defines 3 stages:

  • the "builder" stage installs npm packages and builds static assets. It produces artifacts (/app and /root/.cache) that are used by the cypress and deploy stages. It is also used to build an image used to run ESLint and TypeScript.
  • the "cypress" stage uses a different base image from the "builder" stage and is used to run cypress tests
  • the final deploy stage copies the /app directory from the "builder" stage and sets NODE_ENV to "production" and exposes port 3000
ARG BASE_IMAGE=node:14.16-alpine

# ================================================================
# builder stage
# ================================================================
FROM $BASE_IMAGE as builder
ENV NODE_ENV=test
ENV NEXT_TELEMETRY_DISABLED=1
RUN apk add --no-cache bash git
WORKDIR /app
COPY ./package.json ./
COPY ./package-lock.json ./
RUN CI=true npm ci
COPY . ./
RUN NODE_ENV=production npm run build

# ================================================================
# cypress stage
# ================================================================
FROM cypress/base:14.16.0 as cypress
WORKDIR /app
# copy cypress from the builder image
COPY --from=builder /root/.cache /root/.cache/
COPY --from=builder /app ./
ENV NODE_ENV=test
ENV NEXT_TELEMETRY_DISABLED=1

# ================================================================
# final deploy stage
# ================================================================
FROM $BASE_IMAGE
WORKDIR /app
COPY --from=builder /app ./
ENV NODE_ENV=production
ENV NEXT_TELEMETRY_DISABLED=1
EXPOSE 3000
CMD ["npm", "start"]

.gitlab-ci.yml

  • 3 images are built: test, cypress, and deploy. The test image is used for running ESLint and TypeScript and is needed for cypress and deploy. The cypress image is used for running Cypress.
  • it uses Docker BuildKit to make caching easier. (With BuildKit, cached layers will be automatically pulled when needed. Without BuildKit, images used for caching need to be explicitly pulled.) For comparison, see this diff adding BuildKit. Note DOCKER_BUILDKIT is set to 1 to enable BuildKit.
variables:
  # enable docker buildkit. Used with `BUILDKIT_INLINE_CACHE=1` below
  DOCKER_BUILDKIT: 1
  DOCKER_TLS_CERTDIR: "/certs"
  IMAGE_TEST: $CI_REGISTRY_IMAGE/test:latest
  IMAGE_CYPRESS: $CI_REGISTRY_IMAGE/cypress:latest
  IMAGE_DEPLOY: $CI_REGISTRY_IMAGE/deploy:latest

stages:
  - build
  - misc
  - deploy

.base:
  image: docker:latest
  services:
    - docker:dind
  before_script:
    - docker --version
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"

build:builder:
  extends: .base
  stage: build
  script:
    - docker build --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from "$IMAGE_TEST" --target builder -t "$IMAGE_TEST" .
    - docker push "$IMAGE_TEST"

build:deployimage:
  extends: .base
  stage: misc
  needs: ["build:builder"]
  script:
    - docker build --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from "$IMAGE_DEPLOY" --cache-from "$IMAGE_TEST" --cache-from "$IMAGE_CYPRESS" -t "$IMAGE_DEPLOY" .
    - docker push "$IMAGE_DEPLOY"

test:cypress:
  extends: .base
  stage: misc
  needs: ["build:builder"]
  script:
    - docker build --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from "$IMAGE_CYPRESS" --cache-from "$IMAGE_TEST" --target cypress -t "$IMAGE_CYPRESS" .
    - docker push "$IMAGE_CYPRESS"
    - docker run "$IMAGE_CYPRESS" npm run cy:citest

test:eslint:
  extends: .base
  stage: misc
  needs: ["build:builder"]
  script:
    - docker run "$IMAGE_TEST" npm run eslint

test:typescript:
  extends: .base
  stage: misc
  needs: ["build:builder"]
  script:
    - docker run "$IMAGE_TEST" npm run tsc

deploy:
  stage: deploy
  needs: ["build:deployimage", "test:cypress", "test:eslint", "test:typescript"]
  script:
    - echo "deploy here"

.dockerignore

Adding the .git directory to .dockerignore prevented cache invalidation for the COPY . ./ command in the Dockerfile.

.git

References