SaltyCrane Blog — Notes on JavaScript and web development

How to run Docker in Docker on Mac

Docker in Docker can be used in GitLab CI/CD to build Docker images. This is how to run Docker in Docker on Mac.

  • create directory

    mkdir /tmp/my-project
    cd /tmp/my-project
    
  • create docker-compose.yml file:

    version: "3"
    services:
      docker-daemon:
        container_name: "my-docker-daemon"
        environment:
          DOCKER_TLS_CERTDIR: ""
        image: "docker:dind"
        networks:
          "my-network":
            aliases:
              - "docker"
        privileged: true
      docker-client:
        command: sh -c 'while [ 1 ]; do sleep 1000; done'
        container_name: "my-docker-client"
        depends_on:
          - "docker-daemon"
        environment:
          DOCKER_HOST: "tcp://docker:2375"
        image: "docker:latest"
        networks:
          "my-network": {}
    
    networks:
      "my-network":
        name: "my-network"
    
  • run the docker daemon and client containers

    docker-compose up -d
    
  • run a shell in the client container

    docker exec -it my-docker-client sh
    
  • run a docker command in the docker client container

    / # docker ps
    CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
    

References

Next.js Cypress GitLab CI example

This is an example Next.js project that runs a Cypress test in Docker using a GitLab CI pipeline. It also uses the GitLab Container Registry for caching purposes.

.gitlab-ci.yml

variables:
  DOCKER_TLS_CERTDIR: "/certs"

stages:
  - test

test-cypress:
  stage: test
  image: docker:latest
  services:
    - docker:dind
  variables:
    IMAGE_TAG: $CI_REGISTRY_IMAGE:latest
  script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker pull $IMAGE_TAG || true
    - docker build --cache-from $IMAGE_TAG -t $IMAGE_TAG .
    - docker push $IMAGE_TAG
    - docker run $IMAGE_TAG npm run cy:citest

Dockerfile

This uses the official Cypress Docker image (Dockerfile).

FROM cypress/base:14.16.0

WORKDIR /app
# run npm install before adding app code for better Docker caching
# https://semaphoreci.com/docs/docker/docker-layer-caching.html
COPY ./package.json /app
COPY ./package-lock.json /app
# CI=true suppresses Cypress progress log spam
RUN CI=true npm ci
COPY . /app
RUN npm run build

package.json

{
  "scripts": {
    "build": "next build",
    "cy:citest": "start-server-and-test start http://localhost:3000 cy:run",
    "cy:run": "cypress run",
    "dev": "next",
    "start": "next start"
  },
  "dependencies": {
    "next": "^10.0.9",
    "react": "^17.0.2",
    "react-dom": "^17.0.2"
  },
  "devDependencies": {
    "cypress": "^6.8.0",
    "start-server-and-test": "^1.12.1"
  }
}

cypress/integration/index_spec.js

describe("index page", () => {
  it("loads successfully", () => {
    cy.visit("http://localhost:3000");
    cy.contains("Index");
  });
});

References

Example Next.js GitLab CI/CD Amazon ECR and ECS deploy pipeline

I've created an example Next.js project with a GitLab CI/CD pipeline that builds a Docker image, pushes it to Amazon ECR, deploys it to an Amazon ECS Fargate cluster, and uploads static assets (JS, CSS, etc.) to Amazon S3. The example GitLab repo is here: https://gitlab.com/saltycrane/next-aws-ecr-ecs-gitlab-ci-cd-example

Interesting files

Here are the interesting parts of some of the files. See the full source code in the GitLab repo.

  • .gitlab-ci.yml (view at gitlab)

    • the variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and ECR_HOST are set in the GitLab UI under "Settings" > "CI/CD" > "Variables"
    • this uses the saltycrane/aws-cli-and-docker Docker image which provides the aws v2 command line tools and docker in a single image. It is based on amazon/aws-cli and installs bc, curl, docker, jq, and tar. This idea is from Valentin's tutorial.
    variables:
      DOCKER_HOST: tcp://docker:2375
      DOCKER_TLS_CERTDIR: ""
      AWS_DEFAULT_REGION: "us-east-1"
      CI_APPLICATION_REPOSITORY: "$ECR_HOST/next-aws-ecr-ecs-gitlab-ci-cd-example"
      CI_APPLICATION_TAG: "$CI_PIPELINE_IID"
      CI_AWS_S3_BUCKET: "next-aws-ecr-ecs-gitlab-ci-cd-example"
      CI_AWS_ECS_CLUSTER: "next-aws-ecr-ecs-gitlab-ci-cd-example"
      CI_AWS_ECS_SERVICE: "next-aws-ecr-ecs-gitlab-ci-cd-example"
      CI_AWS_ECS_TASK_DEFINITION: "next-aws-ecr-ecs-gitlab-ci-cd-example"
      NEXT_JS_ASSET_URL: "https://$CI_AWS_S3_BUCKET.s3.amazonaws.com"
    
    stages:
      - build
      - deploy
    
    build:
      stage: build
      image: saltycrane/aws-cli-and-docker
      services:
        - docker:dind
      script:
        - ./bin/build-and-push-image-to-ecr
        - ./bin/upload-assets-to-s3
    
    deploy:
      stage: deploy
      image: saltycrane/aws-cli-and-docker
      services:
        - docker:dind
      script:
        - ./bin/ecs update-task-definition
    
  • Dockerfile (view at gitlab)

    The value of NEXT_JS_ASSET_URL is passed in using the --build-arg option of the docker build command run in bin/build-and-push-image-to-ecr. It is used like an environment variable in the RUN npm run build command below. In this project it is assigned to assetPrefix in next.config.js.

    FROM node:14.16-alpine
    ARG NEXT_JS_ASSET_URL
    ENV NODE_ENV=production
    WORKDIR /app
    COPY ./package.json ./
    COPY ./package-lock.json ./
    RUN npm ci
    COPY . ./
    RUN npm run build
    EXPOSE 3000
    CMD ["npm", "start"]
    
  • bin/build-and-push-image-to-ecr (view at gitlab)

    # log in to the amazon ecr docker registry
    aws ecr get-login-password | docker login --username AWS --password-stdin "$ECR_HOST"
    
    # build docker image
    docker pull "$CI_APPLICATION_REPOSITORY:latest" || true
    docker build --build-arg "NEXT_JS_ASSET_URL=$NEXT_JS_ASSET_URL" --cache-from "$CI_APPLICATION_REPOSITORY:latest" -t "$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG"     -t "$CI_APPLICATION_REPOSITORY:latest" .
    
    # push image to amazon ecr
    docker push "$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG"
    docker push "$CI_APPLICATION_REPOSITORY:latest"
    
  • bin/upload-assets-to-s3 (view at gitlab)

    LOCAL_ASSET_PATH=/tmp/upload-assets
    
    mkdir $LOCAL_ASSET_PATH
    
    # copy the generated assets out of the docker image
    docker run --rm --entrypoint tar "$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG" cf - .next | tar xf - -C $LOCAL_ASSET_PATH
    
    # rename .next to _next
    mv "$LOCAL_ASSET_PATH/.next" "$LOCAL_ASSET_PATH/_next"
    
    # remove directories that should not be uploaded to S3
    rm -rf "$LOCAL_ASSET_PATH/_next/cache"
    rm -rf "$LOCAL_ASSET_PATH/_next/server"
    
    # gzip files
    find $LOCAL_ASSET_PATH -regex ".*\.\(css\|svg\|js\)$" -exec gzip {} \;
    
    # strip .gz extension off of gzipped files
    find $LOCAL_ASSET_PATH -name "*.gz" -exec sh -c 'mv $1 `echo $1 | sed "s/.gz$//"`' - {} \;
    
    # upload gzipped js, css, and svg assets
    aws s3 sync --no-progress $LOCAL_ASSET_PATH "s3://$CI_AWS_S3_BUCKET" --cache-control max-age=31536000 --content-encoding gzip --exclude "*" --include "*.js"     --include "*.css" --include "*.svg"
    
    # upload non-gzipped assets
    aws s3 sync --no-progress $LOCAL_ASSET_PATH "s3://$CI_AWS_S3_BUCKET" --cache-control max-age=31536000 --exclude "*.js" --exclude "*.css" --exclude "*.svg" --exclude "*.map"
    
  • bin/ecs (view full file) (This file was copied from the gitlab-org repo)

    #!/bin/bash -e
    
    update_task_definition() {
      local -A register_task_def_args=( \
        ['task-role-arn']='taskRoleArn' \
        ['execution-role-arn']='executionRoleArn' \
        ['network-mode']='networkMode' \
        ['cpu']='cpu' \
        ['memory']='memory' \
        ['pid-mode']='pidMode' \
        ['ipc-mode']='ipcMode' \
        ['proxy-configuration']='proxyConfiguration' \
        ['volumes']='volumes' \
        ['placement-constraints']='placementConstraints' \
        ['requires-compatibilities']='requiresCompatibilities' \
        ['inference-accelerators']='inferenceAccelerators' \
      )
    
      image_repository=$CI_APPLICATION_REPOSITORY
      image_tag=$CI_APPLICATION_TAG
      new_image_name="${image_repository}:${image_tag}"
    
      register_task_definition_from_remote
    
      new_task_definition=$(aws ecs register-task-definition "${args[@]}")
      new_task_revision=$(read_task "$new_task_definition" 'revision')
      new_task_definition_family=$(read_task "$new_task_definition" 'family')
    
      # Making sure that we at least have one running task (even if desiredCount gets updated again with new task definition below)
      service_task_count=$(aws ecs describe-services --cluster "$CI_AWS_ECS_CLUSTER" --services "$CI_AWS_ECS_SERVICE" --query "services[0].desiredCount")
    
      if [[ $service_task_count == 0 ]]; then
        aws ecs update-service --cluster "$CI_AWS_ECS_CLUSTER" --service "$CI_AWS_ECS_SERVICE" --desired-count 1
      fi
    
      # Update ECS service with newly created task defintion revision.
      aws ecs update-service \
                --cluster "$CI_AWS_ECS_CLUSTER" \
                --service "$CI_AWS_ECS_SERVICE" \
                --task-definition "$new_task_definition_family":"$new_task_revision"
    
      return 0
    }
    
    read_task() {
      val=$(echo "$1" | jq -r ".taskDefinition.$2")
      if [ "$val" == "null" ];then
        val=$(echo "$1" | jq -r ".$2")
      fi
      if [ "$val" != "null" ];then
        echo -n "${val}"
      fi
    }
    
    register_task_definition_from_remote() {
      task=$(aws ecs describe-task-definition --task-definition "$CI_AWS_ECS_TASK_DEFINITION")
      current_container_definitions=$(read_task "$task" 'containerDefinitions')
      new_container_definitions=$(echo "$current_container_definitions" | jq --arg val "$new_image_name" '.[0].image = $val')
      args+=("--family" "${CI_AWS_ECS_TASK_DEFINITION}")
      args+=("--container-definitions" "${new_container_definitions}")
      for option in "${!register_task_def_args[@]}"; do
        value=$(read_task "$task" "${register_task_def_args[$option]}")
        if [ -n "$value" ];then
          args+=("--${option}" "${value}")
        fi
      done
    }
    
    update_task_definition
    

Usage - set up AWS resources

Below are the minimum steps I needed to create the required AWS services for my example. I use the AWS region "us-east-1". For info about creating some of these services via the command line, see my Amazon ECS notes.

Create an ECR repository

Create an ECS Fargate cluster

Create an ECS task definition

  • click "Create new Task Definition" here: https://console.aws.amazon.com/ecs/home?region=us-east-1#/taskDefinitions
  • select "FARGATE" and click "Next step"
  • configure task
    • for "Task Definition Name" enter "next-aws-ecr-ecs-gitlab-ci-cd-example"
    • for "Task Role" select "None"
    • for "Task execution role" select "Create new role"
    • for "Task memory" select "0.5GB"
    • for "Task CPU" select "0.25 vCPU"
    • click "Add container"
      • for "Container Name" enter "next-aws-ecr-ecs-gitlab-ci-cd-example"
      • for "Image" enter "asdf" (this will be updated by the gitlab ci/cd job)
      • leave "Private repository authentication" unchecked
      • for "Port mappings" enter "3000"
      • click "Add"
    • click "Create"

Create an ECS service

  • click "Create" here: https://us-east-1.console.aws.amazon.com/ecs/home?region=us-east-1#/clusters/next-aws-ecr-ecs-gitlab-ci-cd-example/services
  • configure service
    • for "Launch type" select "FARGATE"
    • for "Task Definition" enter "next-aws-ecr-ecs-gitlab-ci-cd-example"
    • for "Cluster" select "next-aws-ecr-ecs-gitlab-ci-cd-example"
    • for "Service name" enter "next-aws-ecr-ecs-gitlab-ci-cd-example"
    • for "Number of tasks" enter 1
    • for "Deployment type" select "Rolling update"
    • click "Next step"
  • configure network
    • select the appropriate "Cluster VPC" and two "Subnets"
    • click "Next step"
  • set Auto Scaling
    • click "Next step"
  • review
    • click "Create Service"

Open port 3000

Create a S3 bucket

  • click "Create bucket" here: https://s3.console.aws.amazon.com/s3/home?region=us-east-1
  • for "Bucket name" enter "next-aws-ecr-ecs-gitlab-ci-cd-example"
  • uncheck "Block all public access"
  • check the "I acknowledge that the current settings might result in this bucket and the objects within becoming public" checkbox
  • click "Create bucket"

Update permissions for S3 bucket

Create an IAM user

  • create an IAM user. The user must have at least ECR, ECS, and S3 permissions.
  • take note of the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

Usage - run the CI/CD pipeline

Fork the example gitlab repo and configure CI/CD variables

  • fork https://gitlab.com/saltycrane/next-aws-ecr-ecs-gitlab-ci-cd-example
  • go to "Settings" > "CI/CD" > "Variables" and add the following variables. You can choose to "protect" and "mask" all of them.
    • AWS_ACCESS_KEY_ID
    • AWS_SECRET_ACCESS_KEY
    • ECR_HOST (This is the part of the ECR repository URI before the /. It looks something like XXXXXXXXXXXX.dkr.ecr.us-east-1.amazonaws.com)

Edit variables in .gitlab-ci.yml

If you used names other than "next-aws-ecr-ecs-gitlab-ci-cd-example", edit the variables in .gitlab-ci.yml.

Test it

References (build/push)

References (deploy)

GitLab CI/CD hello world examples

These are my notes for creating "hello world" and "docker hello world" GitLab CI/CD pipelines.

GitLab CI/CD hello world (gitlab repo)

  • create git repo, push it to gitlab, and set origin. Replace saltycrane with your username.

    mkdir gitlab-ci-cd-hello-world
    cd gitlab-ci-cd-hello-world
    git init
    touch .gitignore
    git add .
    git commit -m 'first commit'
    git push --set-upstream [email protected]:saltycrane/gitlab-ci-cd-hello-world.git --all
    git remote add origin [email protected]:saltycrane/gitlab-ci-cd-hello-world.git
    
  • add a .gitlab-ci.yml file

    build-hello:
      script:
        - echo "hello world"
    
  • commit and push

    git add .
    git commit -m 'add ci/cd config'
    git push origin
    
  • see the pipeline run (replace saltycrane with your username): https://gitlab.com/saltycrane/gitlab-ci-cd-hello-world/-/pipelines

GitLab CI/CD Docker hello world (gitlab repo)

  • create git repo, push it to gitlab, and set origin. Replace saltycrane with your username.

    mkdir gitlab-ci-cd-docker-hello-world
    cd gitlab-ci-cd-docker-hello-world
    git init
    touch .gitignore
    git add .
    git commit -m 'first commit'
    git push --set-upstream [email protected]:saltycrane/gitlab-ci-cd-docker-hello-world.git --all
    git remote add origin [email protected]:saltycrane/gitlab-ci-cd-docker-hello-world.git
    
  • add a Dockerfile file

    FROM alpine
    RUN echo "hello"
    
  • add a .gitlab-ci.yml file

    variables:
      DOCKER_TLS_CERTDIR: "/certs"
    
    build-docker:
      image: docker:latest
      services:
        - docker:dind
      script:
        - docker build -t hello .
    
  • commit and push

    git add .
    git commit -m 'add Dockerfile and ci/cd config'
    git push origin
    
  • see the pipeline run (replace saltycrane with your username): https://gitlab.com/saltycrane/gitlab-ci-cd-docker-hello-world/-/pipelines

Amazon ECS notes

These are my notes for creating a Docker image, pushing it to Amazon ECR (Elastic Container Registry), and deploying it to Amazon ECS (Elastic Container Service) using AWS Fargate (serverless for containers) using command line tools.

Create docker image on local machine

  • install docker (macOS)

    brew install homebrew/cask/docker
    
  • create directory

    mkdir /tmp/my-project
    cd /tmp/my-project
    
  • create /tmp/my-project/Dockerfile:

    FROM python:3.9-alpine3.13
    WORKDIR /app
    RUN echo 'Hello' > ./index.html
    EXPOSE 80
    CMD ["python", "-m", "http.server", "80"]
    
  • create Docker image

    docker build -t my-image .
    
  • test running the Docker image locally

    docker run -p 80:80 my-image
    
  • go to http://localhost in the browser and see the text "Hello"

Install and configure AWS command line tools

  • install AWS command line tools

    brew install awscli
    
  • create an IAM user

  • run aws configure and enter:

    • AWS Access Key ID
    • AWS Secret Access Key

    This creates the file ~/.aws/credentials

Create ECR repository and push image to it

From https://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-basics.html#use-ecr

  • create an Amazon ECR repository using aws ecr create-repository

    aws ecr create-repository --repository-name my-repository --region us-east-1
    

    output:

    {
        "repository": {
            "repositoryArn": "arn:aws:ecr:us-east-1:AAAAAAAAAAAA:repository/my-repository",
            "registryId": "AAAAAAAAAAAA",
            "repositoryName": "my-repository",
            "repositoryUri": "AAAAAAAAAAAA.dkr.ecr.us-east-1.amazonaws.com/my-repository",
            "createdAt": "2021-03-17T10:48:18-07:00",
            "imageTagMutability": "MUTABLE",
            "imageScanningConfiguration": {
                "scanOnPush": false
            },
            "encryptionConfiguration": {
                "encryptionType": "AES256"
            }
        }
    }
    

    Take note of the "registryId" and use it in place of "AAAAAAAAAAAA" below.

  • tag the docker image with the repositoryUri

    docker tag my-image AAAAAAAAAAAA.dkr.ecr.us-east-1.amazonaws.com/my-repository
    
  • log in to the Amazon ECR registry using aws ecr get-login-password

    aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin AAAAAAAAAAAA.dkr.ecr.us-east-1.amazonaws.com
    
  • push the docker image to the Amazon ECR repository

    docker push AAAAAAAAAAAA.dkr.ecr.us-east-1.amazonaws.com/my-repository
    
  • see the image in AWS console https://console.aws.amazon.com/ecr/repositories?region=us-east-1

Install ECS command line tools

  • install ecs-cli. Note there is ecs-cli in addition to aws ecs tools. The reason is probably similar to why some services are named "Amazon Service" and some are named "AWS Service". (It seems like ecs-cli provides higher level commands.)
    brew install amazon-ecs-cli
    

Create Amazon ECS Fargate cluster

From https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-cli-tutorial-fargate.html

  • create a cluster using ecs-cli up
    ecs-cli up --cluster my-cluster --launch-type FARGATE --region us-east-1
    
    output:
    INFO[0001] Created cluster                               cluster=my-cluster region=us-east-1
    INFO[0002] Waiting for your cluster resources to be created...
    INFO[0002] Cloudformation stack status                   stackStatus=CREATE_IN_PROGRESS
    VPC created: vpc-BBBBBBBBBBBBBBBBB
    Subnet created: subnet-CCCCCCCCCCCCCCCCC
    Subnet created: subnet-DDDDDDDDDDDDDDDDD
    Cluster creation succeeded.
    
    Take note of the VPC (virtual private cloud), and two subnet IDs to use later. See the cluster in the AWS console UI: https://console.aws.amazon.com/ecs/home?region=us-east-1#/clusters

Gather parameters required to deploy to ECS cluster

From https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-cli-tutorial-fargate.html

Create task execution IAM role
  • create a file /tmp/my-project/task-execution-assume-role.json

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "",
          "Effect": "Allow",
          "Principal": {
            "Service": "ecs-tasks.amazonaws.com"
          },
          "Action": "sts:AssumeRole"
        }
      ]
    }
    
  • create the task execution role using aws iam create-role

    aws iam create-role --role-name my-task-execution-role --assume-role-policy-document file:///tmp/my-project/task-execution-assume-role.json --region us-east-1
    
  • attach the task execution role policy using aws iam attach-role-policy

    aws iam attach-role-policy --role-name my-task-execution-role --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy --region us-east-1
    
Get security group ID
  • get the default security group ID for the virtual private cloud (VPC) created when creating the ECS cluster using aws ec2 describe-security-groups. Replace "vpc-BBBBBBBBBBBBBBBBB" with your VPC ID

    aws ec2 describe-security-groups --filters Name=vpc-id,Values=vpc-BBBBBBBBBBBBBBBBB --region us-east-1
    

    output:

    {
        "SecurityGroups": [
            {
                "Description": "default VPC security group",
                "GroupName": "default",
                "IpPermissions": [
                    {
                        "IpProtocol": "-1",
                        "IpRanges": [],
                        "Ipv6Ranges": [],
                        "PrefixListIds": [],
                        "UserIdGroupPairs": [
                            {
                                "GroupId": "sg-EEEEEEEEEEEEEEEEE",
                                "UserId": "AAAAAAAAAAAA"
                            }
                        ]
                    }
                ],
                "OwnerId": "AAAAAAAAAAAA",
                "GroupId": "sg-EEEEEEEEEEEEEEEEE",
                "IpPermissionsEgress": [
                    {
                        "IpProtocol": "-1",
                        "IpRanges": [
                            {
                                "CidrIp": "0.0.0.0/0"
                            }
                        ],
                        "Ipv6Ranges": [],
                        "PrefixListIds": [],
                        "UserIdGroupPairs": []
                    }
                ],
                "VpcId": "vpc-BBBBBBBBBBBBBBBBB"
            }
        ]
    }
    

    Take note of the "GroupId" to be used later

Deploy to Amazon ECS cluster

From https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-cli-tutorial-fargate.html

  • create /tmp/my-project/ecs-params.yml replacing "subnet-CCCCCCCCCCCCCCCCC", "subnet-DDDDDDDDDDDDDDDDD", and "sg-EEEEEEEEEEEEEEEEE" with appropriate IDs from above. ECS Parameters docs

    version: 1
    task_definition:
      task_execution_role: my-task-execution-role
      ecs_network_mode: awsvpc
      task_size:
        mem_limit: 0.5GB
        cpu_limit: 256
    run_params:
      network_configuration:
        awsvpc_configuration:
          subnets:
            - "subnet-CCCCCCCCCCCCCCCCC"
            - "subnet-DDDDDDDDDDDDDDDDD"
          security_groups:
            - "sg-EEEEEEEEEEEEEEEEE"
          assign_public_ip: ENABLED
    
  • create /tmp/my-project/docker-compose.yml replacing AAAAAAAAAAAA with the registryId:

    version: '3'
    services:
      web:
        image: 'AAAAAAAAAAAA.dkr.ecr.us-east-1.amazonaws.com/my-repository'
        ports:
          - '80:80'
    
  • deploy to the ECS cluster using ecs-cli compose service up. This creates a task definition and service. This uses the docker-compose.yml file in the current directory.

    ecs-cli compose --cluster my-cluster --project-name my-project --ecs-params ecs-params.yml --region us-east-1 service up --launch-type FARGATE
    

    see the service in the web UI: https://console.aws.amazon.com/ecs/home?region=us-east-1#/clusters/my-cluster/services

Hit the server in the browser

  • configure security group to allow inbound access on port 80 using aws ec2 authorize-security-group-ingress

    aws ec2 authorize-security-group-ingress --group-id sg-EEEEEEEEEEEEEEEEE --protocol tcp --port 80 --cidr 0.0.0.0/0 --region us-east-1
    
  • get the IP address using ecs-cli compose service ps

    ecs-cli compose --cluster my-cluster --project-name my-project --region us-east-1 service ps
    

    output:

    Name                                             State    Ports                    TaskDefinition  Health
    my-cluster/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/web  RUNNING  FF.FF.FF.FF:80->80/tcp  my-project:1    UNKNOWN
    

    Take note of the IP address under "Ports"

  • visit in the browser: http://FF.FF.FF.FF replacing "FF.FF.FF.FF" with your IP address

Destroy

  • delete the ECS service using ecs-cli compose service down

    ecs-cli compose --cluster my-cluster --project-name my-project --region us-east-1 service down
    
  • delete the ECS cluster using ecs-cli down

    ecs-cli down --force --cluster my-cluster --region us-east-1
    
  • delete the ECR repository using aws ecr delete-repository

    aws ecr delete-repository --repository-name my-repository --region us-east-1 --force
    

References

Examples of CSS text-orientation and writing-mode

Shared CSS

CSS
.ex { border: 1px solid #ccc; height: 120px; padding: 5px; width: 120px; }

Examples of permutations of writing-mode and text-orientation

horizontal-tb, mixed
Some example text
HTML
<div class="ex" id="example-0a"> Some example text </div>
CSS
#example-0a { writing-mode: horizontal-tb; text-orientation: mixed; }
horizontal-tb, upright
Some example text
HTML
<div class="ex" id="example-0b"> Some example text </div>
CSS
#example-0b { writing-mode: horizontal-tb; text-orientation: upright; }
vertical-rl, mixed
Some example text
HTML
<div class="ex" id="example-1a"> Some example text </div>
CSS
#example-1a { writing-mode: vertical-rl; text-orientation: mixed; }
vertical-rl, upright
Some example text
HTML
<div class="ex" id="example-1b"> Some example text </div>
CSS
#example-1b { writing-mode: vertical-rl; text-orientation: upright; }
vertical-lr, mixed
Some example text
HTML
<div class="ex" id="example-2a"> Some example text </div>
CSS
#example-2a { writing-mode: vertical-lr; text-orientation: mixed; }
vertical-lr, upright
Some example text
HTML
<div class="ex" id="example-2b"> Some example text </div>
CSS
#example-2b { writing-mode: vertical-lr; text-orientation: upright; }

How to download pull request metadata using the GitHub GraphQL API

This is a TypeScript Node.js script to download GitHub pull request information (title, body, comments, etc.) using the GitHub GraphQL API. The data is saved in a JSON file.

The GitHub repo is here: download-github-prs.

Create a GitHub personal access token

Create a GitHub personal access token as described here (no checkboxes need to be selected for public repos): https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/creating-a-personal-access-token This is used to access the GitHub GraphQL API.

Intall libraries

  • Create a directory and cd into it
    $ mkdir -p /tmp/my-prs
    $ cd /tmp/my-prs 
    
  • Create package.json file:
    {
      "scripts": {
        "download": "ts-node index.ts"
      },
      "dependencies": {
        "node-fetch": "^2.6.1"
      },
      "devDependencies": {
        "@types/node-fetch": "^2.5.7",
        "ts-node": "^9.0.0",
        "typescript": "^4.1.2"
      }
    }
    
  • Install
    $ npm install 
    
  • Create script

    Creat a file, /tmp/my-prs/index.ts, replacing XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX with the GitHub personal access token described above.

    import * as fs from "fs";
    import fetch from "node-fetch";
    
    const GITHUB_TOKEN = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX";
    const GITHUB_GRAPHQL_API_URL = "https://api.github.com/graphql";
    const OUTPUT_DIR = "/tmp/my-prs";
    const REPO_OWNER = "facebookexperimental";
    const REPO_NAME = "Recoil";
    
    fetchPullRequest(1);
    
    /**
     * fetchPullRequest
     */
    async function fetchPullRequest(prNumber: number) {
      const reactionFragment = `
        content
        user {
          login
        }
      `;
      const userContentEditFragment = `
        createdAt
        deletedAt
        deletedBy {
          login
        }
        diff
        editedAt
        editor {
          login
        }
        updatedAt
      `;
      const commentFragment = `
        author {
          login
        }
        body
        createdAt
        reactions(first: 100) {
          nodes {
            ${reactionFragment}
          }
        }
        userContentEdits(first: 100) {
          nodes {
            ${userContentEditFragment}
          }
        }
      `;
      const query = `
      query {
        repository(owner: "${REPO_OWNER}", name: "${REPO_NAME}") {
          nameWithOwner
          pullRequest(number: ${prNumber}) {
            author { login }
            baseRefName
            baseRefOid
            body
            closedAt
            comments(first: 100) {
              nodes {
                ${commentFragment}
              }
            }
            commits(first: 250) {
              nodes {
                commit {
                  oid
                }
              }
            }
            createdAt
            files(first: 100) {
              nodes { path }
            }
            headRefName
            headRefOid
            mergeCommit { oid }
            merged
            mergedAt
            mergedBy { login }
            number
            publishedAt
            reactions(first: 10) {
              nodes {
                ${reactionFragment}
              }
            }
            reviews(first: 10) {
              nodes {
                author { login }
                body
                comments(first: 10) {
                  nodes {
                    ${commentFragment}
                  }
                }
                commit {
                  oid
                }
                createdAt
                editor { login }
                publishedAt
                reactions(first: 10) {
                  nodes {
                    ${reactionFragment}
                  }
                }
                resourcePath
                submittedAt
                updatedAt
                userContentEdits(first: 10) {
                  nodes {
                    ${userContentEditFragment}
                  }
                }
              }
            }
            state
            title
            updatedAt
            userContentEdits(first: 10) {
              nodes {
                ${userContentEditFragment}
              }
            }
          }
        }
      }
      `;
    
      // make graphql query and strigify the response
      const resp = await fetchQuery(query);
      const respStr = JSON.stringify(resp, null, 2);
    
      // save json file
      const filepath = [
        `${OUTPUT_DIR}/`,
        `${REPO_NAME}-pr-${String(prNumber).padStart(4, "0")}.json`,
      ].join("");
      console.log(`Saving ${filepath}...`);
      fs.writeFileSync(filepath, respStr);
    }
    
    /**
     * fetchQuery
     */
    function fetchQuery(query: string, variables: Record<string, any> = {}) {
      return fetch(GITHUB_GRAPHQL_API_URL, {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
          Authorization: `bearer ${GITHUB_TOKEN}`,
        },
        body: JSON.stringify({
          query,
          variables,
        }),
      }).then((response) => {
        return response.json();
      });
    }
    

    Run it

    $ npm run download 
    

    Output

    This produces the following JSON file

    $ cat /tmp/my-prs/Recoil-pr-0001.json
    {
      "data": {
        "repository": {
          "nameWithOwner": "facebookexperimental/Recoil",
          "pullRequest": {
            "author": {
              "login": "facebook-github-bot"
            },
            "baseRefName": "master",
            "baseRefOid": "40e870caadc159a87e81be291ff641410ab32e8f",
            "body": "This is pull request was created automatically because we noticed your project was missing a Contributing file.\n\nCONTRIBUTING files explain how a developer can contribute to the project - which you should actively encourage.\n\nThis PR was crafted with love by Facebook's Open Source Team.",
            "closedAt": "2020-05-13T04:12:15Z",
            "comments": {
              "nodes": [
                {
                  "author": {
                    "login": "davidmccabe"
                  },
                  "body": "Already added this manually.",
                  "createdAt": "2020-05-13T04:12:15Z",
                  "reactions": {
                    "nodes": []
                  },
                  "userContentEdits": {
                    "nodes": []
                  }
                }
              ]
            },
            "commits": {
              "nodes": [
                {
                  "commit": {
                    "oid": "96f91679540362fa96a6c92611a8ef5621447b42"
                  }
                }
              ]
            },
            "createdAt": "2020-05-06T22:31:01Z",
            "files": {
              "nodes": [
                {
                  "path": "CONTRIBUTING.md"
                }
              ]
            },
            "headRefName": "automated_fixup_contributing_file_exists",
            "headRefOid": "96f91679540362fa96a6c92611a8ef5621447b42",
            "mergeCommit": null,
            "merged": false,
            "mergedAt": null,
            "mergedBy": null,
            "number": 1,
            "publishedAt": "2020-05-06T22:31:01Z",
            "reactions": {
              "nodes": []
            },
            "reviews": {
              "nodes": []
            },
            "state": "CLOSED",
            "title": "Adding Contributing file",
            "updatedAt": "2020-10-07T20:23:05Z",
            "userContentEdits": {
              "nodes": []
            }
          }
        }
      }
    }
    

Notes on Fabric 2 and Python 3

Fabric 2 is a Python package used for running commands on remote machines via SSH. Fabric 2 supports Python 3 and is a rewrite of the Fabric I used years ago. Here are my notes on using Fabric 2 and Python 3.

Set up SSH config and SSH agent

  • Create or edit your ~/.ssh/config file to contain your remote host parameters
    Host myhost
        User myusername
        HostName myhost.com
        IdentityFile ~/.ssh/id_rsa
    
  • Add your private key to your SSH agent
    $ ssh-add ~/.ssh/id_rsa
    

Create a project, create a virtualenv, and install fabric2

$ mkdir -p /tmp/my-project
$ cd /tmp/my-project
$ python3 -m venv venv
$ source venv/bin/activate
$ pip install fabric2

Create a fabfile.py script

Create a file /tmp/my-project/fabfile.py with the following contents. Note: "myhost" is the same name used in ~/.ssh/config described above.

from fabric2 import task

hosts = ["myhost"]

@task(hosts=hosts)
def mytask(c):
    print("Starting mytask...")
    with c.cd("/var"):
        c.run("ls -l")
    print("Done.")

Run the fabric script

In /tmp/my-project, with the virtualenv activated, run the fabric task to list the contents of /var on the remote host.

$ fab2 mytask 

Output:

Starting mytask...

total 48
drwxr-xr-x  2 root root   4096 backups
drwxr-xr-x  9 root root   4096 cache
drwxrwxrwt  2 root root   4096 crash
drwxr-xr-x 38 root root   4096 lib
drwxrwsr-x  2 root root   4096 local
drwxrwxrwt  2 root root   4096 lock
drwxrwxr-x 14 root root   4096 log
drwxrwsr-x  2 root root   4096 mail
drwxr-xr-x  2 root root   4096 opt
drwxr-xr-x  5 root root   4096 spool
drwxrwxrwt  2 root root   4096 tmp
drwxr-xr-x  3 root root   4096 www

Done.

See also / References

How height is set in CSS

Here are some CSS experiments to test when elements adjust to the height of their container and when they adjust to the height of their content. The script to generate the experiments is on github and the results are shown below. See also my companion page on CSS width.

By default, elements adjust to height of their container when they are
By default, elements adjust to the height of their content when they are
Some elements can be made to adjust to the height of their container by
Some elements can be made to adjust to the height of their content by
Some miscellaneous cases:
  • elements in flex column containers with flex-grow set expand to the height of their content for tall content
  • for elements in grid containers, setting height to 100% does NOT limit their height by the height of their container
  • for elements in grid containers, setting overflow limits an element's height to the height of its container for tall content
  • for elements in grid containers, setting overflow has no effect when align-items is set
  • for elements in table cell containers, setting height to 100% does NOT expand an element's height to the height of the table cell container if the height of the table cell container is not explicitly set

See also

Block containers

Block container - elements in block containers adjust to the height of their content by default
#container-1a
#example-1a
#content-1a
HTML
<div id="container-1a"> <div id="example-1a"> <div id="content-1a" /> </div> </div>
CSS
#container-1a { display: block; /* default */ height: 400px; } #example-1a {} #content-1a { height: 80px; }
Block container, tall content - elements in block containers adjust to the height of their content even when the content is taller than the container
#container-1b
#example-1b
#content-1b
HTML
<div id="container-1b"> <div id="example-1b"> <div id="content-1b" /> </div> </div>
CSS
#container-1b { display: block; /* default */ height: 400px; } #example-1b {} #content-1b { height: 420px; }
Block container, height 100% - elements in block containers expand to the height of their container if height is set to 100%
#container-1c
#example-1c
#content-1c
HTML
<div id="container-1c"> <div id="example-1c"> <div id="content-1c" /> </div> </div>
CSS
#container-1c { display: block; /* default */ height: 400px; } #example-1c { height: 100%; } #content-1c { height: 80px; }
Block container, tall content, height 100% - elements in block containers contract to the height of their container if height is set to 100%
#container-1d
#example-1d
#content-1d
HTML
<div id="container-1d"> <div id="example-1d"> <div id="content-1d" /> </div> </div>
CSS
#container-1d { display: block; /* default */ height: 400px; } #example-1d { height: 100%; } #content-1d { height: 420px; }

Flex row containers

Flex row container - elements in flex row containers expand to the height of their container by default
#container-3a
#example-3a
#content-3a
HTML
<div id="container-3a"> <div id="example-3a"> <div id="content-3a" /> </div> </div>
CSS
#container-3a { align-items: normal; /* default - behaves as stretch in this case */ display: flex; flex-direction: row; /* default */ height: 400px; } #example-3a {} #content-3a { height: 80px; }
Flex row container, tall content - elements in flex row containers contract to the height of their container by default
#container-3b
#example-3b
#content-3b
HTML
<div id="container-3b"> <div id="example-3b"> <div id="content-3b" /> </div> </div>
CSS
#container-3b { align-items: normal; /* default - behaves as stretch in this case */ display: flex; flex-direction: row; /* default */ height: 400px; } #example-3b {} #content-3b { height: 420px; }
Flex row container, align-items set - elements in flex row containers adjust to the height of their content if align-items is set to something other than stretch
#container-3c
#example-3c
#content-3c
HTML
<div id="container-3c"> <div id="example-3c"> <div id="content-3c" /> </div> </div>
CSS
#container-3c { align-items: flex-start; display: flex; flex-direction: row; /* default */ height: 400px; } #example-3c {} #content-3c { height: 80px; }
Flex row container, align-items set, tall content - elements in flex row containers adjust to the height of their content if align-items is set to something other than stretch
#container-3d
#example-3d
#content-3d
HTML
<div id="container-3d"> <div id="example-3d"> <div id="content-3d" /> </div> </div>
CSS
#container-3d { align-items: flex-start; display: flex; flex-direction: row; /* default */ height: 400px; } #example-3d {} #content-3d { height: 420px; }

Flex column containers

Flex column container - elements in flex columns containers adjust to the height of the content by default
#container-4a
#example-4a
#content-4a
HTML
<div id="container-4a"> <div id="example-4a"> <div id="content-4a" /> </div> </div>
CSS
#container-4a { display: flex; flex-direction: column; justify-content: normal; /* default */ height: 400px; } #example-4a {} #content-4a { height: 80px; }
Flex column container, tall content - elements in flex columns containers adjust to the height of their content even for tall content
#container-4b
#example-4b
#content-4b
HTML
<div id="container-4b"> <div id="example-4b"> <div id="content-4b" /> </div> </div>
CSS
#container-4b { display: flex; flex-direction: column; justify-content: normal; /* default */ height: 400px; } #example-4b {} #content-4b { height: 420px; }
Flex column container, height: 100% - elements in flex column containers expand to the height of their container if height is set to 100%
#container-4c
#example-4c
#content-4c
HTML
<div id="container-4c"> <div id="example-4c"> <div id="content-4c" /> </div> </div>
CSS
#container-4c { display: flex; flex-direction: column; justify-content: normal; /* default */ height: 400px; } #example-4c { height: 100%; } #content-4c { height: 80px; }
Flex column container, height: 100%, tall content - elements in flex column containers contract to the height of their container if height is set to 100%
#container-4d
#example-4d
#content-4d
HTML
<div id="container-4d"> <div id="example-4d"> <div id="content-4d" /> </div> </div>
CSS
#container-4d { display: flex; flex-direction: column; justify-content: normal; /* default */ height: 400px; } #example-4d { height: 100%; } #content-4d { height: 420px; }
Flex column container, flex-grow - elements in flex column containers expand to the height of their container when flex-grow is set to 1
#container-4e
#example-4e
#content-4e
HTML
<div id="container-4e"> <div id="example-4e"> <div id="content-4e" /> </div> </div>
CSS
#container-4e { display: flex; flex-direction: column; justify-content: normal; /* default */ height: 400px; } #example-4e { flex-grow: 1; } #content-4e { height: 80px; }
Flex column container, flex-grow, tall content - elements in flex column containers expand to the height of their content when flex-grow is set to 1
#container-4f
#example-4f
#content-4f
HTML
<div id="container-4f"> <div id="example-4f"> <div id="content-4f" /> </div> </div>
CSS
#container-4f { display: flex; flex-direction: column; justify-content: normal; /* default */ height: 400px; } #example-4f { flex-grow: 1; } #content-4f { height: 420px; }

Grid containers

Grid container - elements in grid containers expand to the height of the container by default
#container-5a
#example-5a
#content-5a
HTML
<div id="container-5a"> <div id="example-5a"> <div id="content-5a" /> </div> </div>
CSS
#container-5a { align-items: normal; /* default - behaves like stretch in this case */ display: grid; height: 400px; } #example-5a {} #content-5a { height: 80px; }
Grid container, tall content - elements with tall content in grid containers expand to the height of the content
#container-5b
#example-5b
#content-5b
HTML
<div id="container-5b"> <div id="example-5b"> <div id="content-5b" /> </div> </div>
CSS
#container-5b { align-items: normal; /* default - behaves like stretch in this case */ display: grid; height: 400px; } #example-5b {} #content-5b { height: 420px; }
Grid container, tall content, height 100% - surprisingly, setting height to 100% does not contract an element's height to the height of its container. The element adjusts to the height of the content.
#container-5c
#example-5c
#content-5c
HTML
<div id="container-5c"> <div id="example-5c"> <div id="content-5c" /> </div> </div>
CSS
#container-5c { align-items: normal; /* default - behaves like stretch in this case */ display: grid; height: 400px; } #example-5c { height: 100%; } #content-5c { height: 420px; }
Grid container, tall content, overflow - elements with tall content in grid containers adjust to the height of the container if overflow is set to something other than visible
#container-5d
#example-5d
#content-5d
HTML
<div id="container-5d"> <div id="example-5d"> <div id="content-5d" /> </div> </div>
CSS
#container-5d { align-items: normal; /* default - behaves like stretch in this case */ display: grid; height: 400px; } #example-5d { overflow: auto; } #content-5d { height: 420px; }
Grid container, align-items - if a grid container sets align-items to something other than stretch (the default), then the element will adjust to the height of the content MDN docs on align-items.
#container-5e
#example-5e
#content-5e
HTML
<div id="container-5e"> <div id="example-5e"> <div id="content-5e" /> </div> </div>
CSS
#container-5e { align-items: start; display: grid; height: 400px; } #example-5e {} #content-5e { height: 80px; }
Grid container, align-items, tall content - elements with tall content in grid containers expand to the height of the content
#container-5f
#example-5f
#content-5f
HTML
<div id="container-5f"> <div id="example-5f"> <div id="content-5f" /> </div> </div>
CSS
#container-5f { align-items: start; display: grid; height: 400px; } #example-5f {} #content-5f { height: 420px; }
Grid container, align-items, tall content, overflow - elements with tall content in grid containers expand to the height of the content even if overflow is set
#container-5g
#example-5g
#content-5g
HTML
<div id="container-5g"> <div id="example-5g"> <div id="content-5g" /> </div> </div>
CSS
#container-5g { align-items: start; display: grid; height: 400px; } #example-5g { overflow: auto; } #content-5g { height: 420px; }

Absolute positioning

Absolutely positioned -
#container-6a
#example-6a
#content-6a
HTML
<div id="container-6a"> <div id="example-6a"> <div id="content-6a" /> </div> </div>
CSS
#container-6a { display: block; /* default */ position: relative; height: 400px; } #example-6a { position: absolute; } #content-6a { height: 80px; }
Absolutely positioned, tall content -
#container-6b
#example-6b
#content-6b
HTML
<div id="container-6b"> <div id="example-6b"> <div id="content-6b" /> </div> </div>
CSS
#container-6b { display: block; /* default */ position: relative; height: 400px; } #example-6b { position: absolute; } #content-6b { height: 420px; }
Absolutely positioned, height 100% -
#container-6c
#example-6c
#content-6c
HTML
<div id="container-6c"> <div id="example-6c"> <div id="content-6c" /> </div> </div>
CSS
#container-6c { display: block; /* default */ position: relative; height: 400px; } #example-6c { height: 100%; position: absolute; } #content-6c { height: 80px; }
Absolutely positioned, height 100%, tall content -
#container-6d
#example-6d
#content-6d
HTML
<div id="container-6d"> <div id="example-6d"> <div id="content-6d" /> </div> </div>
CSS
#container-6d { display: block; /* default */ position: relative; height: 400px; } #example-6d { height: 100%; position: absolute; } #content-6d { height: 420px; }

Floated elements

Floated elements - elements that set float adjust to the height of their content. MDN docs on float.
#container-7a
#example-7a
#content-7a
HTML
<div id="container-7a"> <div id="example-7a"> <div id="content-7a" /> </div> </div>
CSS
#container-7a { display: block; /* default */ height: 400px; } #example-7a { float: left; } #content-7a { height: 80px; }
Floated elements, tall content - elements that set float adjust to the height of their content
#container-7b
#example-7b
#content-7b
HTML
<div id="container-7b"> <div id="example-7b"> <div id="content-7b" /> </div> </div>
CSS
#container-7b { display: block; /* default */ height: 400px; } #example-7b { float: left; } #content-7b { height: 420px; }
Floated elements, height 100% -
#container-7c
#example-7c
#content-7c
HTML
<div id="container-7c"> <div id="example-7c"> <div id="content-7c" /> </div> </div>
CSS
#container-7c { display: block; /* default */ height: 400px; } #example-7c { float: left; height: 100%; } #content-7c { height: 80px; }
Floated elements, height 100%, tall content -
#container-7d
#example-7d
#content-7d
HTML
<div id="container-7d"> <div id="example-7d"> <div id="content-7d" /> </div> </div>
CSS
#container-7d { display: block; /* default */ height: 400px; } #example-7d { float: left; height: 100%; } #content-7d { height: 420px; }

Table cell containers

Table cell container - elements in table cell containers (td and th) adjust to the height of their content by default
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.
#td-container-8a
#example-8a
#content-8a
HTML
<div id="td-container-8a"> <div id="example-8a"> <div id="content-8a" /> </div> </div>
CSS
#td-container-8a { /* td element */ vertical-align: top; } #example-8a {} #content-8a { height: 80px; }
Table cell container, height 100%, explicit table cell height - elements in table cell containers (td and th) expand to the height of their container if height is set to 100% and the table cell container has an explicit height set
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.
#td-container-8b
#example-8b
#content-8b
HTML
<div id="td-container-8b"> <div id="example-8b"> <div id="content-8b" /> </div> </div>
CSS
#td-container-8b { /* td element */ vertical-align: top; height: 400px; } #example-8b { height: 100%; } #content-8b { height: 80px; }
Table cell container, height 100%, table cell height not set - surprisingly, setting height to 100% does not expand an element's height to the height of a table cell container (td and th) if a height for the table cell is not explicitly set. The element adjusts to the height of the content. Maybe the fifth bullet in the CSSWG's "Incomplete List of Mistakes in CSS" explains this: Percentage heights should be calculated against fill-available rather than being undefined in auto situations.
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.
#td-container-8c
#example-8c
#content-8c
HTML
<div id="td-container-8c"> <div id="example-8c"> <div id="content-8c" /> </div> </div>
CSS
#td-container-8c { /* td element */ vertical-align: top; } #example-8c { height: 100%; } #content-8c { height: 80px; }

Graphene and Relay example GraphQL pagination API and UI

beginning screenshot middle screenshot

This describes an example GraphQL API using Relay connections to power a discretely paged Amazon-like pagination UI. Connections are designed for infinite-scroll UIs, however Artsy Engineering developed a solution to use them with discretely paged UIs as described in their article: Effortless Pagination with GraphQL and Relay? Really!

I created a github repo with an implementation of the Artsy pagination API in Graphene, Django, and Python and a corresponding web app using Relay, React, and TypeScript.

For a vanilla Graphene Relay pagination API for an infinite scroll UI, see my basic example.

Graphene GraphQL backend

The Artsy pagination API augments the Relay Connection type with an additional pageCursors field that contains metadata used to implement the pagination UI. I ported their TypeScript code in artsy/metaphysics and relay-cursor-paging to Python using Graphene and Django.

Data is stored in a PostgreSQL database running in Docker. It is seeded with fish fixture data from ACNHAPI. Django is configured to log SQL queries to the console and show that SQL queries use COUNT, LIMIT, and OFFSET. Example:

SELECT COUNT(*) AS "__count" FROM "fishes_fish";
SELECT "fishes_fish"."id", "fishes_fish"."description", "fishes_fish"."icon_url", "fishes_fish"."name", "fishes_fish"."price" FROM "fishes_fish" ORDER BY "fishes_fish"."name" ASC LIMIT 5 OFFSET 5;
  • Install Python 3.8

  • Install packages, set up database, and run dev server

    $ git clone https://github.com/saltycrane/graphene-relay-pagination-example.git
    $ cd graphene-relay-pagination-example
    $ cd graphene-api
    $ python3 -m venv venv
    $ source venv/bin/activate
    $ pip install -r requirements.txt
    $ docker-compose up -d
    $ ./bin/resetdb
    $ ./manage.py migrate
    $ ./manage.py loaddata fishes
    $ ./manage.py runserver
    
  • Go to http://127.0.0.1:8000/graphql/ in the browser

  • Run the following query:

    {
      allFishes(first: 5, orderBy: "name") {
        pageCursors {
          previous {
            cursor
          }
          first {
            cursor
            page
          }
          around {
            cursor
            isCurrent
            page
          }
          last {
            cursor
            page
          }
          next {
            cursor
          }
        }
        edges {
          cursor
          node {
            name
          }
        }
      }
    }
    GraphiQL query screenshot
    graphiql screenshot
Generate a GraphQL schema
$ ./manage.py graphql_schema --schema pagination_ex_api.schema.schema --out ../schema.graphql

React Relay Next.js frontend

In addition to Relay, React, and TypeScript, the frontend UI uses relay-hooks, Next.js, and reactstrap. It takes advantage of Relay fragments and Next.js routing to store pagination state. Server-side rendering (SSR) is disabled because it's difficult to set up and isn't important for this example.

  • Install Node.js 14

  • Install packages and run dev server

    $ cd react-relay-webapp
    $ npm install
    $ npm run devserver
    
  • Go to http://127.0.0.1:3000 in the browser

My backend codeMy frontend codeExternal TypeScript codeExternal Python code