Gitlab CI/CD for Kubernetes, Helm and Laravel

31 Mar 2022Gitlab Kubernetes DevOps

Context: This post is an update of Gitlab CI/CD example with Laravel, Kubernetes and Helm that come from a series Kubernetes, Helm, Istio, Laravel, PHP-FPM, Nginx, GitLab the DevOps Way. It should be easier to see changes made for this post by keeping the original.

GOAL: Let's have a full Gitlab pipeline, to deploy a Laravel application with Continous Delivery, here is the technology stack as remember.

Stack:

Kubernetes (x2, prod and staging) / GKE / Istio
GCP
Gitlab (2 runners, shell and docker)
PHP-FPM / Laravel / Nginx
Laravel migration
Test suite, audit.

I will not go over every detail as some was already discuss on the Original Post.

Stages in the .gitlab-ci.yaml look like this.

stages:
  - nginx
  - build
  - test
  - audit
  - migration_needed
  - maintenance_down
  - migration
  - delivery

Here is a brief overview of every stages.

nginx. Laravel is run on php-fpm, nginx container inside the pod. I'll build his onw nginx docker image, that contains the configuration. More on the Helm used here.
build. The main application Docker build.
test. phpunit tests.
audit. Some audit with php and npm.
migration_needed only answer to the question is a migration is needed.
maintenance_down, if a migration is needed, the maintenance Laravel mode is turn on
migration. The migration itself
delivery. Let's push in the Kubernetes cluster.

nginx

nginx:
  stage: nginx
  tags:
    - gcp-shell
  only:
    - master
    - develop
  script:
    - docker build -t eu.gcr.io/nci-login-01/nci-nginx:${CI_PROJECT_PATH_SLUG} devops/nginx
    - gcloud auth activate-service-account gitlab-push-container@nci-login-01.iam.gserviceaccount.com --key-file=/gitlab.json
    - gcloud auth configure-docker
    - docker push eu.gcr.io/nci-login-01/nci-nginx:${CI_PROJECT_PATH_SLUG}

This is a simple build and push example of a nginx. The Dockerfile is not shown here, but only copy configuration needed.

build

build:
  stage: build
  tags:
    - gcp-shell
  except:
    - tags
  before_script:
    - echo "$GCS_key" > somekey.json
  script:
    - docker build -t ${IMAGE}:${TAG} .
    - gcloud auth activate-service-account gitlab-push-container@nci-login-01.iam.gserviceaccount.com --key-file=/gitlab.json
    - gcloud auth configure-docker
    - docker push ${IMAGE}:${TAG}

Again, a simple build and push stage, the only thing here is the key is injected not as a Kubernetes Secret but with a Gitlab variable inside the image build.

test

test:
  stage: test
  image: ${IMAGE}:${TAG}
  services:
    - mysql:latest
  variables:
    MYSQL_DATABASE: laravel_test
    MYSQL_ROOT_PASSWORD: root
  tags:
    - gcp-docker
  except:
    - tags
  before_script:
    - curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
    - cd /var/www/html
    - composer install # on a besoin des dépendences de dev (contrairement à la prod)
    - apk add gettext # envsubst
    - envsubst < /var/www/html/devops/.env.test > /var/www/html/.env
    - php artisan migrate
    - php artisan route:clear
  script:
    - ./vendor/phpunit/phpunit/phpunit

I used envsubst to inject the APP_KEY part inside the .env like so.

APP_NAME=Laravel
APP_ENV=test
APP_KEY=$APP_KEY_PREPROD

DB_CONNECTION=mysql
DB_HOST=mysql
DB_PORT=3306
DB_DATABASE=laravel_test
DB_USERNAME=root
DB_PASSWORD=root

The idea here is to get a fresh mysql as a service, install developments requirements to run phpunit test suite.

audit

I don't want the stage to fail the pipeline, this is the allow_failure part.
I needed to force the exit status of the gitlab pipeline to get this work.
If vulnerabilities are found, I'll display them in the stage output.
Both NPM and PHP audit are run in parallel.
PHP audit come from the Symfony community.
jq is a fantastic tool to manipulate json output of a command. Don't know it yet? You should, always usefull.

npm_audit:
  stage: audit
  image: ${IMAGE}:${TAG}
  tags:
    - gcp-docker
  only:
    - develop
  allow_failure: true
  before_script:
    - apk add --update nodejs npm jq
  script:
    - if [ $(npm audit --package-lock-only --json | jq .metadata.vulnerabilities.critical) == '0' ] ; then exit 0; else npm audit; fi

php_audit:
  stage: audit
  image: ${IMAGE}:${TAG}
  tags:
    - gcp-docker
  only:
    - develop
  allow_failure: true
  before_script:
    - wget https://github.com/fabpot/local-php-security-checker/releases/download/v1.2.0/local-php-security-checker_1.2.0_linux_amd64
    - mv local-php-security-checker_1.2.0_linux_amd64 checker
    - chmod +x checker
    - apk add --update jq
  script:
    - if [ $(./checker -format json | jq '. | length') == '0' ] ; then exit 0; else ./checker; fi

migration_needed

This is another example of a previous post Change value of variables between jobs in Gitlab. In this case is more about keep the value that changing it. I could say How to keep values of variables between jobs in Gitlab.

The idea is simple, a dry-run of the migration, store in the artifacts.reports.dotenv.
I could not make it work with a Docker Runner. Only Shell Runner
Do I need to make a migration? Yes, No? Keep the answere for the next stage.
Once again, envsubst help me to get variables fixed in .env.master and .env.develop that would be outputted in .env

migration_needed:
  stage: migration_needed
  tags:
    - gcp-shell
  image: ${IMAGE}:${TAG}
  variables:
    DOCKER_NAME: "${CI_PROJECT_NAME}_${CI_JOB_ID}"
  only:
    - master
    - develop
  before_script:
    - |
      docker run -d \
      --name ${DOCKER_NAME} \
      -e DB_PASSWORD_PREPROD=${DB_PASSWORD_PREPROD} \
      -e DB_PASSWORD_PROD=${DB_PASSWORD_PROD} \
      ${IMAGE}:${TAG}
    - docker exec ${DOCKER_NAME} apk add gettext # envsubst
    - docker exec ${DOCKER_NAME} /bin/sh -c "envsubst < /var/www/html/devops/.env.${CI_COMMIT_BRANCH} > /var/www/html/.env"
    - docker exec ${DOCKER_NAME} php artisan key:generate
  script:
    - |
      if [[ "$(docker exec ${DOCKER_NAME} php artisan migrate --pretend)" == "Nothing to migrate." ]]; then
        echo "MIGRATION=FALSE" >> build.env
      else
        echo "MIGRATION=TRUE" >> build.env
      fi
  after_script:
    - docker rm --force ${DOCKER_NAME}
  artifacts:
    reports:
      dotenv: build.env

maintenance_down

Only if needed as seen in the previous stage.
The command is based on Execute Command on Each Pod of a Kubernetes Deployment

maintenance_down:
  stage: maintenance_down
  variables:
    CONNECT_CLUSTER: ${CONNECT_K8S_PREPROD_CMD}
  rules:
    - if: $CI_COMMIT_BRANCH == "master"
      when: on_success
      variables:
        CONNECT_CLUSTER: ${CONNECT_K8S_PROD_CMD}
    - if: $CI_COMMIT_BRANCH == "develop"
      when: on_success
      variables:
        CONNECT_CLUSTER: ${CONNECT_K8S_PREPROD_CMD}
    - when: never
  tags:
    - gcp-shell
  script:
    - ${CONNECT_CLUSTER}
    - (if [ "$MIGRATION" == "TRUE" ]; then kubectl get po -l app.kubernetes.io/name=${CI_PROJECT_NAME} -o name | xargs -I{} kubectl exec {} -c ${CI_PROJECT_NAME}-backend -- php artisan down; fi);

migration

Not much to say.

migration:
  stage: migration
  dependencies:
    - migration_needed
  only:
    - master
    - develop
  image: ${IMAGE}:${TAG}
  tags:
    - gcp-docker
  before_script:
    - apk add gettext # envsubst
    - cd /var/www/html
    - envsubst < /var/www/html/devops/.env.${CI_COMMIT_BRANCH} > /var/www/html/.env
    - php artisan key:generate
  script:
    - (if [ "$MIGRATION" == "TRUE" ]; then php artisan migrate --force; fi);

delivery

Lots was explained in the previous Gitlab example
helm has two command, install and upgrade. It seems like I cannot use only one of them. This mean the first installation would fail if I used only upgrade.

delivery:
  stage: delivery
  variables:
    CONNECT_CLUSTER: ${CONNECT_K8S_PREPROD_CMD}
    VALUES_FILE: devops/helm/stage.yaml
  rules:
    - if: $CI_COMMIT_BRANCH == "master"
      when: on_success
      variables:
        CONNECT_CLUSTER: ${CONNECT_K8S_PROD_CMD}
        VALUES_FILE: devops/helm/prod.yaml
    - if: $CI_COMMIT_BRANCH == "develop"
      when: on_success
      variables:
        CONNECT_CLUSTER: ${CONNECT_K8S_PREPROD_CMD}
        VALUES_FILE: devops/helm/stage.yaml
    - when: never
  tags:
    - gcp-shell
  script:
    - ${CONNECT_CLUSTER}
    - envsubst < ${VALUES_FILE} > devops/helm/dist.yaml
    - |
      if [[ $(helm list -n ${CI_PROJECT_NAME} -q) ]]; then
        helm upgrade -f devops/helm/dist.yaml ${CI_PROJECT_NAME} devops/helm/ -n ${CI_PROJECT_NAME}
      else
        envsubst < devops/namespace.yaml > devops/ns.yaml
        kubectl apply -f devops/ns.yaml
        helm install -f devops/helm/dist.yaml ${CI_PROJECT_NAME} devops/helm/ -n ${CI_PROJECT_NAME}
      fi

I like to make it short, but: Hope it can help someone.

Next: Kubernetes and Google Cloud Container Registry
Previous: Is DevOps For You

Gitlab CI/CD for Kubernetes, Helm and Laravel

nginx #

build #

test #

audit #

migration_needed #

maintenance_down #

migration #

delivery #