Sign in Subscribe

Production Ready Pipelines: GitLab, GitHub, Jenkins, and Harness

Production Ready Pipelines: GitLab, GitHub, Jenkins, and Harness

This post is not your average "What is CI/CD?" or "Tool A vs Tool B" post. We are going to dig into what makes a production ready pipeline. A CI/CD pipeline is one of the cornerstones of a successful system, and I'm going to give it away for free. Not just one pipeline, but four!

This is the first of a four-part series revealing what a production-ready pipeline looks like on each of the four leading software delivery platforms: GitLab, GitHub, Jenkins, and Harness. This series is designed for both beginners stepping into the CI/CD realm and seasoned professionals on the lookout for a slight improvement to their existing set of patterns.

In this post we will provide an overview of the series, a brief rundown of the four platforms, and then the good stuff: a production-ready pipeline for GitLab. As always, I have provided a demonstrative repository that includes plenty of extra goodies on top of what we cover here. Feel free to use whatever appeals to you.

Series Overview

GitLab

A comprehensive software delivery platform, GitLab not only provides repository management but also offers CI/CD capabilities right out of the box. Its integrated approach makes it a favorite for teams looking for a unified solution. The race is pretty tight, but GitLab is easily one of the front runners in this area.

GitHub

In the second part of this series we will dive into GitHub CI/CD capabilities, namely GitHub Actions. Founded in 2008, GitHub quickly became an integral component of both open and closed-source projects. In 2018, Microsoft acquired GitHub, solidifying its position in the developer ecosystem. GitHub is so synonymous with git repository hosting that all major software delivery platforms have some form of GitHub integration.

Jenkins

In the third part of this series we will walk you through Jenkins, an open-source automation server that offers a robust set of tools for CI/CD. Created by Kohsuke Kawaguchi in 2006, Jenkins provides an extensible, self-contained platform to automate all sorts of tasks related to building, testing, and delivering software. Its plugin architecture allows developers to expand its capabilities and integrate with a vast array of tools. Given its versatility and wide adoption, Jenkins has become a foundational tool in a majority of engineering teams around the world.

Harness

In the fourth and final part of this series we will cover Harness, a rising star in the software delivery platform world. With a focus on simplifying the deployment and release processes, Harness emphasizes the use of automation, AI, and ML to provide a smarter, safer, and more efficient way to release software. Harness aims to reduce the complexities often associated with traditional deployment pipelines, offering features such as automated rollback in case of issues, performance analysis, and cloud cost optimization. It positions itself as a solution for businesses looking to achieve rapid yet reliable software releases.

What is a production-ready pipeline?

I define a production-ready pipeline as meeting the following goals:

  • Check patterns and practices (e.g., naming, formatting, syntax)
  • Test code and mutations (e.g., unit, integration, contract, system)
  • Build and publish images (e.g., container images)
  • Deploy to environments (e.g., integration, testing, staging, production)
  • Report release metrics (e.g., coverage, quality, cycle time)

The above stages are ordered in a traditional manner, from low to high impact. For example, it is low impact to check patterns and practices and thus we want that as early as possible in the pipeline. Testing is higher impact so while we must fail a pipeline as early as possible for failed tests, it is typically positioned after the check phase. Build is very high impact due to the nature of producing container images thus we delay it till just before deploy - which is towards the very end of a pipeline due to its dependency on the previous stages.

I typically advise my clients to position reporting between the test and build phases. Why? Well, for two reasons: 1) if a pipeline fails we may still want to see metrics for the failed run; and 2) deploy can be gated by one or more report metrics. This is non-traditional but the practice is gaining traction.

The GitLab Pipeline

So, let's take a look shall we. Let's see what a GitLab production-ready pipeline looks like.

The grammar that GitLab uses for its pipelines is similar to the other three, where the main elements are:

Image

image: mcr.microsoft.com/dotnet/sdk:7.0

The pipeline is configured to run using the .NET SDK image, which includes all the tools necessary to interact with a .NET repository. The runner image really depends on the runtimes and frameworks used in the repository. Sometimes there is no image that includes everything you need and in those situations your pipeline will have a step that installs whatever is missing.

Variables

variables:
  DOTNET_NOLOGO: 'true'

Use of global variables in a pipeline should be limited, for the same reasons we programmers limit their use in code. That said there are typically a few global variables, such as configuration that applies to most or all jobs.

Stages

stages:
  - check
  - test
  - report
  - build
  - deploy

From above you can see that the pipeline stage order is: check, test, report, build, and deploy. The concept of stages is how GitLab groups jobs; jobs within the same stage are run in parallel (by default). And speaking of jobs...

Check

check:
  stage: check
  script:
    - dotnet format --verify-no-changes --exclude Tests --verbosity quiet

For .NET projects the above is a common way to check patterns and practices in the repository. In other runtimes and languages this is handled by a linter.

Test

test:
  stage: test
  script:
    - dotnet test --environment DOTNET_ENVIRONMENT=Test --verbosity normal --filter "Category!=Service&Category!=Integration" --collect:"XPlat Code Coverage" /p:ExcludeByAttribute=\"Obsolete,GeneratedCodeAttribute,CompilerGeneratedAttribute,ExcludeFromCodeCoverage\"
    - dotnet tool install --global dotnet-reportgenerator-globaltool; ~/.dotnet/tools/reportgenerator "-reports:**/TestResults/**/coverage.cobertura.xml" "-targetdir:.ignored/coverage-reports" "-classfilters:-*Program*" \"-reporttypes:Html\;Badges\;JsonSummary\"
  artifacts:
    paths:
      - .ignored/coverage-reports

This is typically where the most gobbledygook shows up because of nature of the phase. The above performs two steps: run tests and generate coverage reports. All the noise you see is mostly around excluding certain code (e.g., generated code). For conciseness I have omitted contract, system, and mutation testing - but your pipeline should include those to properly test the system.

Report

pages:
  stage: report
  needs:
    - job: test
  only:
    - master
  script:
    - mv .ignored/coverage-reports public
  artifacts:
    paths:
      - public

The above is commonly what you will see in a GitLab pipeline to publish release metrics. The reporting parameters will differ for your situation, but generally this job copies report content to a path (public above) and then publishes that to a pipeline page in the GitLab platform. You may publish the content wherever you wish, so tailor this step to your specific needs.

Build

build:
  stage: build
  only:
    - master
  image: docker
  services:
    - docker:dind
  script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker build --quiet --build-arg PROJECT=Api -f Api/Dockerfile -t $CI_REGISTRY_IMAGE/api -t $CI_REGISTRY_IMAGE/api:$CI_COMMIT_SHORT_SHA -t $CI_REGISTRY_IMAGE/api:latest .
    - docker push --quiet $CI_REGISTRY_IMAGE/api
    - docker push --quiet $CI_REGISTRY_IMAGE/api:$CI_COMMIT_SHORT_SHA
    - docker push --quiet $CI_REGISTRY_IMAGE/api:latest

This job is the first occurrence of using an alternate image for the job. Since the primary task for this job is producing container images, this job is configured to use the Docker image. This is also the first time we see variables in use. All four platforms support pipeline variables, and GitLab has its own set of pre-defined variables. There are two functions of pipeline variables: 1) parameters that vary the behavior of job execution (e.g., whether to skip a job); and 2) secrets that necessarily must not be hard-coded into the pipeline.

Deploy

deploy:
  stage: deploy
  rules:
    - if: '$CI_COMMIT_BRANCH == "master" && $GITLAB_ENVIRONMENT == "Local"'
      when: always
    - when: never
  image: bitnami/kubectl
  script:
    - kubectl apply -f .k8s/api/deployment.yml

The deploy phase is where we install the system to one or more environments (preferably to all). In our example above we are installing the Api component to a Kubernetes cluster using kubectl. This of course is the one of the lowest levels of Kubernetes API interaction. Your organization may use Kustomize manifests or perhaps you use Helm charts. Whatever mechanism you use, once this phase completes the pipeline has done its primary job: update your system!

Scripts vs Statements

Before we wrap this up, a quick word about scripts vs statements as job work. There are two non-mutually exclusive methods for executing job work: scripts and statements. In the above example, and for all my clients, we use inline statements (e.g., docker build ...). The other option would be to combine statements together into a script and then execute that script instead. This approach has the advantage we programmers call refactoring. And that is great. But when it comes to pipelines I advise against it. The level of indirection can make it difficult to know what work is being done. If a statement within the script generates an error, it can be difficult to know which line produced the error.

Until tooling advances a little bit more I continue to advise my clients to use inline statements rather than scripts wherever possible.

Conclusion

And there you have it! That's what I think a minimum production-ready pipeline looks like for GitLab. My objective with this post (and this series) is to provide you with a really good jumping off point.

In the next part of this series we will skip all the overviews and jump right into what that looks like for GitHub. Until then, happy deploying!

Code well, and shape the future.

Resources