accessibility--color 4K 4K--filled accessibility--color--filled accessibility--alt accessibility account activity alarm alarm--add align--horizontal-center align--horizontal-left add-comment align--horizontal-right align--bottom-vertical align--vertical-top analytics align--center-vertical aperture alarm--subtract API api archive app apple arrow--down-left application arrow--down-right arrival arrow--down arrow--up-left arrow--left arrow--up arrows--vertical arrow--up-right arrow-shift-down arrows--horizontal arrows--vertical asleep asset asleep--filled at light attachment audio-console augmented-reality badge basketball up-to-top battery--full bar battery--charging battery--empty battery--quarter battery--half-full bee bicycle battery--low binoculars blockchain bluetooth--off blog bluetooth bookmark brightness-contrast box fork building--insights-1 building--insights-2 bullhorn building--insights-3 bus building calculator CAD calculator--check coffee calendar calibrate camera-action camera car-front caret--right carbon car caret--down caret--sort caret--right caret--sort caret--sort carousel--horizontal caret--right carousel--vertical categories category--add catalog category--new category--and category--new-each category CDA change-catalog certificate center--circle character-patterns charging-station charging-station--filled center--square chart--bubble chart--line chart--pie chart--bar certificate--check chart-network chart--line chart--scatter chart--venn-diagram chat-bot chat checkbox--indeterminate--filled checkbox--checked checkbox--checked--filled checkbox--indeterminate checkbox--indeterminate--filled checkbox--indeterminate checkbox checkmark chevron--sort chemistry chevron--sort chevron--sort chip choices circle--dash circle--filled checkmark classifier--language classification closed-caption--alt close closed-caption--filled closed-caption cloud--download bolt cloud--upload fog cloud--rain cloud--snow cloud-foundry--1 cloud-foundry--2 cloudy code cloud cognitive collaborate collapse-categories collapse-all color palette column color-toggle compare compass connect concept composer-edit mobile-data container-software copy--file corn contrast copy corner credentials course crop cursor--1 CSV cursor--2 covariate cut dashboard data--1 bike-riding data-2 data-check data--connected data-error data-base data-set data-structured data-table data-class data-share data-refinery data-vis--1 data--reference datastore data-vis--2 data-vis--4 data-vis--3 data--unstructured debug trash-can departure deploy-rules deploy delivery-truck development devices diagram distribute--horizontal-center distribute--vertical-top distribute--horizontal-right distribute--horizontal-left distribute--vertical-bottom DOC distribute--vertical-center document--add document--download document--blank document--subtract document--import document--export document--pdf document--view document--tasks document download document-sentiment dot-mark drag--horizontal down-to-bottom draw double-integer drag--vertical draggable earth--europe-africa--filled drop-photo drop-photo--filled DVR americas earth--americas--filled earth--southeast-asia--filled europe-africa earth--southeast-asia earth--filled earth education Asset 2 email edt-loop edit email--new erase enterprise checkmark event error event-schedule exam-mode events--alt events exit expand-all export face--activated--add expand-categories eyedropper face--activated face--activated--filled face--cool face--add face--dissatisfied face--dissatisfied--filled face--dizzy--filled face--neutral--filled face--dizzy face--neutral face--satisfied face--pending--filled face--pending face--wink--filled face--satisfied--filled face--wink factor fade favorite--filled fetch-upload fetch-upload--cloud favorite filter filter--edit finance fingerprint-recognition fish--multiple fish flash--filled fit-to-screen flag flash--off--filled flagging-taxi flow--data flash--off bolt flow--stream flow folder--add fog folder--off folder--details folder--shared folder folders forum forward--5 forward--30 fork fruit-bowl forward--10 function game--console game--wireless gamification gift generate-pdf GIF globe graphical-data-flow grid group hashtag hail HD--filled haze haze--night HDR HD headphones headset help-desk health-cross home hearing help--filled help hotel HTTP hourglass humidity hurricane hybrid-networking ibm-security ibm-cloud idea image identification medical-image improve-relevance in-progress increase-level industry information information--filled interactions insert integration inventory-management iot--connect iot--platform ISO--filled ISO ISO--outline jump-link JPG JSON keyboard language launch laptop legend letter--Aa letter--Cc letter--Bb letter--Ee letter--Dd letter--Hh letter--Ff letter--Gg letter--Ii letter--Jj letter--Kk letter--Ll letter--Nn letter--Mm letter--Qq letter--Pp letter--Oo letter--Rr letter--Ss letter--Tt letter--Uu letter--Vv letter--Ww light--filled letter--Yy letter--Xx letter--Zz light bolt link list--checked list--bulleted list list--checked list--numbered location dropdown-list locked login logo--facebook logo--digg logo--glassdoor logo--github logo--flickr logo--instagram logo--jupyter logo--livestream logo--medium logo--linkedin logo--openshift logo--r-script logo--pinterest logo--python logo--quora logo--skype logo--snapchat logo--slack logo--tumblr logo--twitter logo--vmware logo--xing logo--youtube logo--yelp mac-command health--exam-mode-cine logout mac-shift machine-learning magic-wand mail--all magic-wand--filled map manage-protection managed-solutions maximize medication--alert medication--reminder meter medication microscope microphone--filled microphone--off microphone--off--filled microphone migrate--alt migrate checkmark mixed-rain-hail mobile--add minimize misuse--alt mobile--check mobile--audio mobile--download mobile--landscape model--reference mobile model-builder money asleep model MOV partly-cloudy mostly-cloudy--night movement move MP3 MP4 MPEG MPG2 network--2 music network--1 network--4 network--3 new-tab no-image no-ticket nominal not-sent not-sent--filled notebook notebook--reference notification--filled unavailable noodle-bowl notification--new notification--off--filled notification--off number--1 number--0 number--3 number--2 notification number--5 number--4 number--6 number--7 number--8 number--9 number--small--1 number--small--0 number--small--3 number--small--2 number--small--4 number--small--6 number--small--5 number--small--7 omega number--small--8 open-panel--bottom number--small--9 opacity open-panel--solid--bottom open-panel--solid--left open-panel--solid--right open-panel--solid--top open-panel--left open-panel--right open-panel--top operations--field operations--record overflow-menu--horizontal overflow-menu--vertical ordinal package panel-expansion paragraph paint-brush paint-brush--alt partly-cloudy--night parent-child partnership partly-cloudy password paste pause--outline--filled pause--filled pause--outline pedestrian pause PDF percentage--filled pedestrian-child percentage person--favorite person phone--filled phrase-sentiment phone--off--filled phone--off phone pills pills--add pin pills--subtract plane play--filled play--outline play--outline--filled play--solid--filled play playlist policy PNG popup portfolio presentation-file power PPT product printer purchase qr-code quotes query-queue rain--drizzle radio-button radio-button--checked rain--heavy radio rain--scattered--night rain--scattered rain-drop cloud--rain recently-viewed RAW receipt recommend recording reminder--medical repeat--one renew reminder repeat reply report--alt report request-quote research--bloch-sphere research--matrix research--hinton-plot reset--alt reset restaurant restaurant--fine restart rewind--10 rewind--5 rocket rewind--30 rotate--clockwise--alt--filled rotate--clockwise--filled rotate--counterclockwise--alt--filled rotate--clockwise rotate--counterclockwise--alt rotate--counterclockwise--alt rotate--counterclockwise--filled rotate rotate--counterclockwise row ruler rule ruler--alt run save--model save scale scan scalpel scooter scooter--front screen--off schematics desktop SDK script send--alt search send--filled send send--alt--filled service-desk settings--adjust share-knowledge settings shopping--bag share shrink-screen--filled shopping--cart catalog shrink-screen shuffle shuttle sigma skip--back--filled skill-level skip--forward skip--back skip--forward--filled sleet smell smoke snooze snow--blizzard snow--heavy cloud--snow snow--scattered snow--scattered--night snowflake spell-check soccer split-screen spray-paint sprout star--half SQL stamp star star--filled stop--filled stop--solid--filled stop--outline--filled stop stop--outline shop strawberry string-text string-integer sunny sunny SVG sunset sunrise table-of-contents table--split table sys-provision tablet--landscape task--view tablet tag--edit tag--group tag task taxi taste temperature--frigid tennis-ball temperature--hot temperature template tennis term terminal align--center text--align--justify text-bold align--left text--all-caps align--right text-color text-fill text-creation text-highlight indent--less indent--more text--kerning text-italic text--indent indent text--leading new-line text--scale text--strikethrough text-selection text--superscript text--subscript text-underline text--small-caps text--tracking text--wrap text-link text-link--analysis text-mining thumbnail--1 text-mining--applier thumbnail--2 thumbsup thumbs-down thunderstorm--scattered--night thunderstorm--severe thunderstorm--scattered thunderstorm--strong cloud--lightning ticket time tool-box TIF timer tornado tools touch-1 tram touch-2 train translate trash-can tree-view--alt tree-view trophy tree tropical-storm TSV trophy--filled TXT type-pattern undefined--filled types undefined unlocked unknown--filled unlink up-to-top unknown USB upload user--admin user--activity user--avatar user--avatar--filled--alt user--avatar--filled user--certification user--data user--favorite--alt--filled user--favorite user--filled user--favorite--alt user--profile user--follow user--online user--identification user--role user--x-ray user--simulation user-profile--alt user--speaker video--add van user video--chat video--off--filled video--filled video--off video view--filled view view--mode-2 view--off--filled view--mode-1 view--off virtual-column virtual-column--key virtual-machine virtual-private-cloud voicemail visual-recognition virtual-private-cloud--alt vmdk-disk volume--down volume--mute--filled volume--down--filled volume--up--filled volume--mute warning--alt--filled VPN volume--up wallet warning--alt-inverted--filled warning--alt warning--alt-inverted warning-square warning-square--filled watson warning watch wheat wikis wifi--off watson--machine-learning wifi wireless-checkout windy--snow windy--strong wintry-mix windy--dust x-axis WMV windy XLS workspace XML y-axis z-axis ZIP zoom--in zoom--out zoom--reset

02 Settembre 2020

Heptapod: the complete guide from Docker to CI/CD

THUX offers a comprehensive easy guide to get Heptapod (GitLab with Mercurial) deployed using Docker behind a Træfik load balancer. The final setup is completed by activation of the built-in Docker registry and runners for optimal CI/CD.

Discover the guide to Heptapod:

 

Goal

The goal of this post is to bolster the adoption of Heptapod through the very few steps really needed to get it working via docker and get a complete setup with working CI (Continuous Integration) and registry for the image produced within the CI.

The target of the article are system administrators with a little knowledge of docker. No previous knowledge of Gitlab is required as we’ll lead you through the main concepts.

THUX has always used mercurial as the Version Control System of choice for our code and in the last years we hosted the repositories in Bitbucket, until the moment the infamous decision of Bitbucket to discontinue the support of mercurial forced us to take a decision. 
We considered switching to git but I didn’t like to do that just for a non-technical reason and we definitely like mercurial more than git.

Heptapod came to the rescue, especially its docker image, ready to deploy in no-time. Heptapod brings Mercurial SCM support to Gitlab, so that any feature of gitlab is available to the Mercurial world. 
It honestly took some time to get to the final configuration, but in hindsight it all was just very simple and can be replicated with no effort by anybody just following the right directions.

Let’s start from the big picture to offer a sneak peak of the outcome:

  1. We have a docker host that has a Træfik load balancer (version 2+), that handles reachability of all docker instances on port 80 and 443. Træfik is great as it also manages certificates from LetsEncrypt with no hassle at all.
  2. Heptapod runs in a docker and we want to reach it on port 22 as well as 443, so that we can hg clone <my-repo> in a natural way. That implies we reach docker server via ssh on port 23.
  3. We have a runner (actually we can have several different runners for different scopes) that handles the jobs needed when a commit is done. Eg: check the code, run some tests, build a docker image… 
    Heptapod makes it effortless to inspect the result of each job: a little icon informs us of the state of the job and a click on it will just drop us to the console where the job is running.
  4. We have a registry to host our images that in our case will be exposed with a separate name and will be reachable via Træfik as well.

 

Docker

We will take for granted you have a server with docker and docker-compose installed. Similarly I suppose you have 2 entries in DNS pointing to this server: one for the heptapod and one for the registry. The working configuration and a streamlined README can be downloaded from the public area of our Heptapod

SSH

As anticipated, we want to use mercurial via ssh so that we opted to change port of the docker server to 23 in /etc/ssh/sshd_config:

   Port 23

followed by a systemctl reload ssh.service

Træfik

Træfik is a very useful load balancer, suited for the docker environment. We run it in a docker, it forwards http requests to other servers. In our case it forwards requests to other dockers. 
It has several different “providers” that is to say, different ways to configure it’s routing. Here we use only the “docker” provider, that reads the docker socket to read the “labels” each docker can publish. This way, the configuration of the routing to a particular docker is delegated to that same docker and it is defined in its docker-compose.yml.
You can see how this mechanism is used to publish an internal dashboard of traefik:

traefik.http.routers.api.rule: Host(`monitor.thux.it`)
traefik.http.routers.api.service: api@internal

that you can read as: all trafik to monitor.thux.it is to be routed to the service api@internal
 

version: "3.3"

services:
  traefik:
    image: traefik:v2.2
    restart: always
    ports:
      - "80:80" # <== http
      - "443:443" # <== https
    command:
    #### These are the CLI commands that will configure Traefik and tell it how to work! ####
      - --api.dashboard=true # <== Enabling the dashboard to view services, middlewares, routers, etc...
      - --log.level=DEBUG # <== Setting the level of the logs from traefik
      - --accessLog.filePath=/logs/access.log
      - --log.filePath=/logs/traefik.log
      ## Provider Settings - https://docs.traefik.io/providers/docker/#provider-configuration ##
      - --providers.docker=true # <== Enabling docker as the provider for traefik
      - --providers.docker.exposedbydefault=false # <== Don't expose every container to traefik, only expose enabled ones
      - --providers.docker.network=web # <== Operate on the docker network named web
      ## Entrypoints Settings - https://docs.traefik.io/routing/entrypoints/#configuration ##
      - --entrypoints.web.address=:80 # <== Defining an entrypoint for port :80 named web
      - --entrypoints.web-secured.address=:443 # <== Defining an entrypoint for https on port :443 named web-secured
      ## Certificate Settings (Let's Encrypt) -  https://docs.traefik.io/https/acme/#configuration-examples ##
      - --certificatesresolvers.leresolver.acme.tlschallenge=true # <== Enable TLS-ALPN-01 to generate and renew ACME certs
      - --certificatesresolvers.leresolver.acme.email=myemail@domain.com # <== Setting email for certs
      - --certificatesresolvers.leresolver.acme.storage=/letsencrypt/acme.json # <== Defining acme file to store cert information
    volumes:
      - ./logs:/logs # <== Volume for certs (TLS)
      - ./letsencrypt:/letsencrypt # <== Volume for certs (TLS)
      - /var/run/docker.sock:/var/run/docker.sock # <== Volume for docker admin
      - ./dynamic.yaml:/dynamic.yaml # <== Volume for dynamic conf file, **ref: line 27
    networks:
      - web # <== Placing traefik on the network named web, to access containers on this network
    labels:
      #### Labels define the behavior and rules of the traefik proxy for this container ####
      traefik.enable: true
      traefik.http.routers.api.rule: Host(`monitor.thux.it`)
      traefik.http.routers.api.service: api@internal 
      traefik.http.routers.api.middlewares: redirect-to-https
      traefik.http.middlewares.redirect-to-https.redirectscheme.scheme: https
      # let’s define a redirect to https valid for any request:
      traefik.http.routers.http-catchall.rule: hostregexp(`{host:.+}`)
      traefik.http.routers.http-catchall.entrypoints: web
      traefik.http.routers.http-catchall.middlewares: redirect-to-https
      traefik.http.middlewares.redirect-to-https.redirectscheme.scheme: https
      traefik.http.routers.traefik.rule: Host(`monitor.thux.it`)
      traefik.http.routers.traefik.service: api@internal
      traefik.http.routers.traefik.tls.certresolver: leresolver
networks:
 web:
   external: true

 

Note we tell Træfik that it must request certificates to LetsEncrypt. 
3 lines of configuration are enough to forget about ssl certificates from now on. As soon as Træfik receives from the label of a docker machine that it needs a certificate it will issue the request to LetsEncrypt for us and it will renew it when needed.

Heptapod

Let’s move on to the core of our challenge: Heptapod.
As of this writing, the current version is 0.15.1 but 1 will soon be available. Current version is built on Gitlab 13.1.5 and mercurial 5.4.2 while the last release of Gitlab is 13.2.1. 

You can configure GitLab via file gitlab.rb (ruby syntax) or via the environment configuration GITLAB_OMNIBUS_CONFIG that will overwrite any configuration found in the default config template. I’ll follow this second possibility as it is easier to copy/paste and to keep updated when you update your docker image. 
My docker-compose.yml starts as follows:

version: '3.4'

services:
  heptapod:
    image: octobus/heptapod:0.15.1
    hostname: hg.thux.dev
    restart: always
    networks:
      - web
    ports:
      - "22:22"
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'https://gitlab.thux.it'
        gitlab_rails['gitlab_email_from'] = 'hg@thux.it'
        gitlab_rails['gitlab_email_display_name'] = 'Thux Mercurial Server'
        gitlab_rails['gitlab_email_reply_to'] = 'name@example.com'
        gitlab_rails['smtp_enable'] = true
        gitlab_rails['smtp_address'] = "smtp.example.com"
        gitlab_rails['smtp_port'] = 25
        gitlab_rails['smtp_authentication'] = "plain"
        gitlab_rails['smtp_enable_starttls_auto'] = true
        gitlab_rails['smtp_domain'] = "thux.it"
                 # Registry settings
        registry_external_url 'https://registry.gitlab.thux.it'
        gitlab_rails['registry_enabled'] = true
        registry['enable'] = true
        gitlab_shell['migration'] = { enabled: true, features: ["hg"] }
        nginx['listen_port'] = 80
        nginx['listen_https'] = false
        nginx['proxy_set_headers'] = {
         "Host" => "$$http_host_with_default",
         "X-Real-IP" => "$$remote_addr",
         "X-Forwarded-For" => "$$proxy_add_x_forwarded_for",
         "X-Forwarded-Proto" => "https",
         "X-Forwarded-Ssl" => "on",
         "Upgrade" => "$$http_upgrade",
         "Connection" => "$$connection_upgrade"
        }
        registry_nginx['listen_https'] = false
        registry_nginx['listen_port'] = 80
  volumes:
       - ./gitlab/config:/etc/gitlab
       - ./gitlab/logs:/var/log/gitlab
       - ./gitlab/data:/var/opt/gitlab

 

Before delving into the GitLab configuration, we can note:

  • This configuration requests docker daemon to forward port 22 to port 22 of this docker instance. 
  • The real data will be kept outside the docker, in a directory of the host and bind-mounted within the docker instance so as to be available to Heptapod. The volume section should result pretty clear.
  • Logs are kept in a volume as well, that is bind-mounted
  • Docker is attached to an external network web that is the network where Træfik will forward traffic to
  • Environment variable GITLAB_OMNIBUS_CONFIG is defined using yaml literal style - so that newlines are preserved. Double $$ is needed to pass a single $ to gitlab and avoid early interpolation by docker.
  • gitlab/conf is an empty folder the first time we start but will be filled by gitlab with gtilab.rb and auto-generated secrets. It’s linked to a VOLUME and if you don’t bind-mount a host folder it will be regenerated each time with problems for the runners (When I did I couldn’t even open the Runners’ page)
  • Both settings registry_nginx[... ] and nginx[...] are needed!!

GitLab configuration

As stated above we preferred to configure gitlab via variable GITLAB_OMNIBUS_CONFIG so we don’t need to mount a conf file and when we upgrade to a newer version of heptapod we’re sure any default will correctly be picked from the template. We can inspect the default configuration reading the file /etc/gitlab/gitlab.rb:

 docker-compose exec heptapod cat /etc/gitlab/gitlab.rb

That file is exactly the default, while configuration from environment variable GITLAB_OMNIBUS_CONFIG is applied on top of that. You will need to look at it to know which values you can customize. In case you prefer to edit it directly remember to bind-mount it.  

The first section is very easy to understand: it just enables email and configures the relay server. 
The second section configures how to enable and serve registry service. As per our big picture, we decided to use Træfik to handle the ssl certificate so the connection between 
træfik and gitlab is on port 80, that is what the nginx config part does. The result can be inspected by running:

# docker-compose exec heptapod cat /var/opt/gitlab/nginx/conf/gitlab-registry.conf

whose output is:

# This file is managed by gitlab-ctl. Manual changes will be erased! server {
  listen *:80;
  server_name registry.gitlab.thux.it;
  ...
  location / {     proxy_set_header Host $http_host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Forwarded-Ssl on;
    ...
    proxy_pass          http://localhost:5000;
  }
}

 

Up to now we have not yet configured Træfik that knows nothing about our docker services. It’s time to add one more section in our docker-compose file:

   labels:
   traefik.enable: true
   traefik.http.routers.hg.rule: Host(`gitlab.thux.it`)
   traefik.http.routers.hg.entrypoints: web-secured
   traefik.http.routers.hg.tls.certresolver: leresolver
   traefik.http.routers.hg.tls: true
   traefik.http.routers.hg.service: hg    
   traefik.http.routers.registry.rule: Host(`registry.gitlab.thux.it`)
   traefik.http.routers.registry.entrypoints: web-secured
   traefik.http.routers.registry.service: registry
   traefik.http.routers.registry.tls.certresolver: leresolver
   traefik.http.routers.registry.tls: true    

   traefik.http.services.hg.loadbalancer.server.port: 80
   traefik.http.services.registry.loadbalancer.server.port: 80

 

Starting from the last 2 lines: docker informs Træfik that it offers 2 services on port 80 named hg and registry, these service names are used in the router definitions above. Each of the two blocks defines a router (named again hg and registry) with entry point port 443 (named web-secured in our Træfik docker-compose), how to get a certificate and which service should be attached to. Both services are handled by nginx that proxies them according to their domain name and we saw earlier an excerpt.

If we start this docker machine with command:

mkdir -p gitlab/config gitlab/logs gitlab/data
docker-compose pull
docker-compose up -d

 

… and wait enough (some minutes) for the services to be started we should see a page where the password for the root user is prompted (“Please create a password for your new account”). That is where we set it. Going to administration area we should see something similar to this:

Please note that Container registry appears enabled.

Should you fiddle around with gitlab-ctl, you should call it via exec. The proper way to invoke the help is:

   docker-compose exec heptapod gitlab-ctl help

Shared Runners

So far so good! Shared Runners are enabled but we have none yet. Shared Runners are a key part in continuous integrations. They are the servers that execute the jobs. Each of our projects has a configuration file named .gitlab-ci.yml that dictates which jobs will be run on each commit and under which conditions. We could decide to check code quality, to run tests or to produce the docker image… we can decide to run some jobs on each commit and an image build on each commit of the default branch. The configuration of .gitlab-ci.yml is outside the scope of this article but I’ll give a couple examples to generate docker images as it goes along with different Runner configurations.

Let’s start from the docker-compose configurations for our runner:

  heptapod-runner:
    image: octobus/heptapod-runner:0.3.0
    restart: always
    networks:
      - web
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./gitlab-runner:/etc/gitlab-runner

 

Please note:

  • configuration is in gitlab-runner that is a folder, not a file. We don’t need to pass a file as it will be created in the registration process
  • we bind-mount /var/run/docker.sock ie: the runner will see the same docker daemon as the docker daemon we’re running on. This is just one of at least 3 possibilities. I’ll register and configure two runners 
    • Thux Socket Binding: that takes advantage of this shared socket 
    • Thux Priviledged: that uses dind (docker in docker) to have a completely independent docker daemon

we will use job’s tags to select the correct runner for each job

  • no labels are defined as runners are not exposed to the external. In fact you can run runners anywhere, on any servers. They will start working for your Heptapod after you have registered them.

Registering the runner

To register a runner we need to get the token from the administration area, following “GitLab runner” link and run:

# docker-compose exec heptapod-runner gitlab-runner register

The registration process will prompt us with some simple questions and the result will be written to the config.toml. You can add more runners specialized in different jobs. After adding 2 runners and a little bit of hand editing, this is the resulting configuration:

# cat gitlab-runner/config.toml
concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800 [[runners]]
  name = "ThuxRunnerPriviledged"
  url = "https://hg.thux.dev/"
  token = "SecretToken"
  executor = "docker"
  environment = ["DOCKER_DRIVER=overlay2"]
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
  [runners.docker]
    tls_verify = false
    image = "docker:19.03.12"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/certs/client", "/cache"]
    shm_size = 0 [[runners]]
  name = "Thux Runner Socket Binding"
  url = "https://hg.thux.dev/"
  token = "SecretToken"
  executor = "docker"
  environment = ["DOCKER_DRIVER=overlay2"]
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
  [runners.docker]
    tls_verify = false
    image = "docker:19.03.12"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
    shm_size = 0

 

Please note:

  • Thux Priviledged has 
    • priviledged = True
    • a volume for the certificates
  • Thux Runner Socket Binding has the socket in the volumes
  • The previous points have been hand edited after the registration. Upon saving the file, the changes are automatically applied.
  • DOCKER_DRIVER=overlay2 is already default in all my docker server. Don’t set it if you happen to have a btrfs underlying fs (use btrfs in that case)
  • You can’t see here, but while registering you are given the opportunity to set tags that will be used to couple job and runner. I added dind to "Thux Priviledged" and Python to the other

A third option, that may be more secure, uses kaniko, a builder of docker images that doesn’t require root privileges. 

Working example

You can test with a simple project. Let’s start with a fake Python project. No code and random dependencies:

  echo “ipython\nrequests” > requirements.txt

The Dockerfile is very basic but uses BuildKit (a toolkit for converting source code to build artifacts in an efficient, expressive and repeatable manner - note the syntax = part, it's not a comment) that in turns implies caching is working::

# syntax = docker/dockerfile:experimental
FROM python:3.8 as production-stage
COPY requirements.txt /requirements.txt RUN \
  mkdir /code/ \
  && pip install --no-cache-dir -r /requirements.txt COPY . /code/
WORKDIR /code/
USER ${APP_USER}:${APP_USER} CMD ["ipython"]

 

Let’s now move on to preparing a couple of .gitlab-ci.yml for different runners.

Example with dind

build_image:
  image: docker
  tags:
    - dind
    
  services:
    - name: "docker:19.03.12-dind"
      command: ["--experimental"]   variables:
    DOCKER_BUILDKIT: 1
    IMAGE_NAME: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
    DOCKER_DRIVER: overlay2
    # Create the certificates inside this directory for both the server
    # and client. The certificates used by the client will be created in
    # /certs/client so we only need to share this directory with the
    # volume mount in `config.toml`.
    DOCKER_TLS_CERTDIR: "/certs"   script:
    - mkdir $HOME/.docker
    - "echo -e '{\n  \"experimental\": \"enabled\"\n}' | tee $HOME/.docker/config.json"
    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY
    - docker build -t  $IMAGE_NAME .
    - docker push $IMAGE_NAME

 

Some remarks:

  • docker image is used to execute this job, entrypoint command realizes that no DOCKER_HOST is defined and uses tcp://docker:2375 (its default, when there’s no docker socket) i.e. the docker service from the docker:19.03.12-dind image
  • DOCKER_BUILDKIT enables BuildKit 
  • “experimental”, both in server and client, gives access to experimental features that you may not need
  • We believe this is not the correct way to use buildkit with dind. Build time in my case is much longer than the other runner. Here is an article that explains how to use both dind and caching to speed up image build

Inspect result

Add our .gitlab-ci.yml, commit and push. 
A little icon will appear in the project page indicating the state of the job, clicking on that icon will get you to a pipeline page where you can investigate what is going on. 


Example with bind-mount

build_image:
  image: docker   variables:
    DOCKER_BUILDKIT: 1
    IMAGE_NAME: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG   script:
    - mkdir $HOME/.docker
    - "echo -e '{\n  \"experimental\": \"enabled\"\n}' | tee $HOME/.docker/config.json"
    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY
    - docker build -t  $IMAGE_NAME .
    - docker push $IMAGE_NAME

 

Some remarks:

  • tag python will make Thux Runner Socket Binding choose it
  • docker command (cli) is provided by docker image whose entrypoint, missing a DOCKER_HOST variable, looks for a /var/run/docker.sock socket (that we passed as bind-mount from the main docker daemon) 
  • the special user gitlab-ci-token is granted access to the repository

This time let’s follow the icons:


to land in the terminal where we see the job execution logs

Final remarks

We want to thank the Octobus team for the effort done to add mercurial to Gitlab in this wonderful way. I found the team very helpful, a special mention goes to Georges Racinet. I would say it is not a minor detail to have good and prompt support. They’re also super reactive in the mercurial list and that makes it a perfect match. 
We started investigating Heptapod just to have a shared repo for our code and we landed in a much more up-to-date setup than we were used before.

You can download the ready-to-use configuration from the public area of our Heptapod site.

 

This article is written and curated by Alessandro Dentella

SYSTEMS