Automating image deployment workflow with GitLab CI/CD

devops
Published

August 26, 2025

In previous posts, we have provisioned an Oracle Compute Linux VM and deployed a simple helloworld Django app to it. We then ran the same app as a Docker container. To streamline our workflow, we also identified an integrated platform to store both our code and container image.

Now let’s build on top of this progress to migrate our existing containerized application workflow to a fully automated CI/CD pipeline using GitLab. In this post, we will

By using a continuous and integrated workflow, a developer only needs to push changes to a Gitlab project repo, and the Pipeline will take care of BOTH building and deploying. This involves just 1 step and beats our previous workflow of pushing changes to Github, building image manually in dev box and pushing to Docker Hub, then pulling the image and manually running the container in VM. Now we gain: - clarity: everything including code changes (via PR or push) and build process (each step clearly outlined in yaml and Dockerfile) are documented and versioned. - reproducible: the steps are deterministic and can be repeated with expected results. - automation: image creation, delivery and container initialization are automatically executed by the CI/CD Pipeline.

Let’s go!

Prerequisites

Before we dive into GitLab CI/CD, ensure you have the following:

  • GitLab account
  • Linux VM access: SSH access to your Linux VM with a user that has sudo permissions to manage Docker containers (i.e., belongs to the dockergroup), and install and configure the Gitlab runner.
  • Docker installed on VM: Docker Engine must be installed and running on Linux VM.
  • Application Dockerfile: A Dockerfile in the project’s root directory that correctly builds the application’s Docker image.

Project setup

new

  1. Create a new group to invite team collaborators.

  2. Create/Import Project

    In the same interface, choose Create new project/repository. We can either Create blank project or Import project from our existing Github repo. When creating the new project, make it under the group namespace we created in #1.

  3. Sync to Gitlab repo

    If we choose to create a blank project, we will push our local code to the new repo manually. We need to update the Git remote to point the local repo to the new GitLab repo instead of the old GitHub one. Here are the steps:

    • Get your new GitLab repo URL in its project page, it should look something like https://gitlab.com/your-namespace/your-project.git
    • In the local project folder, see what branch we’re on with git branch. If it is not main, rename it with git branch -M main
    • List the available remotes and their URLs: git remote -v
    • If you want to keep the Github remote for future use:
      • rename it as a secondary with git remote rename <remote-name> github.
      • create a new remote entry to the new Gitlab repo with git remote add origin <gitlab-url>
    • To completely remove the Github remote, replace it with Gitlab: git remote set-url <remote-name> <gitlab-url> to update it.
    • Check that the remotes are correctly setup with git remote -v
    • Push your existing code to GitLab with git push -u origin main. Git will then redirect to Gitlab login screen to authenticate for the first time. It then pushes the current local branch main to the remote called origin, and sets the upstream or tracking branch so future git pull, fetch or push commands will target origin/main automatically.

GitLab CI/CD components

GitLab CI/CD uses a file named .gitlab-ci.yml in the root of the repository to define a pipeline. This file tells GitLab Runner what to do when changes are pushed to the repository. Here are the components in a pipeline:

  • Stages: A pipeline is composed of stages (e.g., build, test, deploy). Stages run sequentially. If you don’t define a stages: array at all, GitLab falls back to its default stage list in this order:

    .pre
build
test
deploy
.post

    .pre and .post are special hidden stages for setup/cleanup and run before/after everything else.

    You can define your own stages: array to customize an ordered list of phases your pipeline will run through. This will replace GitLab’s default stage ordering (except .pre and .post, which still work without being listed).

  • Jobs: Each stage contains one or more jobs. A job defines a set of commands to execute. Jobs without a stage: key automatically land in the default test stage. If you do set stage: on a job, GitLab will accept any stage name that appears in its internal default list (above), even if you didn’t explicitly declare it.

    Jobs in the same stage run in parallel, as long as there are enough runners available and no needs: or dependencies: to force a specific order. needs: lets you run jobs earlier than their stage would normally allow. For example, you could run a deploy job while test is still running if it only depends on build.

  • Runners: GitLab uses Runners to execute jobs. These are machines (virtual or physical) that pick up jobs from GitLab. GitLab provides shared runners, or we can register our own private runners in our VM. In this post, we will employ BOTH: We will leverage shared runners in the GitLab cloud to build, and a self-hosted runner on the local VM to deploy.

Building the Docker Image

  1. Create .gitlab-ci.yml

    In the root of your GitLab project, create a new file named .gitlab-ci.yml. You can do it in 3 different ways

    • click + above the repo directory and select new file
    • on the right Project Information pane, select Setup CI/CD -> Configure Pipeline. The Pipeline Editor opens with default build/test/deploy stages that only echo some messages.
    • the best way: on the left Project pane, select Build/Pipelines. Under Ready to set up CI/CD for your project?, select the right template. Since we are planning to build and deploy Docker containers, scroll to Docker and click Use Template. The Pipeline Editor opens with working scripts for logging in, building and pushing to the registry.

  2. Define stages

    Assuming that we start from scratch, we will define our stages array. For this workflow, build and deploy will suffice. This tells GitLab:

    1. Run all jobs in build stage first.

    2. If they pass, run all jobs in deploy stage.

      stages:
  - build
  - deploy

  3. Define Build job in .gitlab-ci.yml

    Add a job named build_image to the build stage. This job will login to the project’s Container Registry, build an image with 2 tags, and push them to Container Registry. It will only run if certain files are changed and checked in to specific branches.

    # Define the stages of your CI/CD pipeline
stages:
  - build
  - deploy

# Job to build and push the Docker image to the GitLab Container Registry
build_image:
  stage: build
  image: docker:git
  services:
  - docker:dind
  script:
    - echo "Logging into GitLab Container Registry..."
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - echo "Building Docker image, tag with comit SHA and latest..."
    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA -t $CI_REGISTRY_IMAGE:latest .
    - echo "Pushing both tagged versions of image to Container Registry..."
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest
  rules:
      # job runs if commit to main or master branch
      - if: '$CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH == "master"'
      # if any of these paths changed
        changes:
        - Dockerfile
        - requirements.txt
        - helloworld/**  
        - web/**  
      # allow Manual trigger fallback 
      - when: manual

    Let’s go over each line:

    • stage: build: tells GitLab which phase from stages: array this job runs in.

    • image: docker:git : runs the CI job with docker CLI

    • services: docker:dind uses the Docker-in-Docker service with the Docker daemon and runtime to build images within the GitLab Runner environment.

    • docker login logs in to the GitLab Container Registry using predefined environment variables (CI_REGISTRY, CI_REGISTRY_USER, CI_REGISTRY_PASSWORD).

      • CI_REGISTRY: The URL of our GitLab Container Registry.
      • CI_REGISTRY_USER: GitLab username for the registry.
      • CI_REGISTRY_PASSWORD: GitLab password or a Personal Access Token with read_registry and write_registry scopes. > GitLab injects CI_REGISTRY_USER and CI_REGISTRY_PASSWORD automatically for projects with a Container Registry enabled, and if we are using shared runners and building within GitLab. >

    • docker build builds our Docker image. It’s common to publish two tags for the same image:

      • Immutable, unique tag: $CI_COMMIT_SHORT_SHA for traceability. Every commit gets a different SHA (shortened to 8 characters), so our image tag is guaranteed to be unique. We can use this tag to pull exactly the build that came from a given commit. Unlike latest which constantly moves, a SHA tag will never be overwritten so we can roll back to it reliably.
      • Mutable, convenience tag: latest (or branch name) so team members and automation can easily pull the newest build for quick test , > Again, Gitlab will automatically inject CI_REGISTRY_IMAGE with the full path to our Docker image in the registry (e.g., registry.gitlab.com/your-group/your-project). >

    • docker push publishes the built image to the GitLab Container Registry.

    • rules lists conditions under which this job will execute.

      • if: limits the job run on main or master branch
      • changes: means it only runs if any of those files or folders changed. For example, if we only edit the .gitlab-ci.yml and commit, the pipeline will not execute the build_image job.
      • when: tells Gitlab if none of the above rules match, schedules the job in a manual state. The whole pipeline pauses until someone triggers it. Go to the Pipelines page and click your pipeline. You’ll see your job with a ▶ Run button. Click that, and the job will start immediately without needing a new commit.

      manual manual jobs

      The reason for skipping unnecessary builds is to save both our time and cost, as GitLab’s free tier provides only 400 CI/CD minutes per month per namespace on GitLab’s shared runners. Using rules save build minutes for only app code or the Docker build context changes, not pure CI config edits.

Pushing to Container Registry

The docker push commands in the build_image job handle this automatically. Once pushed, you can view your images in GitLab under Deployments > Container Registry.

When the pipeline is run for the first time, Gitlab will verify your account with phone and a tedious CAPCHA program. This is enforced per account, not per project.

Deploying to Linux VM

Deploying to the Linux VM requires a way for GitLab CI/CD to connect to our VM and execute commands. The most common and secure method is a GitLab Runner self-hosted on the Linux VM.

In this setup, GitLab sends the job definition to the VM runner, the runner executes commands locally with direct access to Docker daemon and system tools. This eliminates network latency of SSH connections from Gitlab. It also provides safer secrets handling as secrets stay on the VM (via local env vars or config), not in GitLab. Jobs on our own runners also consume zero of our free 400 CI/CD minutes.

Here are the steps:

  1. Install GitLab Runner on VM

    In Ubuntu, install dependencies

    sudo apt update
sudo apt install -y curl

    Add the official GitLab Runner GPG key and repo and install

    curl -fsSL https://packages.gitlab.com/gpg.key | sudo gpg --dearmor -o /usr/share/keyrings/gitlab-runner-archive-keyring.gpg

echo "deb [signed-by=/usr/share/keyrings/gitlab-runner-archive-keyring.gpg] https://packages.gitlab.com/runner/gitlab-runner/ubuntu/ noble main" \
  | sudo tee /etc/apt/sources.list.d/gitlab-runner.list

sudo apt install -y gitlab-runner

    Once done, verify installation by gitlab-runner --version

    For Oracle Linux or any CentOS/RHEL-based image, use a similar process with rpm or yum/dnf commands:

    curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.rpm.sh" | sudo bash
sudo yum install gitlab-runner
# Or for newer CentOS/RHEL versions
# sudo dnf install gitlab-runner

  2. Register runner with GitLab project

    Register it with our GitLab instance on the VM with sudo gitlab-runner register

    Answer the prompts:

    • GitLab instance URL: https://gitlab.com/
    • Registration token: This binds it to a scope (project or group):
      • Use a Project token if you want this runner only for this project. We will use this to keep the runner private to our repo. In GitLab.com project’s left pane: Settings → CI/CD → Runners → New project runner → copy the registration token and paste here.
      • Use a Group token (Group → Settings → CI/CD → Runners) if you want one runner for all projects in that group.
    • Description: e.g. oci-vm-runner. Provide a descriptive name to identify it in GitLab.
    • categories: oci,deploy. This is crucial. Jobs in .gitlab-ci.yml will use these tags to select this specific runner.
    • Executor: Defines how the runner executes jobs.
      • shell: Runs directly on the VM. Best for local deploys that need access to Docker, systemd, files, Nginx, etc. We will pick this as it’s simplest for deployments to our VM using local Docker.
      • docker: Runs each job inside an isolated container on this VM. Good for clean, reproducible build/test environments. For Docker‑in‑Docker builds, you’ll need privileged mode and to pick a Default Docker image, e.g., alpine:3.20 for light jobs, or docker:24 for Docker CLI out of the box.

  3. Setup permission for runner

    The runner will need appropriate permissions to execute Docker commands, so we will add the runner user to docker group. Then we’ll restart the runner service so group membership takes effect.

    sudo usermod -aG docker gitlab-runner
sudo systemctl restart gitlab-runner

  4. Verify runner status

    On the VM, run sudo gitlab-runner list. You should see your new runner. You can always check the service status on the VM with sudo systemctl status gitlab-runner.

    If it’s inactive or failed, run sudo systemctl enable --now gitlab-runner. This ensures it runs now and starts on every boot.

    The new runner will also show up as “online” in Gitlab’s Project Settings → CI/CD → Runners page. (Note: you need to refresh the Runner page to see it updated).

    Note: The runner must be able to make outbound HTTPS requests to gitlab.com:443. On OCI Portal, make sure your instance’s public subnet has enabled outbound HTTPS in the security list. Do the same for your VM’s firewall rules.

  5. Define Deploy job in .gitlab-ci.yml

    Add a deploy_to_vm job to the deploy stage. This job will target our local runner using at least one tags that matches one of the runner’s tags. The script section would directly execute the deployment commands, which

    • login to the registry, similar to the build job
    • pull the image, similar to the push in build
    • if the local docker daemon is running a container with the same name, stop and remove it.
    • run the container with port mapping using the pulled image
    # ... (previous stages and build_image job) ...
deploy_to_vm:
  stage: deploy
  categories: [ "oci", "deploy" ]   # Assign this job to your VM runner with these tags
  script:
    - echo "Deploying directly on the VM via GitLab Runner..."
    # Log in to the registry directly from the VM's runner environment
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"
    - APP_NAME="django_hello"
    - docker pull "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"
    - docker stop APP_NAME || true
    - docker rm APP_NAME || true
    - docker run -d --name APP_NAME -p 7777:8000 "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"
    - echo "Deployment complete for $APP_NAME!"

Test the setup

  1. Commit and Push: Commit the updated .gitlab-ci.yml file to our GitLab repository and push it to the main or master branch.

  2. Monitor Pipeline: Go to CI/CD > Pipelines in GitLab. A new pipeline should trigger, running only the deploy job (since we didn’t change the source code).

  3. Check Job Logs: Monitor the deploy_to_vm job’s logs. You should see it picked up by the new runner and executing the Docker commands directly on the VM.

Complete .gitlab-ci.yml Pipeline:

stages:
  - build
  - deploy

build_image:
  stage: build
  image: docker:git
  services:
  - docker:dind
  script:
    - echo "Logging into GitLab Container Registry..."
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - echo "Building Docker image with commit SHA and latest tags..."
    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA -t $CI_REGISTRY_IMAGE:latest .
    - echo "Pushing Docker images to Container Registry..."
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest
  rules:
      # job runs if commit to main or master branch
      - if: '$CI_COMMIT_BRANCH == "main" || $CI_COMMIT_BRANCH == "master"'
      # if any of these paths changed
        changes:
        - Dockerfile
        - requirements.txt
        - helloworld/**  
        - web/**  
      # allow Manual trigger fallback 
      - when: manual

# Job to deploy the Docker image to the Linux VM

deploy_to_vm:
     stage: deploy
     categories: [ "oci", "deploy" ]   # Assign this job to your VM runner with these tags
     script:
       - echo "Deploying directly on the VM via GitLab Runner..."
       # Log in to the registry directly from the VM's runner environment
       - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"
       - APP_NAME="django_hello"
       - docker pull "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"
       - docker stop APP_NAME || true
       - docker rm APP_NAME || true
       - docker run -d --name APP_NAME -p 7777:8000 "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"
       - echo "Deployment complete for $APP_NAME!"

Appendix: Common tasks

Rerun a past pipeline

If you just want to repeat a failed run, go to the Pipelines list, click the pipeline ID, and hit Retry to re-execute the pipeline. Alternatively, hit the Refresh icon of a job to run a single job only.

rerun

Edit a private runner’s setting

You can edit the runner’s settings in 2 ways

  • in the VM, by sudo nano /etc/gitlab-runner/config.toml
  • in Gitlab’s Project Settings → CI/CD → Runners page, by clicking the pencil edit button next to the runner
    • Can edit tags and descriptions
    • Lock to current project: Yes, so the runner can’t be used elsewhere.
    • Run untagged jobs: Off, if you only want tagged jobs (safer).
    • Protected: On, if you only want it to run on protected branches/tags (e.g., main).
    • Privileged (docker executor only): Required for Docker‑in‑Docker builds.
Back to top