Create Azure linux VHD to use in Pipeline

Question

I would like to prepare an ubuntu image with preinstalled dependencies (python, go, etc.) so that the pipelines do not waste times installing them each time. I expected to find some kind of Dockerfile-like interface, but I have not been able to find an easy way to accomplish this.

What I understood is that I need to create a new virtual machine scale set with this image and then use it for the agent pool. But how do I create the vhd image?

Answer 1

You can use one of the below tools to build images on Azure.

1. packer - more industry-standard across clouds (like Terraform from Hashicorp)
2. Azure Image Builder - works only on Azure (from Microsoft)

Both of the above tools can be used to customize base (golden) images available on the Azure marketplace or if you have an existing VM image on-prem, it can be uploaded following documentation here.

If you have multiple subscriptions & you would like to build on one subscription & use them on other subscriptions/other tenants, you would need to use Azure Compute Gallery. This gallery also helps you to version VM images like docker images. In short, it's like a docker registry for VM images.

If you would like to use existing marketplace images (base Ubuntu in your case), you don't need to convert them into VHD. Packer/Azure Image Builder would basically provision a VM out of the base image, install packages (customization of your choice), creates a snapshot of the VM & creates an image out of that.

With packer, you would be writing a JSON builder like the below. Source code is taken from here.

{
  "builders": [{
    "type": "azure-arm",

    "client_id": "f5b6a5cf-fbdf-4a9f-b3b8-3c2cd00225a4",
    "client_secret": "0e760437-bf34-4aad-9f8d-870be799c55d",
    "tenant_id": "72f988bf-86f1-41af-91ab-2d7cd011db47",
    "subscription_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx",

    "managed_image_resource_group_name": "myResourceGroup",
    "managed_image_name": "myPackerImage",

    "os_type": "Linux",
    "image_publisher": "Canonical",
    "image_offer": "UbuntuServer",
    "image_sku": "16.04-LTS",

    "azure_tags": {
        "env": "dev",
        "task": "Image deployment with packer on Azure is fun!"
    },

    "location": "East US",
    "vm_size": "Standard_DS2_v2"
  }],
  "provisioners": [{
    "execute_command": "chmod +x {{ .Path }}; {{ .Vars }} sudo -E sh '{{ .Path }}'",
    "inline": [
      "apt-get update",
      "apt-get upgrade -y",
      "apt-get -y install nginx",
      "/usr/sbin/waagent -force -deprovision+user && export HISTSIZE=0 && sync"
    ],
    "inline_shebang": "/bin/sh -x",
    "type": "shell"
  }]
}

A suggestion from my side:: /usr/sbin/waagent -force -deprovision+user command ran at the end is extremely important when building images on the Azure as it cleans up the cloud-init/waagent user packer provisioned, ssh keys & other things to have a clean start for a customized image.