Using libvirt within Docker container

Question

I am looking to perform a simple test on my VM and see how it behaves within a container and I want to test this using libvirt and QEMU/KVM. I have a domain .xml file that runs the VM just fine on a bare metal server but I am running into issues using libvirt within the container.

My Dockerfile looks like this:

FROM ubuntu:18.04

RUN apt-get update -qy \
 && apt-get upgrade -qy \
 && apt-get install -y \
    bridge-utils \
    iproute2 \
    python3-ipy \
    socat \
    qemu-kvm \
    libvirt-daemon-system \
    libvirt-clients \
    virtinst \
    bridge-utils \
    iptables \
    iproute2 \
    dnsmasq \
    vim \
 && rm -rf /var/lib/apt/lists/*

COPY *.qcow2 /
COPY *.xml /

I successfully create a simple image from this Dockerfile running sudo docker build . -t foo and to run the container I use sudo docker run -i --privileged -t --rm foo /bin/bash. This executes fine but once I am inserted into my containers shell, libvirt is giving me some issues. I am using a confirmed working libvirt domain XML file but when going to define it I get these errors in virsh:

error: failed to connect to the hypervisor
error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory

On a normal environment not nested or containerized it's easier to debug issues with libvirt, usually the daemon is not enabled or running, in the case of a container I am unsure of how to exactly fix the issue. When I check the status of libvirt's daemon, I get this message

System has not been booted with systemd as init system (PID 1). Can't operate.

I assume I should be passing some arguments when running the container that has to do with my network?

Answer 1

Following up on my comment, here's a basic Dockerfile for running a containerized libvirt:

FROM ubuntu:23.04

RUN export DEBIAN_FRONTEND=noninteractive && \
    apt-get update && \
    apt-get -y install \
        bridge-utils \
        dmidecode \
        dnsmasq \
        ebtables \
        iproute2 \
        iptables \
        libvirt-clients \
        libvirt-daemon-system \
        ovmf \
        qemu-efi \
        qemu-kvm \
        tini \
        && \
    apt-get clean

RUN sed -i '/^#stdio_handler/ a\stdio_handler = "file"' /etc/libvirt/qemu.conf

COPY config/pools/* /etc/libvirt/storage/
COPY config/networks/* /etc/libvirt/qemu/networks/
RUN mkdir -p /etc/libvirt/storage/autostart /etc/libvirt/qemu/networks/autostart && \
    for pool in /etc/libvirt/storage/*.xml; do \
        ln -sf "../${pool##*/}" /etc/libvirt/storage/autostart/; \
    done && \
    for net in /etc/libvirt/qemu/networks/*.xml; do \
        ln -sf "../${net##*/}" /etc/libvirt/qemu/networks/autostart/; \
    done

CMD ["/usr/bin/tini", "/usr/sbin/libvirtd"]

In config/pools/default.xml I have:

<pool type='dir'>
  <name>default</name>
  <target>
    <path>/var/lib/libvirt/images</path>
  </target>
</pool>

And in config/networks/default.xml I have:

<network>
  <name>default</name>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr0' stp='on' delay='0'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.10' end='192.168.122.200'/>
    </dhcp>
  </ip>
</network>

If I build an image libvirtd from this Dockerfile and run it like this:

docker run -it --rm \
  -v /tmp/libvirt:/run/libvirt \
  --privileged \
  --name libvirt \
  --device /dev/kvm \
  -v /sys/fs/cgroup:/sys/fs/cgroup \
  libvirtd

Then I can do things like this:

$ virsh -c qemu:///system'?socket=/tmp/libvirt/libvirt-sock' net-list
 Name      State    Autostart   Persistent
--------------------------------------------
 default   active   yes         yes

Or even:

export LIBVIRT_DEFAULT_URI='qemu:///system?socket=/tmp/libvirt/libvirt-sock'

virsh vol-create-as --pool default jammy 653M
virsh vol-upload jammy \
  jammy-server-cloudimg-amd64.img \
  --pool default
virt-install -n jammy -r 4096 \
  --disk pool=default,size=10,backing_store=jammy,backing_format=qcow2 \
  --network network=default \
  --os-variant ubuntujammy \
  --import