How to define static IP address for coreos in VMware ESXi via config-drive

Question

I am deploying several coreos instances on VMware ESXi. etcd seems to be quite finicky with IP addresses, especially if they change.

I would like to be able to define a static IP address via cloud-config in a way that does not confuse the etcd cluster. It looks like what needs to happen during coreos booup is that it needs to bring up the interface with the static IP address first before starting etcd. I tried to do this with this section of my user_data file:

write_files:
    - path: /etc/systemd/network/static.network
      permissions: 0644
      content: |
        [Match]
        Name=ens192

        [Network]
        Address=192.168.1.58/24
        Gateway=192.168.1.1
        DNS=192.168.1.42

What happens is that the coreos-cloudinit process will write this file to the filesystem during boot, but only after it first brings up the network interface with using DHCP. etcd will then start but will be confused since the user_data addr and peer-addr settings do not match DHCP. If I manually reboot the system after it first boots up, it will then begin to use the static IP address that was recorded earlier in the static.network file. Is there away to avoid this second reboot, i.e. start the network after defining static.network?

It seems that the root issue may just be that the vmware provider does not understand the $public_ip4 construct used by other providers such as vagrant:

  etcd:
    # generate a new token for each unique cluster from https://discovery.etcd.io/new
    # WARNING: replace each time you 'vagrant destroy'
    discovery: https://discovery.etcd.io/####
    addr: $public_ipv4:4001
    peer-addr: $public_ipv4:7001

From what I can tell, this issue does not exist when using vagrant/virtualbox.

Answer 1

The way to define the networking for a CoreOS box is using a systemd unit file. Here is the general idea of the one I use:

coreos:
...
  units:
...
    - name: 00-eno1.network
      runtime: true
      content: |
        [Match]
        Name=eno1

        [Network]
        DNS=10.0.0.10
        DNS=10.1.0.10
        Domains=domain.local
        Address=10.0.0.20/24
        Gateway=10.0.0.1
...

Just replace the file content with yours. CoreOS and systemd will handle configuring the static networking prior to doing anything else, such as bringing up etcd.