Proxmox create VLAN not supported

Question

I am setting up 2 servers in a cluster running proxmox V 6.3 I have created a bridge interface vmbr0 and an trying to create vlan interfaces when I go to reload the networking I get the error error: netlink: vmbr0.500: cannot create vlan vmbr0.500 500: operation failed with 'Operation not supported' (95)

I have this setup on another server and it works fine, no issues. I am at a loss with things I have tried.

Below is my interfaces file.

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface eno1 inet manual

auto eno3
iface eno3 inet manual
        address 192.168.8.238
        gateway 192.168.8.1

iface eno2 inet manual

iface eno4 inet manual

auto vmbr0
iface vmbr0 inet manual
    bridge-ports eno1
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094


auto vmbr0.500
iface vmbr0.500 inet static
        address  172.20.0.3
        netmask  255.255.255.0
        broadcast  172.20.0.255
        network 172.20.0.0
        vlan_raw_device vmbr0

Answer 1

I suggest you first create create bond interface like:

auto bond0
iface bond0 inet manual
bond-slaves eno3  #here you should add any trunk interface of proxmox node and it is recommended to add also a second interface to have active-backup bond mode.
bond-miimon 100
bond-mode active-backup

than add bond into vlan

auto bond0.500
iface bond0.500 inet manual

Create birdge

auto vmbr500
iface vmbr500 inet static
    address 172.20.0.3/24
    # gateway # comment it if you want to have any gateway
    bridge-ports bond0.500
    bridge-stp off
    bridge-fd 0
    

change vmbr0 config to

auto vmbr0
iface vmbr0 inet manual
    bridge-ports bond0
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094

Answer 2

As Gaga wrote, first create a bond -> virtual bridge (with vlan aware option). There are examples on the following wiki page: https://pve.proxmox.com/wiki/Network_Configuration

Always use two network interfaces in the bond for redundency, I can see you have enough of them. Use seperate bonds for cluster and data traffic (ex. eno1+eno2 and eno3+eno4) - 10G if possible for cluster network.

If you are using cluster of nodes you should use a minimum of three nodes for cluster qvorum. What happens when one of your nodes goes down? How can a node deceide which one is "alive"? Maybe you are using "pvecm expected 2" which you should use only in emergency.