DEVHIDE
Login Or Sign up

Openstack Victoria SRIOV doesn't have the required PCI devices

443 Views Asked by At

I have followed the Openstack Victoria documentation to configure SRIOV (https://docs.openstack.org/neutron/victoria/admin/config-sriov.html) and I think I am missing something since when creating the port over the SRIOV network that has configured the network provider physnet3 does not find the PCI device.

My servers:

  1. controller: host02
  2. compute: hots09

My settings:

 compute host09:
 
 cat /proc/cmdline 
 BOOT_IMAGE=/vmlinuz-5.4.0-72-generic root=/dev/mapper/vg0-lv--root ro isolcpus=42-55 maybe-ubiquity intel_iommu=on iommu=pt hugepagesz=1G hugepages=32 default_hugepagesz=1G transparent_hugepage=never
 
 lspci -v | grep Mel
 02:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
 86:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
 86:00.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:00.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:00.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:00.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:00.5 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:00.6 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:00.7 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:01.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:01.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:01.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:01.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:01.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:01.5 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:01.6 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:01.7 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:02.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:02.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:02.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:02.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:02.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:02.5 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:02.6 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:02.7 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:03.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:03.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:03.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:03.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:03.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:03.5 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:03.6 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:03.7 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 86:04.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
 
 cat /etc/nova/nova.conf | grep -v "\#" | sed '/^[[:space:]]*$/d'
 [DEFAULT]
 my_ip = 
 state_path = /var/lib/nova
 enabled_apis = osapi_compute,metadata
 log_dir = /var/log/nova
 transport_url = rabbit://
 use_neutron = True
 linuxnet_interface_driver = nova.network.linux_net.LinuxBridgeInterfaceDriver
 firewall_driver = nova.virt.firewall.NoopFirewallDriver
 vif_plugging_is_fatal = True
 vif_plugging_timeout = 300
 cpu_allocation_ratio = 3.0
 reserved_huge_pages = node:0,size:1GB,count:1
 reserved_huge_pages = node:1,size:1GB,count:1
 vcpu_pin_set = 42-55
 [filter_scheduler]
 available_filters=nova.scheduler.filters.all_filters
 enabled_filters=AvailabilityZoneFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,PciPassthroughFilter,NUMATopologyFilter,AggregateInstanceExtraSpecsFilter
 [api]
 auth_strategy = keystone
 [vnc]
 enabled = True
 server_listen = 0.0.0.0
 server_proxyclient_address = $my_ip
 novncproxy_base_url = http://
 [glance]
 api_servers = http://
 [oslo_concurrency]
 lock_path = $state_path/tmp
 [keystone_authtoken]
 www_authenticate_uri = http://
 auth_url = http://
 memcached_servers = 
 auth_type = password
 project_domain_name = default
 user_domain_name = default
 project_name = service
 username = 
 password = 
 [placement]
 auth_url = http://
 os_region_name = RegionOne
 auth_type = password
 project_domain_name = default
 user_domain_name = default
 project_name = service
 username = 
 password = 
 [wsgi]
 api_paste_config = /etc/nova/api-paste.ini
 [neutron]
 auth_url = http://
 auth_type = password
 project_domain_name = default
 user_domain_name = default
 region_name = RegionOne
 project_name = service
 username = 
 password = 
 service_metadata_proxy = True
 metadata_proxy_shared_secret = 
 [cinder]
 os_region_name = RegionOne
 [pci]
 passthrough_whitelist = { "devname": "enp134s0", "physical_network": "physnet3", "trusted":"true", "devname": "enp134s0d1", "physical_network": "physnet4", "trusted":"true"}
 
 cat /etc/neutron/plugins/ml2/ml2_conf.ini | grep -v "\#" | sed '/^[[:space:]]*$/d'
 [DEFAULT]
 path_mtu = 1550
 physical_network_mtus = physnet1:1550
 [ml2]
 type_drivers = flat,vlan,vxlan
 tenant_network_types = vxlan
 mechanism_drivers = linuxbridge
 extension_drivers = port_security
 path_mtu = 1550
 segment_mtu = 1550
 physical_network_mtus = physnet1:1500
 [ml2_type_flat]
 flat_networks = *
 [ml2_type_geneve]
 [ml2_type_gre]
 [ml2_type_vlan]
 network_vlan_ranges = physnet2:1500:1520
 [ml2_type_vxlan]
 vni_ranges = 1:1000
 [ovs_driver]
 [securitygroup]
 [sriov_driver]
 
 cat /etc/neutron/plugins/ml2/sriov_agent.ini | grep -v "\#" | sed '/^[[:space:]]*$/d'
 [DEFAULT]
 [agent]
 [sriov_nic]
 physical_device_mappings = physnet3:enp134s0,physnet4:enp134s0d1
 exclude_devices = enp134s0:86:02.1;86:02.2;86:02.3;86:02.4;86:02.5;86:02.6;86:02.7;86:03.0;86:03.1;86:03.2;86:03.3;86:03.4;86:03.5;86:03.6;86:03.7;86:04.0,enp134s0d1:86:00.1;86:00.2;86:00.3;86:00.4;86:00.5;86:00.6;86:00.7;86:01.0;86:01.1;86:01.2;86:01.3;86:01.4;86:01.5;86:01.6;86:01.7;86:02.0
 [securitygroup]
 firewall_driver = neutron.agent.firewall.NoopFirewallDriver
 
 cat /etc/neutron/neutron.conf | grep -v "\#" | sed '/^[[:space:]]*$/d'
 [DEFAULT]
 core_plugin = ml2
 service_plugins = router
 auth_strategy = keystone
 state_path = /var/lib/neutron
 allow_overlapping_ips = True
 transport_url = rabbit://
 global_physnet_mtu = 1550
 [agent]
 root_helper = sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
 [keystone_authtoken]
 www_authenticate_uri = http://
 auth_url = http://
 memcached_servers = 
 auth_type = password
 project_domain_name = default
 user_domain_name = default
 project_name = service
 username = 
 password = 
 [oslo_concurrency]
 lock_path = $state_path/lock
 
 controller host02:
 
 openstack network agent list --host host09
 +--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
 | ID                                   | Agent Type         | Host         | Availability Zone | Alive | State | Binary                    |
 +--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
 | 0ee3ac1e-8ea2-4da9-8143-3a71e4de130c | Metadata agent     | host09 | None              | :-)   | UP    | neutron-metadata-agent    |
 | 32b5a152-92eb-4a37-b513-8b6cbaf7ec31 | Linux bridge agent | host09 | None              | :-)   | UP    | neutron-linuxbridge-agent |
 | 89e0a1b1-ecfe-4af2-9c4a-9f355eecd38a | NIC Switch agent   | host09 | None              | :-)   | UP    | neutron-sriov-nic-agent   |
 | b8e51a2e-de07-4d62-b6b4-cbe6d545f8cf | DHCP agent         | host09 | nova              | :-)   | UP    | neutron-dhcp-agent        |
 +--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
 
 openstack network agent list --host host02
 +--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
 | ID                                   | Agent Type         | Host         | Availability Zone | Alive | State | Binary                    |
 +--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
 | 59a0f618-a66f-44e6-9763-5d29f071d6da | Linux bridge agent | host02 | None              | :-)   | UP    | neutron-linuxbridge-agent |
 | de9daf1c-6325-4cff-ab0e-dd1be29b57e8 | L3 agent           | host02 | nova              | :-)   | UP    | neutron-l3-agent          |
 +--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
 
 cat /etc/neutron/neutron.conf | grep -v "\#" | sed '/^[[:space:]]*$/d'
 [DEFAULT]
 core_plugin = ml2
 service_plugins = router
 auth_strategy = keystone
 state_path = /var/lib/neutron
 dhcp_agent_notification = True
 allow_overlapping_ips = True
 notify_nova_on_port_status_changes = True
 notify_nova_on_port_data_changes = True
 transport_url = rabbit://
 global_physnet_mtu = 1550
 [agent]
 root_helper = sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
 [keystone_authtoken]
 www_authenticate_uri = http://
 auth_url = http://
 memcached_servers = 
 auth_type = password
 project_domain_name = default
 user_domain_name = default
 project_name = service
 username = 
 password = 
 [database]
 connection = mysql+pymysql://neutron:
 [nova]
 auth_url = http://
 auth_type = password
 project_domain_name = default
 user_domain_name = default
 region_name = RegionOne
 project_name = service
 username = nova
 password = 
 [oslo_concurrency]
 lock_path = $state_path/tmp
 
 cat /etc/neutron/plugins/ml2/ml2_conf.ini | grep -v "\#" | sed '/^[[:space:]]*$/d'
 [DEFAULT]
 path_mtu = 1550
 physical_network_mtus = physnet1:1550
 [ml2]
 type_drivers = flat,vlan,vxlan
 tenant_network_types = vxlan
 mechanism_drivers = linuxbridge,sriovnicswitch
 extension_drivers = port_security
 path_mtu = 1550
 segment_mtu = 1550
 physical_network_mtus = physnet1:1500
 [ml2_type_flat]
 flat_networks = *
 [ml2_type_geneve]
 [ml2_type_gre]
 [ml2_type_vlan]
 network_vlan_ranges = physnet2:1500:1520,physnet3:1504:1506,physnet4:1504:1506
 [ml2_type_vxlan]
 vni_ranges = 1:1000
 [ovs_driver]
 [securitygroup]
 [sriov_driver]
 
 cat /etc/nova/nova.conf | grep -v "\#" | sed '/^[[:space:]]*$/d'
 [DEFAULT]
 my_ip = 
 state_path = /var/lib/nova
 enabled_apis = osapi_compute,metadata
 log_dir = /var/log/nova
 transport_url = rabbit://
 use_neutron = True
 linuxnet_interface_driver = nova.network.linux_net.LinuxBridgeInterfaceDriver
 firewall_driver = nova.virt.firewall.NoopFirewallDriver
 vif_plugging_is_fatal = True
 vif_plugging_timeout = 300
 debug=true
 [filter_scheduler]
 available_filters=nova.scheduler.filters.all_filters
 enabled_filters=AvailabilityZoneFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAffinityFilter,PciPassthroughFilter,NUMATopologyFilter,AggregateInstanceExtraSpecsFilter
 [api]
 auth_strategy = keystone
 [glance]
 api_servers = http://
 [oslo_concurrency]
 lock_path = $state_path/tmp
 [api_database]
 connection = mysql+pymysql://
 [database]
 connection = mysql+pymysql://
 [keystone_authtoken]
 www_authenticate_uri = http://
 auth_url = http://
 memcached_servers = 
 auth_type = password
 project_domain_name = default
 user_domain_name = default
 project_name = service
 username = 
 password = 
 [placement]
 auth_url = http://
 os_region_name = RegionOne
 auth_type = password
 project_domain_name = default
 user_domain_name = default
 project_name = service
 username = 
 password = 
 [wsgi]
 api_paste_config = /etc/nova/api-paste.ini
 [neutron]
 auth_url = http://
 auth_type = password
 project_domain_name = default
 user_domain_name = default
 region_name = RegionOne
 project_name = service
 username = 
 password = 
 service_metadata_proxy = True
 metadata_proxy_shared_secret = 
 
 openstack network show 61e37950-5944-44e1-81df-5aa3a94a016f
 +---------------------------+--------------------------------------+
 | Field                     | Value                                |
 +---------------------------+--------------------------------------+
 | admin_state_up            | UP                                   |
 | availability_zone_hints   |                                      |
 | availability_zones        | nova                                 |
 | created_at                | 2021-04-19T15:52:42Z                 |
 | description               |                                      |
 | dns_domain                | None                                 |
 | id                        | 61e37950-5944-44e1-81df-5aa3a94a016f |
 | ipv4_address_scope        | None                                 |
 | ipv6_address_scope        | None                                 |
 | is_default                | None                                 |
 | is_vlan_transparent       | None                                 |
 | mtu                       | 1550                                 |
 | name                      | sriov-net-1504                       |
 | port_security_enabled     | True                                 |
 | project_id                | 2467cb8a053d40f0a0a8af43d89d1c32     |
 | provider:network_type     | vlan                                 |
 | provider:physical_network | physnet3                             |
 | provider:segmentation_id  | 1504                                 |
 | qos_policy_id             | None                                 |
 | revision_number           | 2                                    |
 | router:external           | Internal                             |
 | segments                  | None                                 |
 | shared                    | False                                |
 | status                    | ACTIVE                               |
 | subnets                   | e5dd4fa3-abe7-482c-8544-b73836c547cf |
 | tags                      |                                      |
 | updated_at                | 2021-04-19T15:59:34Z                 |
 +---------------------------+--------------------------------------+
 
 nova-scheduler.log
 2021-04-20 02:26:08.344 450488 DEBUG nova.scheduler.filters.pci_passthrough_filter [req-dec3be12-9274-4733-94a1-7e29dc4b6de2 3b1d6ae009c94f99800bb1a53cedba4a 2467cb8a053d40f0a0a8af43d89d1c32 - default default] (
 magma-host09, magma-host09) ram: 171084MB disk: 13312MB io_ops: 0 instances: 5 doesn't have the required PCI devices (InstancePCIRequests(instance_uuid=<?>,requests=[InstancePCIRequest])) host_passes /usr/lib/py
 thon3/dist-packages/nova/scheduler/filters/pci_passthrough_filter.py:52

As you can see in the log of the nova scheduler it does not find the PCI device

doesn't have the required PCI devices

I would greatly appreciate the help, regards.

openstack openstack-nova openstack-neutron
Original Q&A
1

There are 1 best solutions below

0
On

I found the error, my passthrough_whitelist on nova compute was wrong:

wrong:
passthrough_whitelist = { "devname": "enp134s0", "physical_network": "physnet3", "trusted":"true", "devname": "enp134s0d1", "physical_network": "physnet4", "trusted":"true"}

Fixed multiple values:
passthrough_whitelist = { "devname": "enp134s0", "physical_network": "physnet3", "trusted":"true" }
passthrough_whitelist = { "devname": "enp134s0d1", "physical_network": "physnet4", "trusted":"true" }

Related Questions in OPENSTACK

Related Questions in OPENSTACK-NOVA

Related Questions in OPENSTACK-NEUTRON

Trending Questions

Popular # Hahtags

javascript python java c# php android html jquery c++ css ios sql mysql r reactjs node.js arrays c asp.net json python-3.x ruby-on-rails .net sql-server swift django angular objective-c pandas excel

Popular Questions

Copyright © 2021 Jogjafile Inc.