I am running Wind River Linux LTS and I am trying to add support to BNXT_EN series NICs. The issue I am facing is that the machine is not detecting nics when the dpdk application is started. Kindly help me troubleshoot this issue. I did google a lot about this. I thought NICs should bound to userspace for them to work (igb_uio) tried to bind manually using dpdk-devbind.py, but the NIC disappears when I do that. Binding it back to bnxt_en, makes it come up, but not work. Thanks in advance. Following are some details that might help:
Edit-1
- As you can see from below, librte_pmd_bnxt.so is built and loaded
- I did set the log level to 8 and dpdk logs are added below.
bash# find . -name "librte*bnxt*"
./usr/lib64/dpdk-18/librte_pmd_bnxt.so.2.1
bash#
bash# cat /sys/bus/pci/drivers/bnxt_en/
0000:00:03.0/ 0000:00:06.0/ module/ remove_id unbind
0000:00:05.0/ bind new_id uevent
After binding them to igb_uio manually:
bash# cat /sys/bus/pci/drivers/igb_uio/
0000:00:03.0/ 0000:00:06.0/ module/ remove_id unbind
0000:00:05.0/ bind new_id uevent
bash# cat /mnt/disk0/dpdk.log
03-02-2021 10:49:43.406347 EAL: Support maximum 128 logical core(s) by configuration.
03-02-2021 10:49:43.406352 EAL: Detected 16 lcore(s)
03-02-2021 10:49:43.406365 EAL: Detected 1 NUMA nodes
03-02-2021 10:49:43.406664 EAL: open shared lib /usr/lib64/dpdk-18/librte_pmd_ixgbe.so.2.1
03-02-2021 10:49:43.407327 EAL: open shared lib /usr/lib64/dpdk-18/librte_pmd_e1000.so.1.1
03-02-2021 10:49:43.409000 EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
03-02-2021 10:49:43.409957 EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or directory)
03-02-2021 10:49:43.409963 EAL: VFIO PCI modules not loaded
03-02-2021 10:49:43.411620 EAL: No free hugepages reported in hugepages-1048576kB
03-02-2021 10:49:43.411767 EAL: Probing VFIO support...
03-02-2021 10:49:43.411777 EAL: Module /sys/module/vfio not found! error 2 (No such file or directory)
03-02-2021 10:49:43.411780 EAL: VFIO modules not loaded, skipping VFIO support...
03-02-2021 10:49:43.411792 EAL: Ask a virtual area of 0x2e000 bytes
03-02-2021 10:49:43.411813 EAL: Virtual area found at 0x100000000 (size = 0x2e000)
03-02-2021 10:49:43.411954 EAL: Setting up physically contiguous memory...
03-02-2021 10:49:43.411964 EAL: Setting maximum number of open files to 4096
03-02-2021 10:49:43.411970 EAL: Detected memory type: socket_id:0 hugepage_sz:1073741824
03-02-2021 10:49:43.411972 EAL: Detected memory type: socket_id:0 hugepage_sz:2097152
03-02-2021 10:49:43.411976 EAL: Creating 4 segment lists: n_segs:32 socket_id:0 hugepage_sz:1073741824
03-02-2021 10:49:43.411979 EAL: Ask a virtual area of 0x1000 bytes
03-02-2021 10:49:43.411983 EAL: Virtual area found at 0x10002e000 (size = 0x1000)
03-02-2021 10:49:43.411999 EAL: Memseg list allocated: 0x100000kB at socket 0
03-02-2021 10:49:43.412002 EAL: Ask a virtual area of 0x800000000 bytes
03-02-2021 10:49:45.37931 EAL: Virtual area found at 0x140000000 (size = 0x800000000)
03-02-2021 10:49:45.52277 EAL: Memseg list allocated: 0x100000kB at socket 0
03-02-2021 10:49:45.52281 EAL: Ask a virtual area of 0x800000000 bytes
03-02-2021 10:49:46.680676 EAL: Virtual area found at 0x980000000 (size = 0x800000000)
03-02-2021 10:49:46.694952 EAL: Ask a virtual area of 0x1000 bytes
03-02-2021 10:49:46.694970 EAL: Virtual area found at 0x1180000000 (size = 0x1000)
03-02-2021 10:49:46.695028 EAL: Memseg list allocated: 0x100000kB at socket 0
03-02-2021 10:49:46.695031 EAL: Ask a virtual area of 0x800000000 bytes
03-02-2021 10:49:48.305421 EAL: Virtual area found at 0x11c0000000 (size = 0x800000000)
03-02-2021 10:49:48.319722 EAL: Ask a virtual area of 0x1000 bytes
03-02-2021 10:49:48.319741 EAL: Virtual area found at 0x19c0000000 (size = 0x1000)
03-02-2021 10:49:48.319798 EAL: Memseg list allocated: 0x100000kB at socket 0
03-02-2021 10:49:48.319827 EAL: Ask a virtual area of 0x800000000 bytes
03-02-2021 10:49:49.950396 EAL: Virtual area found at 0x1a00000000 (size = 0x800000000)
03-02-2021 10:49:49.964673 EAL: Creating 32 segment lists: n_segs:1024 socket_id:0 hugepage_sz:2097152
03-02-2021 10:49:49.964684 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:49.964702 EAL: Virtual area found at 0x2200000000 (size = 0xd000)
03-02-2021 10:49:49.964809 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:49.964813 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:50.62627 EAL: Virtual area found at 0x2200200000 (size = 0x80000000)
03-02-2021 10:49:50.62681 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:50.62690 EAL: Virtual area found at 0x2280200000 (size = 0xd000)
03-02-2021 10:49:50.62782 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:50.62785 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:50.161349 EAL: Virtual area found at 0x2280400000 (size = 0x80000000)
03-02-2021 10:49:50.161395 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:50.161404 EAL: Virtual area found at 0x2300400000 (size = 0xd000)
03-02-2021 10:49:50.161493 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:50.161496 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:50.259251 EAL: Virtual area found at 0x2300600000 (size = 0x80000000)
03-02-2021 10:49:50.259294 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:50.259302 EAL: Virtual area found at 0x2380600000 (size = 0xd000)
03-02-2021 10:49:50.259393 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:50.259397 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:50.358313 EAL: Virtual area found at 0x2380800000 (size = 0x80000000)
03-02-2021 10:49:50.358354 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:50.358363 EAL: Virtual area found at 0x2400800000 (size = 0xd000)
03-02-2021 10:49:50.358447 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:50.358453 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:50.456097 EAL: Virtual area found at 0x2400a00000 (size = 0x80000000)
03-02-2021 10:49:50.456138 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:50.456146 EAL: Virtual area found at 0x2480a00000 (size = 0xd000)
03-02-2021 10:49:50.456227 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:50.456230 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:50.555212 EAL: Virtual area found at 0x2480c00000 (size = 0x80000000)
03-02-2021 10:49:50.555256 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:50.555266 EAL: Virtual area found at 0x2500c00000 (size = 0xd000)
03-02-2021 10:49:50.555341 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:50.555344 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:50.653819 EAL: Virtual area found at 0x2500e00000 (size = 0x80000000)
03-02-2021 10:49:50.653864 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:50.653874 EAL: Virtual area found at 0x2580e00000 (size = 0xd000)
03-02-2021 10:49:50.653967 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:50.653970 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:50.752922 EAL: Virtual area found at 0x2581000000 (size = 0x80000000)
03-02-2021 10:49:50.752966 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:50.752975 EAL: Virtual area found at 0x2601000000 (size = 0xd000)
03-02-2021 10:49:50.753065 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:50.753070 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:50.850820 EAL: Virtual area found at 0x2601200000 (size = 0x80000000)
03-02-2021 10:49:50.850861 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:50.850870 EAL: Virtual area found at 0x2681200000 (size = 0xd000)
03-02-2021 10:49:50.850939 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:50.850942 EAL: A[ 7668.824713] serial8250: too much work for irq4
[ 7668.826866] new_iir c1 ier 7 fcr c2 lcr 13 mcr b lsr 60 msr b0 rx 0 dll 0 dlh 7
sk a virtual area of 0x80000000 bytes
03-02-2021 10:49:50.949796 EAL: Virtual area found at 0x2681400000 (size = 0x80000000)
03-02-2021 10:49:50.949850 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:50.949860 EAL: Virtual area found at 0x2701400000 (size = 0xd000)
03-02-2021 10:49:50.949929 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:50.949932 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:51.47666 EAL: Virtual area found at 0x2701600000 (size = 0x80000000)
03-02-2021 10:49:51.47707 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:51.47716 EAL: Virtual area found at 0x2781600000 (size = 0xd000)
03-02-2021 10:49:51.47784 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:51.47788 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:51.146429 EAL: Virtual area found at 0x2781800000 (size = 0x80000000)
03-02-2021 10:49:51.146469 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:51.146477 EAL: Virtual area found at 0x2801800000 (size = 0xd000)
03-02-2021 10:49:51.146546 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:51.146551 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:51.244248 EAL: Virtual area found at 0x2801a00000 (size = 0x80000000)
03-02-2021 10:49:51.244287 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:51.244295 EAL: Virtual area found at 0x2881a00000 (size = 0xd000)
03-02-2021 10:49:51.244357 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:51.244360 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:51.343355 EAL: Virtual area found at 0x2881c00000 (size = 0x80000000)
03-02-2021 10:49:51.343396 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:51.343404 EAL: Virtual area found at 0x2901c00000 (size = 0xd000)
03-02-2021 10:49:51.343468 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:51.343472 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:51.441217 EAL: Virtual area found at 0x2901e00000 (size = 0x80000000)
03-02-2021 10:49:51.441255 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:51.441265 EAL: Virtual area found at 0x2981e00000 (size = 0xd000)
03-02-2021 10:49:51.441331 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:51.441334 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:51.540519 EAL: Virtual area found at 0x2982000000 (size = 0x80000000)
03-02-2021 10:49:51.540558 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:51.540566 EAL: Virtual area found at 0x2a02000000 (size = 0xd000)
03-02-2021 10:49:51.540635 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:51.540640 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:51.639056 EAL: Virtual area found at 0x2a02200000 (size = 0x80000000)
03-02-2021 10:49:51.639096 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:51.639106 EAL: Virtual area found at 0x2a82200000 (size = 0xd000)
03-02-2021 10:49:51.639169 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:51.639172 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:51.738309 EAL: Virtual area found at 0x2a82400000 (size = 0x80000000)
03-02-2021 10:49:51.738354 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:51.738363 EAL: Virtual area found at 0x2b02400000 (size = 0xd000)
03-02-2021 10:49:51.738427 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:51.738430 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:51.836237 EAL: Virtual area found at 0x2b02600000 (size = 0x80000000)
03-02-2021 10:49:51.836276 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:51.836286 EAL: Virtual area found at 0x2b82600000 (size = 0xd000)
03-02-2021 10:49:51.836351 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:51.836354 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:51.935570 EAL: Virtual area found at 0x2b82800000 (size = 0x80000000)
03-02-2021 10:49:51.935621 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:51.935631 EAL: Virtual area found at 0x2c02800000 (size = 0xd000)
03-02-2021 10:49:51.935706 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:51.935720 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:52.33555 EAL: Virtual area found at 0x2c02a00000 (size = 0x80000000)
03-02-2021 10:49:52.33609 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:52.33618 EAL: Virtual area found at 0x2c82a00000 (size = 0xd000)
03-02-2021 10:49:52.33685 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:52.33688 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:52.132378 EAL: Virtual area found at 0x2c82c00000 (size = 0x80000000)
03-02-2021 10:49:52.132420 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:52.132429 EAL: Virtual area found at 0x2d02c00000 (size = 0xd000)
03-02-2021 10:49:52.132502 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:52.132505 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:52.230175 EAL: Virtual area found at 0x2d02e00000 (size = 0x80000000)
03-02-2021 10:49:52.230216 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:52.230227 EAL: Virtual area found at 0x2d82e00000 (size = 0xd000)
03-02-2021 10:49:52.230295 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:52.230298 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:52.328849 EAL: Virtual area found at 0x2d83000000 (size = 0x80000000)
03-02-2021 10:49:52.328887 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:52.328895 EAL: Virtual area found at 0x2e03000000 (size = 0xd000)
03-02-2021 10:49:52.328958 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:52.328963 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:52.426649 EAL: Virtual area found at 0x2e03200000 (size = 0x80000000)
03-02-2021 10:49:52.426687 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:52.426695 EAL: Virtual area found at 0x2e83200000 (size = 0xd000)
03-02-2021 10:49:52.426756 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:52.426759 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:52.525606 EAL: Virtual area found at 0x2e83400000 (size = 0x80000000)
03-02-2021 10:49:52.525645 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:52.525653 EAL: Virtual area found at 0x2f03400000 (size = 0xd000)
03-02-2021 10:49:52.525718 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:52.525721 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:52.623328 EAL: Virtual area found at 0x2f03600000 (size = 0x80000000)
03-02-2021 10:49:52.623368 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:52.623380 EAL: Virtual area found at 0x2f83600000 (size = 0xd000)
03-02-2021 10:49:52.623447 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:52.623450 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:52.722537 EAL: Virtual area found at 0x2f83800000 (size = 0x80000000)
03-02-2021 10:49:52.722589 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:52.722598 EAL: Virtual area found at 0x3003800000 (size = 0xd000)
03-02-2021 10:49:52.722669 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:52.722672 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:52.821293 EAL: Virtual area found at 0x3003a00000 (size = 0x80000000)
03-02-2021 10:49:52.821339 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:52.821348 EAL: Virtual area found at 0x3083a00000 (size = 0xd000)
03-02-2021 10:49:52.821423 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:52.821426 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:52.920409 EAL: Virtual area found at 0x3083c00000 (size = 0x80000000)
03-02-2021 10:49:52.920452 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:52.920461 EAL: Virtual area found at 0x3103c00000 (size = 0xd000)
03-02-2021 10:49:52.920535 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:[ 7668.970809] serial8250: too much work for irq4
[ 7668.972922] new_iir c1 ier 7 fcr c2 lcr 13 mcr b lsr 60 msr b0 rx 0 dll 0 dlh 7
49:52.920538 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:53.18368 EAL: Virtual area found at 0x3103e00000 (size = 0x80000000)
03-02-2021 10:49:53.18413 EAL: Ask a virtual area of 0xd000 bytes
03-02-2021 10:49:53.18435 EAL: Virtual area found at 0x3183e00000 (size = 0xd000)
03-02-2021 10:49:53.18511 EAL: Memseg list allocated: 0x800kB at socket 0
03-02-2021 10:49:53.18514 EAL: Ask a virtual area of 0x80000000 bytes
03-02-2021 10:49:53.117344 EAL: Virtual area found at 0x3184000000 (size = 0x80000000)
03-02-2021 10:49:53.117674 EAL: Trying to obtain current memory policy.
03-02-2021 10:49:53.403560 EAL: Hugepage /hugepages/rtemap_55 is on socket 0
03-02-2021 10:49:53.403562 EAL: Hugepage /hugepages/rtemap_54 is on socket 0
03-02-2021 10:49:53.403564 EAL: Hugepage /hugepages/rtemap_53 is on socket 0
03-02-2021 10:49:53.403567 EAL: Hugepage /hugepages/rtemap_52 is on socket 0
03-02-2021 10:49:53.403569 EAL: Hugepage /hugepages/rtemap_51 is on socket 0
03-02-2021 10:49:53.403572 EAL: Hugepage /hugepages/rtemap_50 is on socket 0
03-02-2021 10:49:53.403574 EAL: Hugepage /hugepages/rtemap_49 is on socket 0
03-02-2021 10:49:53.403576 EAL: Hugepage /hugepages/rtemap_48 is on socket 0
03-02-2021 10:49:53.403579 EAL: Hugepage /hugepages/rtemap_47 is on socket 0
03-02-2021 10:49:53.403581 EAL: Hugepage /hugepages/rtemap_46 is on socket 0
03-02-2021 10:49:53.403600 EAL: Hugepage /hugepages/rtemap_45 is on socket 0
03-02-2021 10:49:53.403603 EAL: Hugepage /hugepages/rtemap_44 is on socket 0
03-02-2021 10:49:53.403605 EAL: Hugepage /hugepages/rtemap_43 is on socket 0
03-02-2021 10:49:53.403607 EAL: Hugepage /hugepages/rtemap_42 is on socket 0
03-02-2021 10:49:53.403610 EAL: Hugepage /hugepages/rtemap_41 is on socket 0
03-02-2021 10:49:53.403612 EAL: Hugepage /hugepages/rtemap_40 is on socket 0
03-02-2021 10:49:53.403614 EAL: Hugepage /hugepages/rtemap_39 is on socket 0
03-02-2021 10:49:53.403616 EAL: Hugepage /hugepages/rtemap_38 is on socket 0
03-02-2021 10:49:53.403619 EAL: Hugepage /hugepages/rtemap_37 is on socket 0
03-02-2021 10:49:53.403621 EAL: Hugepage /hugepages/rtemap_36 is on socket 0
03-02-2021 10:49:53.403624 EAL: Hugepage /hugepages/rtemap_35 is on socket 0
03-02-2021 10:49:53.403626 EAL: Hugepage /hugepages/rtemap_34 is on socket 0
03-02-2021 10:49:53.403643 EAL: Hugepage /hugepages/rtemap_33 is on socket 0
03-02-2021 10:49:53.403645 EAL: Hugepage /hugepages/rtemap_32 is on socket 0
03-02-2021 10:49:53.403648 EAL: Hugepage /hugepages/rtemap_31 is on socket 0
03-02-2021 10:49:53.403650 EAL: Hugepage /hugepages/rtemap_30 is on socket 0
03-02-2021 10:49:53.403653 EAL: Hugepage /hugepages/rtemap_29 is on socket 0
03-02-2021 10:49:53.403655 EAL: Hugepage /hugepages/rtemap_28 is on socket 0
03-02-2021 10:49:53.403657 EAL: Hugepage /hugepages/rtemap_27 is on socket 0
03-02-2021 10:49:53.403659 EAL: Hugepage /hugepages/rtemap_26 is on socket 0
03-02-2021 10:49:53.403662 EAL: Hugepage /hugepages/rtemap_25 is on socket 0
03-02-2021 10:49:53.403665 EAL: Hugepage /hugepages/rtemap_24 is on socket 0
03-02-2021 10:49:53.403667 EAL: Hugepage /hugepages/rtemap_23 is on socket 0
03-02-2021 10:49:53.403689 EAL: Hugepage /hugepages/rtemap_22 is on socket 0
03-02-2021 10:49:53.403691 EAL: Hugepage /hugepages/rtemap_21 is on socket 0
03-02-2021 10:49:53.403694 EAL: Hugepage /hugepages/rtemap_20 is on socket 0
03-02-2021 10:49:53.403696 EAL: Hugepage /hugepages/rtemap_19 is on socket 0
03-02-2021 10:49:53.403698 EAL: Hugepage /hugepages/rtemap_18 is on socket 0
03-02-2021 10:49:53.403701 EAL: Hugepage /hugepages/rtemap_17 is on socket 0
03-02-2021 10:49:53.403704 EAL: Hugepage /hugepages/rtemap_16 is on socket 0
03-02-2021 10:49:53.403706 EAL: Hugepage /hugepages/rtemap_15 is on socket 0
03-02-2021 10:49:53.403708 EAL: Hugepage /hugepages/rtemap_14 is on socket 0
03-02-2021 10:49:53.403710 EAL: Hugepage /hugepages/rtemap_13 is on socket 0
03-02-2021 10:49:53.403712 EAL: Hugepage /hugepages/rtemap_12 is on socket 0
03-02-2021 10:49:53.403767 EAL: Hugepage /hugepages/rtemap_11 is on socket 0
03-02-2021 10:49:53.403770 EAL: Hugepage /hugepages/rtemap_10 is on socket 0
03-02-2021 10:49:53.403772 EAL: Hugepage /hugepages/rtemap_9 is on socket 0
03-02-2021 10:49:53.403774 EAL: Hugepage /hugepages/rtemap_8 is on socket 0
03-02-2021 10:49:53.403776 EAL: Hugepage /hugepages/rtemap_7 is on socket 0
03-02-2021 10:49:53.403779 EAL: Hugepage /hugepages/rtemap_6 is on socket 0
03-02-2021 10:49:53.403781 EAL: Hugepage /hugepages/rtemap_5 is on socket 0
03-02-2021 10:49:53.403784 EAL: Hugepage /hugepages/rtemap_4 is on socket 0
03-02-2021 10:49:53.403786 EAL: Hugepage /hugepages/rtemap_3 is on socket 0
03-02-2021 10:49:53.403788 EAL: Hugepage /hugepages/rtemap_2 is on socket 0
03-02-2021 10:49:53.403790 EAL: Hugepage /hugepages/rtemap_1 is on socket 0
03-02-2021 10:49:53.403793 EAL: Hugepage /hugepages/rtemap_0 is on socket 0
03-02-2021 10:49:53.404818 EAL: Requesting 59 pages of size 2MB from socket 0
03-02-2021 10:49:53.405062 EAL: Attempting to map 118M on socket 0
03-02-2021 10:49:53.407277 EAL: Allocated 118M on socket 0
03-02-2021 10:49:56.136181 EAL: Added 118M to heap on socket 0
03-02-2021 10:49:56.236341 EAL: TSC frequency is ~1995315 KHz
03-02-2021 10:49:56.236396 EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !
03-02-2021 10:49:56.236513 EAL: Master lcore 0 is ready (tid=7fc631837480;cpuset=[0])
03-02-2021 10:49:56.243340 EAL: lcore 7 is ready (tid=7fc62dff0700;cpuset=[6])
03-02-2021 10:49:56.243747 EAL: lcore 6 is ready (tid=7fc62e0f1700;cpuset=[5])
03-02-2021 10:49:56.243851 EAL: lcore 5 is ready (tid=7fc62e1f2700;cpuset=[4])
03-02-2021 10:49:56.244261 EAL: lcore 8 is ready (tid=7fc62deef700;cpuset=[7])
03-02-2021 10:49:56.244362 EAL: lcore 4 is ready (tid=7fc62e2f3700;cpuset=[3])
03-02-2021 10:49:56.244746 EAL: lcore 9 is ready (tid=7fc62ddee700;cpuset=[8])
03-02-2021 10:49:56.244845 EAL: lcore 10 is ready (tid=7fc62dced700;cpuset=[9])
03-02-2021 10:49:56.245264 EAL: lcore 3 is ready (tid=7fc62e3f4700;cpuset=[2])
03-02-2021 10:49:56.245358 EAL: lcore 11 is ready (tid=7fc62dbec700;cpuset=[10])
03-02-2021 10:49:56.245667 EAL: lcore 12 is ready (tid=7fc62daeb700;cpuset=[11])
03-02-2021 10:49:56.245771 EAL: lcore 13 is ready (tid=7fc62d9ea700;cpuset=[12])
03-02-2021 10:49:56.246146 EAL: lcore 14 is ready (tid=7fc62d8e9700;cpuset=[13])
03-02-2021 10:49:56.246230 EAL: lcore 2 is ready (tid=7fc62e4f5700;cpuset=[1])
03-02-2021 10:49:56.246616 EAL: lcore 16 is ready (tid=7fc62d6e7700;cpuset=[15])
03-02-2021 10:49:56.246706 EAL: lcore 15 is ready (tid=7fc62d7e8700;cpuset=[14])
03-02-2021 10:49:56.247066 EAL: lcore 1 is ready (tid=7fc62e5f6700;cpuset=[0])
03-02-2021 10:49:56.247602 EAL: Module /sys/module/vfio not found! error 2 (No such file or directory)
bash#
bash# cat /etc/*release
ID="wrlinux-small"
NAME="Wind River Linux LTS"
VERSION="10.18.44.16"
VERSION_ID="10.18.44.16"
PRETTY_NAME="Wind River Linux LTS 18.44 Update 16"
bash# ifconfig -a
eth0 Link encap:Ethernet HWaddr 02:00:17:0a:bf:f9
inet6 addr: fe80::17ff:fe0a:bff9/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6 errors:0 dropped:0 overruns:0 frame:0
TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:360 (360.0 B) TX bytes:726 (726.0 B)
eth1 Link encap:Ethernet HWaddr 02:00:17:0a:87:3c
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth2 Link encap:Ethernet HWaddr 02:00:17:0a:3b:71
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
bash#
bash# ls -l /sys/bus/pci/devices/
lrwxrwxrwx 1 root root 0 Mar 2 04:59 0000:00:03.0 -> ../../../devices/pci0000:00/0000:00:03.0
lrwxrwxrwx 1 root root 0 Mar 2 04:59 0000:00:05.0 -> ../../../devices/pci0000:00/0000:00:05.0
lrwxrwxrwx 1 root root 0 Mar 2 04:59 0000:00:06.0 -> ../../../devices/pci0000:00/0000:00:06.0
bash#
bash# ethtool -i eth0 | grep bus-info
bus-info: 0000:00:03.0
bash# ethtool -i eth1 | grep bus-info
bus-info: 0000:00:05.0
bash# ethtool -i eth2 | grep bus-info
bus-info: 0000:00:06.0
bash#
bash# lspci -v
00:03.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme-E Ethernet Virtual Function
Subsystem: Oracle/SUN Device 16d7
Physical Slot: 3
Flags: bus master, fast devsel, latency 0
Memory at fe300000 (64-bit, prefetchable) [size=16K]
Memory at fe000000 (64-bit, prefetchable) [size=1M]
Memory at fe304000 (64-bit, prefetchable) [size=16K]
Capabilities: [a0] MSI-X: Enable+ Count=4 Masked-
Capabilities: [ac] Express Endpoint, MSI 00
Kernel driver in use: bnxt_en
00:05.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme-E Ethernet Virtual Function
Subsystem: Oracle/SUN Device 16d7
Physical Slot: 5
Flags: bus master, fast devsel, latency 0
Memory at fe30c000 (64-bit, prefetchable) [size=16K]
Memory at fe100000 (64-bit, prefetchable) [size=1M]
Memory at fe310000 (64-bit, prefetchable) [size=16K]
Capabilities: [a0] MSI-X: Enable+ Count=4 Masked-
Capabilities: [ac] Express Endpoint, MSI 00
Kernel driver in use: bnxt_en
00:06.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme-E Ethernet Virtual Function
Subsystem: Oracle/SUN Device 16d7
Physical Slot: 6
Flags: bus master, fast devsel, latency 0
Memory at fe314000 (64-bit, prefetchable) [size=16K]
Memory at fe200000 (64-bit, prefetchable) [size=1M]
Memory at fe318000 (64-bit, prefetchable) [size=16K]
Capabilities: [a0] MSI-X: Enable+ Count=4 Masked-
Capabilities: [ac] Express Endpoint, MSI 00
Kernel driver in use: bnxt_en
bash# python3 dpdk-devbind.py --status
Network devices using DPDK-compatible driver
============================================
<none>
Network devices using kernel driver
===================================
0000:00:03.0 'NetXtreme-E Ethernet Virtual Function' if=eth0 drv=bnxt_en unused=igb_uio
0000:00:05.0 'NetXtreme-E Ethernet Virtual Function' if=eth1 drv=bnxt_en unused=igb_uio
0000:00:06.0 'NetXtreme-E Ethernet Virtual Function' if=eth2 drv=bnxt_en unused=igb_uio
Other network devices
=====================
<none>
Crypto devices using DPDK-compatible driver
===========================================
<none>
Crypto devices using kernel driver
==================================
<none>
Other crypto devices
====================
<none>
bash# python3 dpdk-devbind.py -b igb_uio 00:06.0
bash# ifconfig -a
eth0 Link encap:Ethernet HWaddr 02:00:17:04:f9:13
inet addr:10.100.0.38 Bcast:10.100.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:859 errors:0 dropped:0 overruns:0 frame:0
TX packets:783 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:152379 (148.8 KiB) TX bytes:71516 (69.8 KiB)
eth1 Link encap:Ethernet HWaddr 02:00:17:07:1b:5b
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
bash# python3 dpdk-devbind.py -b bnxt_en 00:06.0
bash# ifconfig -a
eth0 Link encap:Ethernet HWaddr 02:00:17:04:f9:13
inet addr:10.100.0.38 Bcast:10.100.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:870 errors:0 dropped:0 overruns:0 frame:0
TX packets:792 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:153065 (149.4 KiB) TX bytes:72030 (70.3 KiB)
eth1 Link encap:Ethernet HWaddr 02:00:17:07:1b:5b
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth2 Link encap:Ethernet HWaddr 02:00:17:0a:5b:58
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
bash#
dmesg:
Based on the information from the logs, the application is built with shared library mode. Currently, it opens and uses
ixgbe and e1000In order to make this work for bnxt PMD, one would need to pass
-d librte_pmd_bnxt.so. This can be done byrte_eal_inituses argc and argv-lrte_rte_bnxt.note: requested for information on DPDK version and application details (is it sample, project, custom). based on the logs, this looks like 18.11