I installed Ubuntu 20.04 on an Intel NUC (call it Beta) and applied the PREEMPT_RT patch so I can run (near) real-time tasks on it. For this to work I need <500us ping to another machine on the network, but right now it is slow and also varies quite a bit:
rtt min/avg/max/mdev = 0.148/0.829/2.360/0.224 ms
I have a similar setup running on another NUC (call it Alpha) where I get much better ping. This is running a different version of the kernel which does not work with the newer NUC (drivers seem incompatible). Comparing the output of cat /proc/interrupts | grep <interface-name> on the two, I found that on Alpha, the settings were configured such that the network interface is "mapped" to all the cores.
Alpha (fast ping)
156: 0 0 0 1 0 0 0 2 IR-PCI-MSI 46137344-edge enp88s0
157: 130151 90842 24003 123920 38044 365508 209425 160927 IR-PCI-MSI 46137345-edge enp88s0-TxRx-0
158: 44674 38892 16777 164096 24533 158460 18639 31108 IR-PCI-MSI 46137346-edge enp88s0-TxRx-1
159: 272798 111246 12902 165234 44009 107443 48298 111897 IR-PCI-MSI 46137347-edge enp88s0-TxRx-2
160: 48671 44626 32107 52454 32232 56264 202022 30701 IR-PCI-MSI 46137348-edge enp88s0-TxRx-3
Beta (slow ping)
162: 0 0 0 4 0 1 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSI 45088768-edge enp86s0
163: 0 0 0 0 0 0 7 262533 0 0 0 0 0 0 0 0 IR-PCI-MSI 45088769-edge enp86s0-TxRx-0
164: 0 0 0 0 0 0 0 10 0 0 0 0 0 0 210802 0 IR-PCI-MSI 45088770-edge enp86s0-TxRx-1
165: 0 0 0 0 0 154417 0 0 8 0 0 0 0 0 0 0 IR-PCI-MSI 45088771-edge enp86s0-TxRx-2
166: 0 0 0 0 0 0 0 0 0 9 0 265804 0 0 0 0 IR-PCI-MSI 45088772-edge enp86s0-TxRx-3
Could this be the problem? And if it is, is there a way to "auto-tune" the interrupt mapping?