I want to write a function that returns the topology of the underlying GPU devices as a graph. I want the connections to indicate where data transfer can occur, and the weights to be the throughput capacity of these connections.
I know that nvidia-smi topo -m returns an adjacency matrix which the GPUs as vertices, but it doesn't show the throughput capacity of these connections. How would I go about getting this information?
This is the output of the nvidia-smi command.
GPU0 GPU1 GPU2 GPU3 GPU4 GPU5 CPU Affinity NUMA Affinity
GPU0 X PIX PIX SYS SYS SYS 0-11 0
GPU1 PIX X PIX SYS SYS SYS 0-11 0
GPU2 PIX PIX X SYS SYS SYS 0-11 0
GPU3 SYS SYS SYS X PIX PIX 12-23 1
GPU4 SYS SYS SYS PIX X PIX 12-23 1
GPU5 SYS SYS SYS PIX PIX X 12-23 1
Legend:
X = Self
SYS = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
PHB = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
PXB = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
PIX = Connection traversing at most a single PCIe bridge
NV# = Connection traversing a bonded set of # NVLinks