How can I synchronize two Deep Reinforcement Learning agents?

Question

I am doing a project in which I simulate a computer network. Each node of the network is a Deep Reinforcement Learning agent and its states will depend on a global matrix from which they have to take data and then modify data. And that I would like to know when would be the most appropriate time to update the state of these agents and what would be the most correct option.

The state has one row more than the matrix containing the MLU of the links. This row will store the packet to be worked on.

#Creamos una matriz que almacenara la mlu en cada momento, inicializada a 0 en los nodos #conectados
matrizMLU = np.full((nodos_red, nodos_red), -1, int)
for i in range(nodos_red):
    for j in range(i+1, nodos_red):
        if j in puertos[i]:
            matrizMLU[i][j] = 0
            matrizMLU[j][i] = 0

class nodoEnv(Env):
    def __init__(self, idNodo):                               #Inicializacion del entorno
        self.id = idNodo
        self.action_space = Discrete(5)               #Acciones
        self.observation_space = Box(low=0, high=100, shape = (len(matrizMLU)+1, len(matrizMLU)))
        self.estado = np.array(np.zeros((len(matrizMLU)+1, len(matrizMLU)), dtype = int))  
        self.camino = calcularCaminos(idNodo)