Memory Architecture of HPX

Question

HPX supports Active Global Address Space.Over a long period of time, I can't able to figure out what "AGAS" really is ? By doing some research with HPX-5 supported memory models. What I can able to see is in "AGAS: memory can be moved to other localities in order to balance the system" but in "PGAS" it is not able to do so. But in hpx we still create remote objects (components) with the parameter where to create it(Global identifiers of localities). But using HPX in desktop really hides this feature and also running HPX in rostam I can't able to differentiate it with "PGAS" memory system. Could you please help me to understand this black magic feature of HPX ?

Answer 1

You can think of AGAS as being a distributed key/value 'in memory' database. When you create an object locally, you get a pointer of the standard variety that is referable using *this or this-> to get access to the internals. However, you cannot pass a this pointer from one node to another and use it arbitrarily. When you create an hpx:: component or register an object that you created with AGAS, it essentially stores the this pointer in the database and gives you an hpx::id_type as a handle (key). This id can be used in function calls on local or remote nodes as a way of referencing the object.

If you move the object from one node to another (using an agas function), then AGAS will update it's internal value to reflect the fact that the this pointer has changed value (it will internally have its destructor called locally and invoke a move of contents into a new one constructed elsewhere) and is located on another node, but the key - the id_type that you have for that object is still valid - note that this is only true if AGAS is doing the relocation - if you just create a copy elsewhere and delete a local object, it's not the same.

On a PGAS system, generally speaking, all the nodes share a block of memory with each other, and each node can 'access' memory/data/objects on the other node, by indexing into this shared memory area. So in PGAS, the addresses of items on other nodes are 'fixed' in the sense that data on node 1 is at shared_region + offset*1, data on node 2 is at + offset*2 and so on. This is a slight simplification, but you get the idea.

In HPX, objects are free to float about and you can reference them via the id_types and let AGAS handle the 'real' address lookups. That is why the 'Active' is in AGAS, as opposed to PGAS. In this way data items (components) can be relocated from one place to another, but the handles that refer to them can be immutable. In this sense the 'Address Space' part of AGAS is saying that hpx::id_type's can be thought of as addresses that span all the nodes in the job.