Multiple connections with shared memory using PyRserve

Question

I am using PyRserve. I am want to perform some R evaluations from multiple running python processes. All these evaluations need to use the same global variables, so it would save a lot of time if every new connection could use these variables after being defined only once. Note that redefining these with every connection is not an option because that is time consuming (for ex some variables are models loaded from files). Is there any way to have multiple PyRserve connections that share certain variables? If not what would be a good workaround here?

I could not find anything definitive in the documentation.

Answer 1

From the pyRserve documentation it would seem shared memory objects cannot* be assigned to connection namespaces:

In its current implementation pyRserve allows to set and access the following base types:

• None (NULL)
• boolean
• integers (32-bit only)
• floating point numbers (64 bit only), i.e. doubles
• complex numbers
• strings

Furthermore the following containers are supported:

• lists
• numpy arrays
• TaggedList
• AttrArray
• TaggedArray

If you are spawning a worker process for each connection, I would direct you towards the python multiprocessing library, which gives you shared memory objects of simple types, as well as higher level managed objects under the mp.Manager class.

* It might actually I haven't tested this...