Nested dir name in python multiprocess shared_memory - Invalid argument

Question

It seems SharedMemory only supports the flat name, does anyone know why or any ref? I have searched on the internet for quite a long time, but didn't find any relative..

from multiprocessing import shared_memory
shm = shared_memory.SharedMemory(name='test_dir/test_name', size=16, create=True)

Got

Invalid argument: '/test_dir/test_name'

Answer 1

This is not a problem on macOS and probably not an issue on any Unix type platform. Better to not use a name at all. You can acquire the generated name after instantiation of the segment.

Are you using Windows? As a guess, Windows creates a file based on the name and if that's the case you can imagine why a directory separator might cause a problem.

Better (in my opinion) to allow the shared memory segment reference (its name) be generated for you. You can then acquire the generated name and pass that to other parts of your code (or, indeed, other processes) that want to attach to the segment you've created.

Here's a rather convoluted "Hello world" program to demonstrate this strategy:

from multiprocessing.shared_memory import SharedMemory

def write_to_shared_memory(name: str, data: bytes) -> None:
    try:
        shm = SharedMemory(name) # attach to existing shared memory segment
        size = len(data)
        shm.buf[:size] = data
    finally:
        shm.close()

if __name__ == "__main__":
    try:
        data = b"Hello world!"
        shm = SharedMemory(size=len(data), create=True) # create the shared memory segment
        # emulate some other process that attaches to the shared memory segment by name
        write_to_shared_memory(shm.name, data)
        print(bytes(shm.buf).decode())
    finally:
        shm.close()
        shm.unlink()