Maps of maps allocated in shared memory

Question

Inside a boost::interprocess::managed_shared_memory, I am trying to create boost::unordered_map inside another boost::unordered_map as value, having key as std::string for both maps. This Map in Map inside a shared memory segment gets accessed by two different processes fetch values from both outer & inner maps.

Below is my implementation & want to know if this is possible/right way or any other better way possible?

boost::interprocess::managed_shared_memory segment(boost::interprocess::open_or_create, "BOOST_SHM", 65536);

    typedef std::string   KeyType;
    typedef std::string   ValueType;
    typedef std::pair<const KeyType, ValueType> MapType;
    typedef boost::interprocess::allocator<MapType, boost::interprocess::managed_shared_memory::segment_manager> ShmemAllocator;
    typedef boost::unordered_map<KeyType, ValueType, boost::hash<KeyType>, std::equal_to<KeyType>, ShmemAllocator> InMap;
    ShmemAllocator alloc_inst(segment.get_segment_manager());
    InMap *inside_map = segment.construct<InMap>("SHM_IN_MAP")(3, boost::hash<KeyType>(), std::equal_to<KeyType>(), alloc_inst);


    typedef std::pair<const KeyType, MapType> MIMType;
    typedef boost::interprocess::allocator<MIMType, boost::interprocess::managed_shared_memory::segment_manager> MIMShmemAllocator;
    typedef boost::unordered_map<KeyType, MapType, boost::hash<KeyType>, std::equal_to<KeyType>, MIMShmemAllocator> OutMap;
    //MIMShmemAllocator alloc_inst(segment.get_segment_manager());   /*Commented due to Error*/
    OutMap *outside_map = segment.construct<OutMap>("SHM_OUT_MAP")(3, boost::hash<KeyType>(), std::equal_to<KeyType>(), alloc_inst);

Other details:

gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) on CentOS 7, BOOST_LIB_VERSION "1_58"

Answer 1

Ok.

So there were a few basic errors, and possibly some confusion.

Next, there are some power tricks that make using nested containers with custom (stateful) allocators much more convenient.

Here's the roll-up of all three hints in a working sample that hopefully helps!

---

1. Your strings must use shared memory allocators too

   Otherwise the data would be illegal to use in another process. Using the strings would result in Undefined Behaviour.

   At the very least, make your strings use the shared memory allocator:

   namespace Shared {
       using Segment   = bip::managed_shared_memory;
   
       template <typename T>
       using Alloc     = bip::allocator<T, Segment::segment_manager>;
   
       using String    = boost::container::basic_string<char, std::char_traits<char>, Alloc<char> >;
       using KeyType   = String;
       using ValueType = String;
   }
   

2. The map allocators were overspecified. The actual node-types wrapping the pair<K const, v> elements in a map are implementation defined anyways. So how do maps know how to allocate these nodes?

   They rebind allocators: see rebind in the docs here

   So, you can just pass Alloc<void>. Or the same allocator as for the Shared::String. The map will figure it out:

   typedef boost::unordered_map<KeyType, ValueType, boost::hash<KeyType>, std::equal_to<KeyType>, Alloc<void> > InMap;
   typedef boost::unordered_map<KeyType, InMap,     boost::hash<KeyType>, std::equal_to<KeyType>, Alloc<void> > OutMap;
   

3. Now for the power tips.

   Passing stateful allocators all the freaking time is annoying. It makes code a mess. Luckily, c++11 (and Boost Containers for c++03) has you covered:

   • scoped_allocator_adaptor<T...>
   • allocator_type
   • uses_allocator<T> trait

   These helpers can make your life a lot easier. They do this by passing the allocator down to element type constructors when applicable. Automatically. Again, implicit conversions from rebound allocator types make things work.

   So, you can actually just construct one outer map with the correct allocator (make it Scoped) and one key, and from there you don't even have to keep specifying allocators.

Here's a full demo:

Live On Coliru

#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/containers/string.hpp>
#include <boost/container/scoped_allocator.hpp>
#include <boost/unordered_map.hpp>
#include <iostream>

namespace bip = boost::interprocess;

namespace Shared {
    using Segment = bip::managed_shared_memory;

    template <typename T>
    using Alloc   = bip::allocator<T, Segment::segment_manager>;
    using Scoped  = boost::container::scoped_allocator_adaptor<Alloc<char> >;

    using String  = boost::container::basic_string<char, std::char_traits<char>, Scoped>;
    using KeyType = String;

    typedef boost::unordered_map<KeyType, String, boost::hash<KeyType>, std::equal_to<KeyType>, Scoped> InMap;
    typedef boost::unordered_map<KeyType, InMap,  boost::hash<KeyType>, std::equal_to<KeyType>, Scoped> OutMap;
}

int main() {
    srand(time(NULL));

    Shared::Segment segment(bip::open_or_create, "BOOST_SHM", 65536);
    auto* mgr = segment.get_segment_manager();

    Shared::OutMap *p_outside_map = segment.find_or_construct<Shared::OutMap> ("SHM_OUT_MAP") (mgr);
    auto& outside_map = *p_outside_map;

    Shared::String sskey(mgr); // reduce shared allocations as they are costly (in terms of fragmentation/overhead)

    char outer_keys[3], inner_keys[3];
    std::generate_n(outer_keys, 3, [] { return rand()%26+'a'; });
    std::generate_n(inner_keys, 3, [] { return rand()%26+'a'; });

    for (auto key : outer_keys) {
        sskey = key;
        auto& inner = outside_map[sskey];

        for (auto more : inner_keys) {
            inner[sskey + "_" + more] += "value";
        }
    }

    for (auto const& oe : outside_map) {
        for (auto const& ie : oe.second) {
            std::cout << "outside_map[" << oe.first << "][" << ie.first << "] == " << ie.second << "\n";
        }
    }
}

Actually, to make it work on Coliru, we need to use a mapped file instead:

Live On Coliru

Run it a few times:

outside_map[s][s_t] == value
outside_map[s][s_r] == value
outside_map[s][s_c] == value
outside_map[f][f_t] == value
outside_map[f][f_r] == value
outside_map[f][f_c] == value
outside_map[o][o_t] == value
outside_map[o][o_r] == value
outside_map[o][o_c] == value

Second run:

outside_map[a][a_d] == value
outside_map[a][a_c] == value
outside_map[a][a_g] == value
outside_map[r][r_d] == value
outside_map[r][r_c] == value
outside_map[r][r_g] == value
outside_map[g][g_d] == value
outside_map[g][g_c] == value
outside_map[g][g_g] == value
outside_map[s][s_t] == value
outside_map[s][s_r] == value
outside_map[s][s_c] == value
outside_map[f][f_t] == value
outside_map[f][f_r] == value
outside_map[f][f_c] == value
outside_map[o][o_t] == value
outside_map[o][o_r] == value
outside_map[o][o_c] == value

Note how each run successfully appends value to 9 keys in 3 inner maps.