boost fast_pool_allocator sometimes requests a huge allocation

Question

I have a highly threaded application that uses boost's fast_pool_allocator (version 1.55) underneath quickfix (1.13.3). The application allocates a large number of objects over the course of the day, growing more-or-less linearly until we shut it down, using on the order of 32G of virtual memory by the end of the day. Most allocations are on the order of 200MB as we watch the virtual memory footprint of the application grow. But then, at some point usually later in the day, boost decides to do a 6GB allocation, even though the flow of transactions through the application doesn't change materially.

Looking at the allocator code, the first thing boost does after the allocation is set up the new block into chunks. The function is simple_segregated_storage<SizeType>::segregate at simple_segregated_storage.hpp:280. We've attached a debugger to the process, and observed that when the giant allocation happens, that function (unsurprisingly) takes a long time to execute, specifically the for loop at line 302. It takes as long as 20-30 seconds, and that code is mutex protected so every other thread trying to do anything in the allocator blocks. This angers our customers.

Questions:

1. Why would it suddenly alloc 6GB when it had been constantly asking for ~200MB blocks all day prior to that?
2. Can the allocation be capped somehow? I'd rather have it ask for smaller pieces more often.
3. Is this the wrong allocator? I suppose that's a question for the quickfix developers, but this appears to be their preferred way to go. The objects that use the allocator are mostly std::map and std::multimap.

Answer 1

As Yakk suggested above, it turns out that the boost pool does allow you to specify a MaxSize in the template. It's a little strange in that it operates in units of "chunks", a concept internal to the pool's implementation. IMHO bytes would have been more natural, but so be it.

fast_pool_allocator is defined with default values for all the template arguments (except the first). I copied those defaults and changed the last one. For this application, the chunk size is 88 bytes, but I assume that's dependent on the class in the map.

typedef std::map <int, std::vector <FieldMap*>, std::less<int>, 
                  boost::fast_pool_allocator<std::pair<const int, std::vector<FieldMap*>>,
                                             boost::default_user_allocator_new_delete,
                                             boost::details::pool::default_mutex,
                                             32,       // NextSize (default from boost)
                                             1500000   // MaxSize, in 88 byte chunks.
                                            >> Groups;

In this case, 1500000 * 88 = 132M, which is about as big as I want the allocations to be for now.