STXXL multidimensional vector makes program freeze

Question

Trying to deal with huge data sets, I have been using the convenient STXXL library.Though I have run into a slight problem when trying to use/generate a multidimensional vector.

The following program freezes the system for a while and then gets killed. No output on the command line:

typedef stxxl::vector<int> vector;
typedef stxxl::vector<vector> vector_2d;
typedef stxxl::vector<vector_2d> vector_3d;    

vector_3d numbers(5);

for (auto & rc : numbers){
    rc = vector_2d(5);
    for (auto & r : rc ){
        r = vector(5);
        std::generate(r.begin(), r.end(), custom_random);
    }
}
for (auto rc : numbers){
    for(auto r : rc){
        for(auto n : r){
            std::cout << n << " ";
        }
        std::cout << std::endl;
    }
    std::cout << "-----" << std::endl;
}

Stepping through the program with a debugger, reveals that the freeze happens on the following line:

vector_3d numbers(5);

I am using version 1.3.1 of STXXL and compiling it on Linux with GCC 4.8.1. I am not sure what I am missing here. Changing the vector to the STD version, makes it work. It also works if reduced to a 2D vector.

Edit: Also tried the latest stable release of STXXL (v 1.4.0), to no avail. The same problem occurs.

Answer 1

According to the "Parameterizing STXXL Containers" section of the STXXL FAQs stxxl::vector<stxxl::vector<T> > is an invalid construct.

STXXL containers can only contain POD types.

Answer 2

I faced this problem and in short I should say you have to change your view of stxxl then you can program anything you want, in this case you could do something like:

typedef stxxl::vector<std::vector<std::vector<int>>> vector_3d;    

vector_3d numbers(5);
vector<vector<int>> a;
vector<int> b;
b.push_back(1);
b.push_back(3);
a.push_back(b);
numbers.push_back(a);

you should know the only purpose of stxxl is "very huge memory" management so you should write your code in a way that stxxl can manage when it is very very big.