Why does including too big resources in my binary makes the compiler crash?

Question

My goal is to generate a *.dll that contains all the resources the program needs (around 5 Mo worth of data). I use a program of my own that converts all the files in a folder into a big .hpp such as:

#pragma once
#include <vector>
#include <map>
#include <string>

std::map<std::string, std::vector<uint8_t>> assets {
{"file_A.md", {
    0x23, 0x20, 0x2A, 0x2A, 0x43, 0x6F, 0x6E, 0x74, 0x72, 0xC3, 0xB4, 0x6C, 0x65, 0x72, 0x20, 0x6C, 
    0x27, 0x61, 0x73, 0x70, 0x65, 0x63, 0x74, 0x20, 0x67, 0xC3, 0xA9, 0x6E, 0xC3, 0xA9, 0x72, 0x61}},
{"file_B.md", {
    0x24, 0x20, 0x2A, 0x2A, 0x43, 0x6F, 0x6E, 0x74, 0x72, 0xC3, 0xB4, 0x6C, 0x65, 0x72, 0x20, 0x6C, 
    0x27, 0x61, 0x73, 0x70, 0x65, 0x63, 0x74, 0x20, 0x67, 0xC3, 0xA9, 0x6E, 0xC3, 0xA9, 0x72, 0x61}},
...
...
};

Then I include this file into another project and use the map to iterate through the files.

When the size of the binaries converted are above around 500ko, mvsc 2019 (driven by CMake) crashes with the message :

[build] C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6006: Arrêt de "CL.exe" avec le code -1073741571. [C:\GitHub\gtb-visual-control-template\build\gtb-visual-control-template.vcxproj]
[proc] The command: "C:\Program Files\CMake\bin\cmake.EXE" --build c:/GitHub/gtb-visual-control-template/build --config Debug --target ALL_BUILD -j 10 -- exited with code: 1
[build] Build finished with exit code 1

What could be the cause of the problem? Is it related to the type of structure I am using for holding the data ?

Answer 1

Whenever you use list initialization syntax (curly braces) to initialize an object that has a real constructor in it, that constructor call has to take an initializer_list. That initializer_list object points to an array of values which must exist somewhere. And that "somewhere" tends to be on a stack at some point.

So you need to make sure that the braced-init-list does not have so many values that it won't fit onto the stack.

So instead of directly initializing the map with that much data, you need to initialize a static C++ array type, and then use that to initialize the map.

Answer 2

Thanks to @Nicol Bolas, I came up to this:

#pragma once
#include <vector>
#include <map>
#include <string>

constexpr uint8_t res_0[9]{
    0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 
};
constexpr uint8_t res_1[52]{
    0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, 0x69, 0x6A, 0x6B, 0x6C, 0x6D, 0x6E, 0x6F, 0x70, 
    0x51, 0x52, 0x53, 0x54, 0x55, 0x56, 0x57, 0x4B, 0x44, 0x4E, 0x45, 0x48, 0x48, 0x45, 0x48, 0x53, 
    0x48, 0x48, 0x53, 0x48, 0x53, 0x48, 0x53, 0x53, 0x48, 0x53, 0x48, 0x53, 0x48, 0x53, 0x4F, 0x4F, 
    0x5A, 0x4F, 0x4F, 0x5A, 
};

std::map<std::string, std::vector<uint8_t>> res {
{"abc.txt", {res_0[0], 9}},
{"def.txt", {res_1[0], 52}},
};

And it works like a charm. Just tried with 20Mo without any issue! I may share the conversion software that takes a folder as input and generates this header if some people want it.