How come Visual Studio does not optimize structs for best memory usage?

Question

My question is why doesnt the Visual Studio 2012 compiler automatically reorder struct members for best memory utilization? The compiler seems to store the members in exactly the order they are declared in the struct definition, with some empty padding as required for member alignment. It seems like reordering would be a more desirable way to align the members than padding, whenever possible. Is there a reason it must be stored in memory in declaration order?

Pertinent details follow;

I have a struct which represents a single element in what will be a large array. The element has a number of members, some 32 bit, some 64 bit. I have default struct member alignment tuned on for best performance.

I was exploring memory in debug mode and found that a signficant percentage of the memory was being wasted. I tracked the problem to how the stuct members were aligned in memory. I know that 32 bit members must be aligned on DWORD boundaries for best performance and it would appear that evidently 64 bit members must be aligned on QWORD boundaries (I would have thought DWORD boundaries would have been adequate)

I was able to fix the problem by changing the order in which I listed the members in the struct definition. I made sure to put 2 32 bit members consequtively whenever possible so that no padding is required to start the next 64 bit member on a QWORD boundary.

Answer 1

Data that is in a standard layout structure or class must make certain layout guarantees. Among other things if there is another standard layout structure or class that is the prefix of the first, you must be able to reinterpret one structure as the other, and the common prefix has to agree.

This basically forces the memory order of standard layout structures to be in the order you declare them.

This is similar to what C requires in terms of structure layout, as described here.

Now, in C++, there is some freedom is given for non-standard layout structures.

[expr.rel]/3 subpoint 3:

If two pointers point to different non-static data members of the same object, or to subobjects of such members, recursively, the pointer to the later declared member compares greater provided the two members have the same access control (Clause 11) and provided their class is not a union.

The order of elements must be maintained within public/private/protected access control domains. Space between elements can be added in nearly arbitrary ways.

This means you can know that &this->x is greater than or less than &this->y, which some programmers may use.

Under the as-if rule, if nobody ever takes an address of such data, the compiler could reorder them. This is hard to prove in the usual compilation models.

The spacing between elements in MSVC matches the spacing in plain old data structures, barring inheritance with virtual playing the game, in my experience. Layout compatibility (beyond the standard) is important to a stable ABI, and code compiled with one version of the compiler is preferred to work in another. Breaking that has a cost.

C++ programmers can reorder data structures as they need, and visual studio provides #pragmas to change the structure packing rules, so if you really need that last bit of performance you can get it.

You can even write a tuple-like data structure that guarantees optimal packing if you need it. (I wouldn't rely on std::tuple, as it has no packing guarantees)

Answer 2

It's the C++ standard, compilers can't modify the order of the fields, probably because programmers may want to access fields via pointer to first field. Take a look at this article if you need to do the reordering by yourself

Section 9.2.13 :

Nonstatic data members of a (non-union) class with the same access control (Clause 11) are allocated so that later members have higher addresses within a class object. The order of allocation of non-static data members with different access control is unspecified (Clause 11). Implementation alignment requirements might cause two adjacent members not to be allocated immediately after each other; so might requirements for space for managing virtual functions (10.3) and virtual base classes (10.1).

Answer 3

Without #pragmas, memory is not packed by C++ and are not reordered because the language guarantees layout consistent with the code. Imagine the havoc that would be caused -- mapping structs to files (memory mapped files) or hardware would never work.

To get a feel for the layout of a class or struct, Visual C++ provide an undocumented command line parameter /d1reportSingleClassLayout which will draw you an ASCII-art diagram of the memory layout of your class/struct, including all members, base members and vtable. If you have a class called foo, for example, add /d1reportSingleClassLayoutfoo to your compiler command line.

Answer 4

I suspect this is at the intersection of overlapping requirements.

1. The layout for the same POD struct in C and C++ should be binary compatible. (Whether this is required by the standard, I don't know, but most compiler vendors probably prioritize it because lots of existing code depends on it.)
2. Structs and classes in C++ are effectively the same thing except for the default visibility.
3. Data members of a class are constructed in the order they are declared, and destroyed in the opposite order.

If the compiler were to reorder the data members for better alignment and/or tighter packing, should that change the order that the construction/destruction order? Nope, that would break lots of code that relies on RAII. But now the memory accesses during construction are less in-order, which might actually be a pessimization, depending on cache behavior, the size of the structure, and the frequency with which those structs are constructed.

You could argue that those concerns don't apply to POD structs, but requirements 1 and 2 say that a C++ compiler must lay out POD structs the same way as classes (and vice versa).