Is gcc's __attribute__((packed)) / #pragma pack unsafe?

Question

In C, the compiler will lay out members of a struct in the order in which they're declared, with possible padding bytes inserted between members, or after the last member, to ensure that each member is aligned properly.

gcc provides a language extension, __attribute__((packed)), which tells the compiler not to insert padding, allowing struct members to be misaligned. For example, if the system normally requires all int objects to have 4-byte alignment, __attribute__((packed)) can cause int struct members to be allocated at odd offsets.

Quoting the gcc documentation:

The `packed' attribute specifies that a variable or structure field should have the smallest possible alignment--one byte for a variable, and one bit for a field, unless you specify a larger value with the `aligned' attribute.

Obviously the use of this extension can result in smaller data requirements but slower code, as the compiler must (on some platforms) generate code to access a misaligned member a byte at a time.

But are there any cases where this is unsafe? Does the compiler always generate correct (though slower) code to access misaligned members of packed structs? Is it even possible for it to do so in all cases?

Answer 1

Yes, __attribute__((packed)) is potentially unsafe on some systems. The symptom probably won't show up on an x86, which just makes the problem more insidious; testing on x86 systems won't reveal the problem. (On the x86, misaligned accesses are handled in hardware; if you dereference an int* pointer that points to an odd address, it will be a little slower than if it were properly aligned, but you'll get the correct result.)

On some other systems, such as SPARC, attempting to access a misaligned int object causes a bus error, crashing the program.

There have also been systems where a misaligned access quietly ignores the low-order bits of the address, causing it to access the wrong chunk of memory.

Consider the following program:

#include <stdio.h>
#include <stddef.h>
int main(void)
{
    struct foo {
        char c;
        int x;
    } __attribute__((packed));
    struct foo arr[2] = { { 'a', 10 }, {'b', 20 } };
    int *p0 = &arr[0].x;
    int *p1 = &arr[1].x;
    printf("sizeof(struct foo)      = %d\n", (int)sizeof(struct foo));
    printf("offsetof(struct foo, c) = %d\n", (int)offsetof(struct foo, c));
    printf("offsetof(struct foo, x) = %d\n", (int)offsetof(struct foo, x));
    printf("arr[0].x = %d\n", arr[0].x);
    printf("arr[1].x = %d\n", arr[1].x);
    printf("p0 = %p\n", (void*)p0);
    printf("p1 = %p\n", (void*)p1);
    printf("*p0 = %d\n", *p0);
    printf("*p1 = %d\n", *p1);
    return 0;
}

On x86 Ubuntu with gcc 4.5.2, it produces the following output:

sizeof(struct foo)      = 5
offsetof(struct foo, c) = 0
offsetof(struct foo, x) = 1
arr[0].x = 10
arr[1].x = 20
p0 = 0xbffc104f
p1 = 0xbffc1054
*p0 = 10
*p1 = 20

On SPARC Solaris 9 with gcc 4.5.1, it produces the following:

sizeof(struct foo)      = 5
offsetof(struct foo, c) = 0
offsetof(struct foo, x) = 1
arr[0].x = 10
arr[1].x = 20
p0 = ffbff317
p1 = ffbff31c
Bus error

In both cases, the program is compiled with no extra options, just gcc packed.c -o packed.

(A program that uses a single struct rather than array doesn't reliably exhibit the problem, since the compiler can allocate the struct on an odd address so the x member is properly aligned. With an array of two struct foo objects, at least one or the other will have a misaligned x member.)

(In this case, p0 points to a misaligned address, because it points to a packed int member following a char member. p1 happens to be correctly aligned, since it points to the same member in the second element of the array, so there are two char objects preceding it -- and on SPARC Solaris the array arr appears to be allocated at an address that is even, but not a multiple of 4.)

When referring to the member x of a struct foo by name, the compiler knows that x is potentially misaligned, and will generate additional code to access it correctly.

Once the address of arr[0].x or arr[1].x has been stored in a pointer object, neither the compiler nor the running program knows that it points to a misaligned int object. It just assumes that it's properly aligned, resulting (on some systems) in a bus error or similar other failure.

Fixing this in gcc would, I believe, be impractical. A general solution would require, for each attempt to dereference a pointer to any type with non-trivial alignment requirements either (a) proving at compile time that the pointer doesn't point to a misaligned member of a packed struct, or (b) generating bulkier and slower code that can handle either aligned or misaligned objects.

I've submitted a gcc bug report. As I said, I don't believe it's practical to fix it, but the documentation should mention it (it currently doesn't).

UPDATE: As of 2018-12-20, this bug is marked as FIXED. The patch will appear in gcc 9 with the addition of a new -Waddress-of-packed-member option, enabled by default.

When address of packed member of struct or union is taken, it may result in an unaligned pointer value. This patch adds -Waddress-of-packed-member to check alignment at pointer assignment and warn unaligned address as well as unaligned pointer

I've just built that version of gcc from source. For the above program, it produces these diagnostics:

c.c: In function ‘main’:
c.c:10:15: warning: taking address of packed member of ‘struct foo’ may result in an unaligned pointer value [-Waddress-of-packed-member]
   10 |     int *p0 = &arr[0].x;
      |               ^~~~~~~~~
c.c:11:15: warning: taking address of packed member of ‘struct foo’ may result in an unaligned pointer value [-Waddress-of-packed-member]
   11 |     int *p1 = &arr[1].x;
      |               ^~~~~~~~~

Answer 2

(The following is a very artificial example cooked up to illustrate.) One major use of packed structs is where you have a stream of data (say 256 bytes) to which you wish to supply meaning. If I take a smaller example, suppose I have a program running on my Arduino which sends via serial a packet of 16 bytes which have the following meaning:

0: message type (1 byte)
1: target address, MSB
2: target address, LSB
3: data (chars)
...
F: checksum (1 byte)

Then I can declare something like

typedef struct {
  uint8_t msgType;
  uint16_t targetAddr; // may have to bswap
  uint8_t data[12];
  uint8_t checksum;
} __attribute__((packed)) myStruct;

and then I can refer to the targetAddr bytes via aStruct.targetAddr rather than fiddling with pointer arithmetic.

Now with alignment stuff happening, taking a void* pointer in memory to the received data and casting it to a myStruct* will not work unless the compiler treats the struct as packed (that is, it stores data in the order specified and uses exactly 16 bytes for this example). There are performance penalties for unaligned reads, so using packed structs for data your program is actively working with is not necessarily a good idea. But when your program is supplied with a list of bytes, packed structs make it easier to write programs which access the contents.

Otherwise you end up using C++ and writing a class with accessor methods and stuff that does pointer arithmetic behind the scenes. In short, packed structs are for dealing efficiently with packed data, and packed data may be what your program is given to work with. For the most part, you code should read values out of the structure, work with them, and write them back when done. All else should be done outside the packed structure. Part of the problem is the low level stuff that C tries to hide from the programmer, and the hoop jumping that is needed if such things really do matter to the programmer. (You almost need a different 'data layout' construct in the language so that you can say 'this thing is 48 bytes long, foo refers to the data 13 bytes in, and should be interpreted thus'; and a separate structured data construct, where you say 'I want a structure containing two ints, called alice and bob, and a float called carol, and I don't care how you implement it' -- in C both these use cases are shoehorned into the struct construct.)

Answer 3

Using this attribute is definitely unsafe.

One particular thing it breaks is the ability of a union which contains two or more structs to write one member and read another if the structs have a common initial sequence of members. Section 6.5.2.3 of the C11 standard states:

6 One special guarantee is made in order to simplify the use of unions: if a union contains several structures that share a common initial sequence (see below), and if the union object currently contains one of these structures, it is permitted to inspect the common initial part of any of them anywhere that a declaration of the completed type of the union is visible. Tw o structures share a common initial sequence if corresponding members have compatible types (and, for bit-fields, the same widths) for a sequence of one or more initial members.

...

9 EXAMPLE 3 The following is a valid fragment:

union {
    struct {
        int    alltypes;
    }n;
    struct {
        int    type;
        int    intnode;
    } ni;
    struct {
        int    type;
        double doublenode;
    } nf;
}u;
u.nf.type = 1;
u.nf.doublenode = 3.14;
/*
...
*/
if (u.n.alltypes == 1)
if (sin(u.nf.doublenode) == 0.0)
/*
...
*/

When __attribute__((packed)) is introduced it breaks this. The following example was run on Ubuntu 16.04 x64 using gcc 5.4.0 with optimizations disabled:

#include <stdio.h>
#include <stdlib.h>

struct s1
{
    short a;
    int b;
} __attribute__((packed));

struct s2
{
    short a;
    int b;
};

union su {
    struct s1 x;
    struct s2 y;
};

int main()
{
    union su s;
    s.x.a = 0x1234;
    s.x.b = 0x56789abc;

    printf("sizeof s1 = %zu, sizeof s2 = %zu\n", sizeof(struct s1), sizeof(struct s2));
    printf("s.y.a=%hx, s.y.b=%x\n", s.y.a, s.y.b);
    return 0;
}

Output:

sizeof s1 = 6, sizeof s2 = 8
s.y.a=1234, s.y.b=5678

Even though struct s1 and struct s2 have a "common initial sequence", the packing applied to the former means that the corresponding members don't live at the same byte offset. The result is the value written to member x.b is not the same as the value read from member y.b, even though the standard says they should be the same.

Answer 4

It's perfectly safe as long as you always access the values through the struct via the . (dot) or -> notation.

What's not safe is taking the pointer of unaligned data and then accessing it without taking that into account.

Also, even though each item in the struct is known to be unaligned, it's known to be unaligned in a particular way, so the struct as a whole must be aligned as the compiler expects or there'll be trouble (on some platforms, or in future if a new way is invented to optimise unaligned accesses).

Answer 5

As ams said above, don't take a pointer to a member of a struct that's packed. This is simply playing with fire. When you say __attribute__((__packed__)) or #pragma pack(1), what you're really saying is "Hey gcc, I really know what I'm doing." When it turns out that you do not, you can't rightly blame the compiler.

Perhaps we can blame the compiler for it's complacency though. While gcc does have a -Wcast-align option, it isn't enabled by default nor with -Wall or -Wextra. This is apparently due to gcc developers considering this type of code to be a brain-dead "abomination" unworthy of addressing -- understandable disdain, but it doesn't help when an inexperienced programmer bumbles into it.

Consider the following:

struct  __attribute__((__packed__)) my_struct {
    char c;
    int i;
};

struct my_struct a = {'a', 123};
struct my_struct *b = &a;
int c = a.i;
int d = b->i;
int *e __attribute__((aligned(1))) = &a.i;
int *f = &a.i;

Here, the type of a is a packed struct (as defined above). Similarly, b is a pointer to a packed struct. The type of of the expression a.i is (basically) an int l-value with 1 byte alignment. c and d are both normal ints. When reading a.i, the compiler generates code for unaligned access. When you read b->i, b's type still knows it's packed, so no problem their either. e is a pointer to a one-byte-aligned int, so the compiler knows how to dereference that correctly as well. But when you make the assignment f = &a.i, you are storing the value of an unaligned int pointer in an aligned int pointer variable -- that's where you went wrong. And I agree, gcc should have this warning enabled by default (not even in -Wall or -Wextra).