Reallocating a previously allocated pointer to SIZE_MAX doesn't set ENOMEM, but reallocating NULL works?

Question

The Problem:

I'm working on re-writing the some malloc functions (malloc, calloc, realloc, and free), and I decided to implement some unit tests to hopefully make things a little easier for myself down the line (plus it seemed like good practice).

As I'm setting things up, I'm using the standard 'alloc's to make sure the tests are running correctly. Unfortunately, I'm running into issues with the following test:

#include "unity.h"

#include <stdlib.h>
#include <errno.h>

#include "nmalloc.h"

void realloc_overflow(void)
{
    // Reset ERRNO
    errno = 0;

    // Verify allocating SIZE_MAX sets ENOMEM when reallocating NULL
    TEST_ASSERT_EQUAL_PTR(NULL, my_realloc(NULL, SIZE_MAX)); // <-- Sets ENOMEM
    TEST_ASSERT_ERRNO(ENOMEM);

    // Allocate 112 bytes
    void* ptr = my_malloc(ALLOC_LEN_112U);
    TEST_ASSERT_TRUE(ptr != NULL);

    // Reset ERRNO
    errno = 0;

    // Verify allocating SIZE_MAX sets ENOMEM when reallocating a valid pointer
    // ! This doesn't set errno to ENOMEM on my system? However (SIZE_MAX + 1) does?
    TEST_ASSERT_EQUAL_PTR(NULL, my_realloc(ptr, SIZE_MAX));

    TEST_ASSERT_ERRNO(ENOMEM);

    my_free(ptr);
}

Which gives an unexpected output:

test/Test-nmalloc.c:326:realloc_overflow:FAIL: Expected 12 Was 0

Weirdly though, using realloc(valid_ptr, SIZE_MAX + 1) shows the expected behavior of setting errno = ENOMEM?

Note: If it becomes important, I'm testing all of this on an M1 Mac

Attempts at a Solution:

I made a minimum workable example:

#include <stdlib.h>
#include <stdio.h>
#include <errno.h>

#include "nmalloc.h"

#define ALLOC_LEN_112U 112U

int main(void)
{
    errno = 0;

    // TEST_ASSERT_EQUAL_PTR(NULL, my_realloc(NULL, SIZE_MAX));
    if (NULL == realloc(NULL, SIZE_MAX))
        printf("PASS: null_ptr is NULL.\n");
    else
    {
        printf("ERR: null_ptr is not NULL!\n");
        return 1;
    }

    // TEST_ASSERT_ERRNO(ENOMEM);
    if (errno == ENOMEM)
        printf("PASS: errno is ENOMEM.\n");
    else
    {
        printf("ERR: errno is not NULL!\n");
        return 1;
    }

    void *valid_ptr = malloc(ALLOC_LEN_112U);

    // TEST_ASSERT_TRUE(ptr != NULL);
    if (valid_ptr == NULL)
    {
        printf("ERR: valid_ptr is NULL!\n");
        return 1;
    }
    else
        printf("PASS: valid_ptr is not NULL.\n");

    errno = 0;

    // TEST_ASSERT_EQUAL_PTR(NULL, realloc(ptr, SIZE_MAX));
    if (NULL == realloc(valid_ptr, SIZE_MAX))
        printf("PASS: invalid_ptr is NULL.\n");
    else
    {
        printf("ERR: invalid_ptr is not NULL!\n");
        return 1;
    }

    // TEST_ASSERT_ERRNO(ENOMEM);
    if (errno == ENOMEM)
        printf("PASS: errno is ENOMEM.\n");
    else
    {
        printf("ERR: errno is not NULL!\n");
        return 1;
    }

    free(valid_ptr);

    printf("Success!");
}

Which gives the correct expected output???

[user@machine ~/test] % clang alloc-test.c nmalloc.o -I./lib -o alloc-test.o && ./alloc-test.o
PASS: null_ptr is NULL.
PASS: errno is ENOMEM.
PASS: valid_ptr is not NULL.
PASS: invalid_ptr is NULL.
PASS: errno is ENOMEM.
Success!
[user@machine ~/test] % 

Even when including the jemalloc library (i.e. Adding an #include <...>, not just adding it to clang), everything seems to act normal. So at this point, I'm not entirely sure what's going on here, so some insight would be MUCH appreciated. Maybe I'm just missing something subtle (or obvious)?

Thanks! :)

Answer 1

1. SIZE_MAX is not the maximum size of memory which can be allocated only the maximum value which can be held by the size_t. (SIZE_MAX + 1) == 0 and the behaviour of the realloc is implementation defined

2. SIZE_MAX is far more than 128TB of the virtual memory in Linux 64. So malloc family function will fail.

Weirdly though, using realloc(valid_ptr, SIZE_MAX + 1) shows the expected behavior of setting errno = ENOMEM?

(MAX_SIZE + 1) is zero and the behaviour in this case is not specified by the standard.