C free variables used with strtok

Question

I already saw post similar to this one, but I found a little difference that drive me in the wrong way.

I have this code:

char * token_one = strtok(my_buffer, " ,.-");
char * token_two = strtok(NULL, " ,.-");

free(token_one);
free(token_two);

I saw post where people says a variable used with strtok should not be freed but why while executing this code I get this:

free(token_one) no errors

free(token_two) I get "invalid pointer"

Why don't I get error in free(token_one)? What is the correct way to handle this?

Answer 1

If you look how strtok() works, it becomes immediately clear:

1. The first call to strtok() returns the given pointer, modifying the string pointed to by it by replacing the next separation character with a NUL byte.
2. Any subsequent call, done with NULL as the first argument, operates on the saved pointer from before which points to one after the replaced character.

So the first free() call succeeds iff your my_buffer came from malloc(). The second one fails because, well, why shouldn't it? It doesn't come from malloc() et al., so calling free() on it is undefined behaviour.

Answer 2

It seems that the pointer my_buffer points to a dynamically allocated character array.

If the string stored in the array does not start from one of this characters " ,.-" then the pointer token_one returned from the function strtok will have the same value as the pointer my_buffer. So it can be used to free the allocated memory.

The pointer token_two points to the middle of the string. So its value does not point to the previously allocated memory. So the program after this call

free(token_two);

has undefined behavior.

This statement

free(token_two);

would be valid if the pointer token_two returned from the call of the function strtok was a null-pointer. But in this case no action will be performed by the call of free.

Consider two examples.

The first one is

#include <stdio.h>
#include <stdlib.h>

int main(void) 
{
    char *p1  = malloc( 10 );
    char *p2 = p1;

    free( p2 );

    return 0;
}

You can use the pointer p2 to free the memory the address of which initially was stored in the pointer p1.

On the other hand, you may not use the pointer p2 to free the allocated memory if its value points to inside the allocated memory

This program is invalid.

#include <stdio.h>
#include <stdlib.h>

int main(void) 
{
    char *p1  = malloc( 10 );
    char *p2 = p1 + 5;

    free( p2 );

    return 0;
}