Futex and pthreads issue

Question

I'm testing futexes with pthreads. I've written following program:

#include <stdio.h>
#include <pthread.h>
#include <stdint.h>
#include <stdatomic.h>
#include <sys/syscall.h>
#include <linux/futex.h>
#include <unistd.h>
#include <errno.h>
#include <stdlib.h>

int64_t sum;
pthread_mutex_t mtx;

uint32_t ftx;

int futex(uint32_t *uaddr, int futex_op, uint32_t val, const struct timespec *timeout, uint32_t *uaddr2, uint32_t val3)
{
    return syscall(SYS_futex, uaddr, futex_op, val, timeout, uaddr2, val3);
}

void fwait(uint32_t *futx)
{
    long st;
    uint32_t one = 1;
    
    do {
        if (atomic_compare_exchange_strong(futx, &one, 0))
            break;
        
        st = futex(futx, FUTEX_WAIT_PRIVATE, 0, NULL, NULL, 0);
        
        if ((st == -1) && (errno != EAGAIN)) {
            printf("error-wait-futex\n");
            exit(1);
        }
    } while (1);
    
}

void fwake(uint32_t *futx)
{
    long st;
    uint32_t zero = 0;
    
    if (atomic_compare_exchange_strong(futx, &zero, 1)) {
        st = futex(futx, FUTEX_WAKE_PRIVATE, 1, NULL, NULL, 0);
        if (st == -1) {
            printf("error-wake-futex\n");
            exit(1);
        }
    }
}

void lock(void)
{
    fwait(&ftx);
}

void unlock(void)
{
    fwake(&ftx);
}

void * t1_handler(void *arg)
{
    int mod;
    uint64_t x;
    
    mod = *(int *)arg;
    
    x = 0;
    while (x < 1000000) {
        lock();
        sum += mod;
        x++;
        unlock();
    }
}

void proc(void)
{
    pthread_t t1, t2;
    int a1, a2;
    
    sum = 0;
    ftx = 1;
    
    a1 = 1;
    a2 = -1;
    
    pthread_mutex_init(&mtx, NULL);
    pthread_create(&t1, NULL, t1_handler, &a1);
    pthread_create(&t2, NULL, t1_handler, &a2);
    
    pthread_join(t1, NULL);
    pthread_join(t2, NULL);

    printf("sum: %lld\n", sum);
}

int main(void)
{
    proc();
    return 0;
}

And sometimes it returns 0 as a sum which is proper value but sometimes the returned value of sum is different than 0.

My question is why the returned values of "sum" are different than 0? I suspect that there is something wrong with locking but for the moment I cannot figure out what.

Answer 1

The problem with your implementation is the way you use the atomic_compare_exchange_strong(). Note that, from C reference that:

[If compare fails], loads the actual contents of memory pointed to by obj into *expected (performs load operation).

In your code, that means that when this atomic operation fails (i.e. futx is not 1) the value of futx will be loaded at one, meaning that now one value is now 0.

From now, the lock function is broken giving that the atomic operation will succeed (i.e. futx has the same value as one, that is 0) even when the lock is taken, meaning that we can have more than one thread holding it at the same time. To fix this you need to either resets one every loop:

do {
    uint32_t one = 1;
    if (atomic_compare_exchange_strong(futx, &one, 0))
        break;
    
    ...
} while (1);

Or even better, always set the lock as taken and check if the previous value was the value for free lock:

if (atomic_exchange(futx, 0) == 1)  

---

If you use defines for your futexes values it makes the understanding easier. In your case:

#define LOCK_FREE 1
#define LOCK_TAKEN 0