How should I register an update in a callback from another thread without using locks?

Question

This is one of those questions that seems to fall into the "naively obvious but probably wrong" category. Certainly, I'm struggling to find a solution that works in all corner cases. It seems like this must be a problem that is encountered all the time.

I have a "hardware" thread and a "processing" thread.

The hardware thread is run in a closed binary library that I don't have access to.

The processing thread registers a callback with the hardware thread to be notified of certain (rare) events related to state changes in the hardware.

I wish to be able to notify the event loop in the processing thread that the state has changed. I'd like to do this without requiring a dependance on an external library or in a non portable way, and without using locks (as I have no idea when the hardware thread may want to notify the processing thread again).

So, my thinking at present as to how to solve this is something like as follows:

#include <signal.h>

// sig_atomic_t is used so update is always in a sane state
static volatile sig_atomic_t update = 0;

// called from hardware thread
int callback_function() {
    update += 1;
}

// called regularly from processing thread
int processing_function() {
    static sig_atomic_t local_update = 0; // The same type as update

    if (local_update != update){
        update_internal_hardware_state(); // We necessarily call once per callback
        local_update += 1;
    }
}

Clearly this is going to break if ever update wraps around and just happens to reach the value of local_update before processing_function is next called (though almost certainly I can assume this will never happen).

Have I missed something subtle (or not so subtle) here?

Is there a better way to solve this whole problem?

The callback can be assumed to be only ever called from one thread (the hardware thread).

Answer 1

One options is to use the standard unix self-pipe trick. The benefit is that the read end of the pipe can be used with select() or epoll() thus integrating nicely with event loops without having to periodically poll the value of an atomic variable.

#include <unistd.h>
#include <errno.h>

int self_pipe[2]; // the read end is non-blocking

// called from hardware thread
int callback_function() {
    write(self_pipe[1], "", 1); // notify by writing a byte
    return 0;
}

// called only when self_pipe[0] is ready for read
int processing_function() {
    for(;;) {
        char buf;
        ssize_t n = read(self_pipe[0], &buf, 1);
        if(n > 0) {
            // callback_function() has been called
        }
        else if(n < 0) {
            if(EAGAIN == errno)
                break;
            // handle an (unlikely) error
        }
        else {
            // handle pipe EOF if necessary
        }
    }
    return 0;
}

If you are on Linux >= 2.6.22 instead of pipe you can use an eventfd, which is basically a semaphore maintained in the kernel and operated with read() and write() syscalls. Again, the benefit of it is that it can be used with event loops.

Answer 2

It seems you have implemented a semaphore by hand.

Instead, why not use a proper semaphore for your platform? Posix Semaphores exist on the ARM platform. I'm not sure why you don't want to use external libraries, if you insist in doing it yourself then this may help.

Answer 3

Do you have access to the compare-and-swap operation? Then you can do something like this in your processing thread:

int ticks_since_last_check = 0, ticks_ret = 0;
do {
    ticks_since_last_check = ticks_ret;   
    ticks_ret = compare_and_swap(&update, ticks_since_last_check, 0));
} while (ticks_since_last_check != ticks_ret)

This will reset the tick counter to 0 every time the processing thread checks its value, so the counter will not overflow. After that you will just need to run the update_internal_hardware_state function ticks_since_last_check times.

Alternatively, you can use atomic decrement:

int ticks_since_last_check = 0;
while (atomic_read(&update) != 0) {
    atomic_decrement(&update);
    ++ticks_since_last_check;
}