Ruby C Extension: run an event loop concurrently

Question

I'm implementing a simple windowing library as a Ruby C extension. Windows have a handle_events! method that enters their native event loop.

The problem is that I want one event loop per window and the method blocks. I'd like the method to return immediately and let the loop run in a separate thread. What would be the best way to achieve this?

I tried using rb_thread_call_without_gvl to call the event loop function, and then use rb_thread_call_with_gvl in order to call the window's callbacks, which are Procs. Full source code can be found here.

It still works, but not as I intended: the method still blocks. Is this even possible with Ruby's threading model?

Answer 1

As far as I understand, using rb_thread_call_with_gvl() still needs to be done on the same thread. i.e.: it's about releasing and taking the global lock, not really about changing threads. For example, a long running gzip function can run without the lock so that other ruby threads can run in parallel.

If you want your Procs called back on another thread, shouldn't you need to create a ruby thread for those Procs? Then on that thread, call out using rb_thread_call_without_gvl() to not hold the GVL (allowing other ruby threads to run), then when you have an event on the secondary window thread, call rb_thread_call_with_gvl() to grab the lock and then you should be right to call the Proc on that same thread.

That's the way I understand it... (not having done the C extension stuff very long.)

Answer 2

I had the very same problem to solve. And as rb_thread_call_with_gvl() was marked as experimental in 1.9.2 and it was not an exported symbol, I toke a different approach:

I called the blocking handle_event! function from a separate thread. I used a second ruby thread, that blocked on a message queue. While blocking on the message queue, the gvl was released with rb_thread_blocking_region().

If now the thread calling handle_event! was unblocked due to an event, it pulled all required information for the Proc's upcall together in a queue element and pushed that element onto the queue. The ruby thread received the element, returned from rb_thread_blocking_region() and thus reacquired the gvl and call the Proc with the information from the received element.

Kind regards Torsten