Can EPOLLLET with Edge Trigger race?

Question

Suppose my process uses epoll with Edge Trigger, and the following scenario takes place:

1. call epoll_wait, succeeds with one fd ready for reading.
2. while recv() succeeds, keep reading all data
3. recv() return EWOULDBLOCK
4. More data comes in now
5. Go to step #1

Will epoll_wait() return immediately? Or wait till next data is incoming?

Answer 1

It will return immediately.

Unluckily, edge-triggered mode is not very well described (not in any way so a normal person could understand it and couldn't make wrong assumptions, anyway, in my opinion).

What it does is, it generates exactly one event when the state of the descriptor changes to "ready" (or, whatever you wait on), which means that epoll_wait() will return exactly once. However, that's not the end of the story.

It then generates the next event once the status has flipped to "not ready" and back. This is very important to note. Data coming in alone is generally not enough (although according to the docs it may be, and incoming data may generate more than one event... bleh).

For anything like a socket or pipe or such, the subtlety doesn't matter because you're going to read the data anyway. It's kinda obvious, too, and if you think about it, it makes perfect sense as well.

The emphasis on flipping once to false and back to true is important for example when your descriptor is an eventfd. There's the assumption that you can just signal the eventfd whenever you need to wake a waiter (and it will not wake a waiting thread more than once, which is nice). Sure, that's how it has to be, not anything different. Nobody cares about the value you posted either, huh... it can store only one value which is overwritten, and I couldn't care less about reading it.

Well, this is not how things work, as I found out the hard way. After waking once, you need to actually read the descriptor to flip the state to "not ready" before waiting again. Otherwise, surprise, the fact that another event came in since you last woke is just irrelevant.
Again, if you think about it, this even makes sense. It only may not be what you expect.

The way your program is written, it will work as expected. Do note, however, that given one descriptor only, it's more efficient to just block.

Answer 2

This is actually answered in the epoll(7) manual page (in the section "Level-triggered and edge-triggered").

What the manual says is that it should work fine, since the EPOLLET event is triggered by a change and that change happens in your step 4.

The manual page even says that the way to solve problems using EPOLLET is

1. with nonblocking file descriptors; and

2. by waiting for an event only after read(2) or write(2) return EAGAIN.

Which is what you already do (even though you use the equivalent EWOULDBLOCK instead of EAGAIN).

In short: When you iterate back to step 1 then epoll_wait should return immediately.