Why Linux kernel never implemented a per data object RCU mechanism?

Question

The core RCU APIs in the Linux kernel applies to all clients in the kernel, which means any reader (even if they are accessing totally unrelated data structures) accessing rcu-backed data will be treated equally. And calls like synchronize_rcu() needs to wait for all readers, even if they are accessing entirely unrelated data structures under the hood.

Why is it that the Linux kernel never added the support for a per data object RCU? Am I missing anything here? I think the implication of the current RCU APIs is that if there are a lot of clients in the kernel, the overall performance of RCU may suffer since they share a global view.

Answer 1

I think the implication of the current RCU APIs is that if there are a lot of clients in the kernel, the overall performance of RCU may suffer since they share a global view.

No, this is wrong implication. RCU implementation in the Linux kernel is perfectly scalable for the number of "clients".

You want to "replace" the single "lock object", used for RCU, with multiple lock objects, so protection of different data could use different lock object. But the RCU implementation does NOT use any lock object at all!

Because of that, RCU implementation is quite complex and uses inner details of the Linux kernel (e.g. scheduler), but this is worth of it. E.g. rcu_read_lock and rcu_read_unlock work much faster than any sort of spin_lock, because of absence of memory contention with other cores.

---

Actually, "lock object" are used for sleepable version of RCU (sRCU). See e.g that article.

Answer 2

I've been asking myself the same question recently, and my reasoning is as follows:

• rcu_synchronize() only waits on readers that were already engaged in a RCU critical section at the time it is called, so "new" clients are not an issue
• it is not allowed to block while inside a RCU critical section, so readers will not linger for long blocking rcu_synchronize()
• there is a limited number of other "contenders": at most 1 per logical CPU

If my understanding/reasoning is correct, I suspect the overhead of possible contention would remain low enough not to justify using client-specific RCU locks. I don't know how that would scale with a large number of CPUs.

This is based on my understanding and reading of kernel code, assuming CONFIG_TREE_RCU. Things may change with other implementations (e.g. with CONFIG_TINY_RCU, synchronize_rcu() is… empty???). Also, AFAIU in some specific cases preemption may occur within the critsec (e.g. with CONFIG_PREEMPT_RCU?). I don't know whether this changes things much but don't expect it to (as probably only a higher-priority task could preempt a RCU critsec?).