The official article introducing Redis pipelining emphasis that the read() and write() system call has a huge negative impact on request processing because of context switch:
Pipelining is not just a way to reduce the latency cost associated with the round trip time, it actually greatly improves the number of operations you can perform per second in a given Redis server. This is because without using pipelining, serving each command is very cheap from the point of view of accessing the data structures and producing the reply, but it is very costly from the point of view of doing the socket I/O. This involves calling the read() and write() syscall, that means going from user land to kernel land. The context switch is a huge speed penalty.
I then made a benchmarking against a remote Redis server, using Jedis, with a connection pool of size 100 and 200 concurrent threads. The result QPS is about 400. Theoretically the number of context switch executed on Redis server per second should be about 800, if every read() and write() syscall involves a context switch, as stated in computer systems: a programmer's perspective:
A context switch can occur while the kernel is executing a system call on behalf of the user...In general, even if a system call does not block, the kernel can decide to perform a context switch rather than return control to the calling process.
However, I observed about only 280 context switch per second using the pidstat command:
11:32:07 AM UID TGID TID cswch/s nvcswch/s Command
11:32:10 AM 0 148993 - 286.00 0.00 redis-server
11:32:10 AM 0 - 148993 286.00 0.00 |__redis-server
11:32:10 AM 0 - 148994 0.00 0.00 |__bio_close_file
11:32:10 AM 0 - 148995 0.00 0.00 |__bio_aof_fsync
11:32:10 AM 0 - 148996 0.00 0.00 |__bio_lazy_free
11:32:10 AM 0 - 148997 0.33 0.00 |__jemalloc_bg_thd
I wonder if this is due to any optimizations conducted by the operating system, on the NIO mechanism used by Redis server or other factors. I would be appreciated if anyone can help!