Efficient circular buffer with constant-time access

Question

In a machine learning project written in python, I need an efficient circular buffer like collections.deque but with constant-time access to any element like numpy.array. The problem is that deque is apparently a linked list. Is there something efficient readily implemented in a python library that I am not aware of for this use-case please?

I could simply have a modified fixed-size numpy.array with a moving 0 index in my use-case, I guess, but that's for my python culture as it is not the first time I need something like that.

Answer 1

The numpy_ringbuffer module at https://pypi.org/project/numpy-ringbuffer/ uses buffers backed by a NumPy array. It should satisfy your efficiency requirements.

Answer 2

collections.deque is not exactly a linked-list. It's a doubly-linked list of arrays of size 64. I'd say it's a pretty decent choice when you want both the random-access and appending on both ends without constant reallocation.

However, if you've done proper performance profiling and that circular buffer is really your bottle-neck then you can implement the buffer in plain C for performance and add bindings to python.