Constant-time access to an arbitrary element in a list (C++)

Question

I'm currently working on the implementation of an algorithm that I would like to show that it can work in constant-time, even with a very large number of elements.

Unfortunately I need a data structure where to store the elements. When the number of elements is very high, but not unreasonable high for my algorithm, both std::vector and std::valarray do not access an arbitrary element in constant-time as you can see in this graph.

Is there a better data structure to store the values? Are there any techniques that I can implement to reach constant-time access?

Answer 1

For high values of n it is very likely that:

You are hitting a caching problem. At some point, every memory access misses the cache, causing a longer memory load.

You are hitting caching problem with memory paging. Modern computer memory is organized in a tree-like structure. Every memory access goes through that tree, making every memory access O(log n) where n is the addressable memory space. You usually don't notice it, because of high arity of that tree and good caching. However, for very high n and random memory access this may become a problem.

A friend of mine was, for example, proving that a counting sort algorithm has O(n log n) time complexity because of random memory access. Quick-sort algorithm - for comparison - has very nice, sequential access to memory, and paging overhead is much much lower.

Bottom line is, you are most likely hitting architecture/OS memory access overhead - something that you won't be able to overcome unless you use some really extreme approach (such as implementing your own OS).