Is small size file read/write slower when the file gets bigger?

Question

I am on Ubuntu 12.04 using ext4. I wrote a python program that does small size (mostly 512 byte) read and write with somewhat random access pattern. I found that as the file gets larger and larger. It takes more and more time to do the same number of I/Os. The relationship is linear. In other words, I get O(n²) where n is the cumulative number of I/Os.

I wonder if there is an inherent reason why small I/O being slower as file size increases.

One more observation: When I mounted a ramdisk and did my File I/O to the ramdisk I do NOT observe this performance degradation.

Answer 1

depending on how you're doing the IO it might be that you're trying to call too much into memory before saving it off.

Answer 2

When you read 512 bytes out of 1024 bytes large file, the whole file is in cache. As the file size grows, smaller portion of file is in the cache, and data is read from the disk more and more frequently. I.e. you get more cache misses as the file grows. Maybe this is what you are experiencing.