does hadoop not suffer the disk seeks as it sits on top of linux filesystem?

179 Views Asked by At

I am new to Hadoop and i know HDFS is 64 mb (min) per block and can increase depending on the system. but as hdfs is installed on top of linux filesystem which is 4kb per block, does hadoop not suffer disk seek? also does hdfs interact with linux filesystem ?

1

There are 1 best solutions below

0
On BEST ANSWER

Your thinking is correct to certain extent but look at the bigger picture. When this 64 MB is stored on the Linux file system, it is distributed across many nodes. Consequently, if you want to read 3 blocks (each 4 KB), stored on 3 different Linux file systems (machines), the seek will be for only 1 seek and not 3 seeks as reading will be in parallel.

I think this might help: How are HDFS files getting stored on underlying OS filesystem?