Why is parfor slow despite slicing?

Question

I have a simple parfor loop given below.

% fileAddr is a cell array of (size N) of file-addresses
sIdx = nan(N,1);
eIdx = nan(N,1);
errMsg = cell(N,1);
parfor i=1:N
    [sIdx(i),eIdx(i),errMsg{i}] = myFunk(fileAddr{i});
end

The function file myFun() loads a file given by fileAddr{i}, makes some calculations and returns results. The file loading part is the most time consuming. My machine has 4 physical cores. I tried parfor() with a pool o of 1,2,3 and 4 workers. Every time, the time consumption is in similar ballpark. My understanding was that if more than one worker is load()ing the files in parallel, the program would run faster but the profiler results show otherwise.

Can anyone please explain where am I making a mistake?

Answer 1

You only have 1 hard drive. Only 1 worker can read from it at a time (its a speeding disk with a magnetic head!). Its slower because the workers are waiting for their turn for the HDD so you win no time. Add to that all the overheard of the data sending and sharing and you make it slower.

Have you tried spmd? But I suspect it will end up in the same result you have with parfor.