Performance of Parallel computing is lower than No parallel computing in Python

Question

I just write an example for working on list and parallel on Numba as bellow by Parallel and No Parallel:

Parallel

@njit(parallel=True)
def evaluate():
  n = 1000000
  a = [0]*n
  sum = 0
  for i in prange(n):
    a[i] = i*i
  for i in prange(n):
    sum += a[i]
  return sum

No parallel

def evaluate2():
  n = 1000000
  a = [0]*n
  sum = 0
  for i in range(n):
    a[i] = i*i
  for i in range(n):
    sum += a[i]
  return sum

and compare the time of evaluation

t.tic()
print(evaluate())
t.toc()

result: 333332833333500000 Elapsed time is 0.233338 seconds.

t.tic()
print(evaluate2())
t.toc()

result: 333332833333500000 Elapsed time is 0.195136 seconds.

Full code can get from Colab

Answer 1

I haven't tried it in Numba yet. However, this exactly happens in Matlab or other programming languages when CPU is used as a non-parallel processor and GPU is used for parallel processing. When small data is processed, the CPU exceeds GPU in processing speed and parallel computing is not useful. Parallel processing is efficient only when the processing data size exceeds a certain value. There are benchmarks that show you when the parallel processing is efficient. I have read papers that they put switches in their codes for choosing between CPU and GPU during processing. Try the same code with a large array and compare the results.

Answer 2

The answer is that the number of operations is still small. When I changed n to 100,000,000, the performance change significantly.