Paralelizing healpix (Healpy) Synfast Functions

21 Views Asked by At

I use Google Colab/ Python environment and Healpy library. I need to create many map by synfast modules inside the for loop which is a heavy task and I want to paralelize.

I use the following code, it runs , but doesn't print out anything. If I add any other line inside gradually_process_iteration, it works. hp.synfast is the problem.

How can I solve this problem ? Is there any way to paralelize hp.synfast ? P.S: Cl_tot and mask are already defined.

Many thanks in advance,

The code is below:

I have tried the following and expected it to create random synfast maps 50 times in paralelized manner.

from concurrent.futures import ProcessPoolExecutor, as_completed
from tqdm.auto import tqdm
import traceback
import healpy as hp

def gradually_process_iteration(i, Cl_tot, mask):
    try:
        finmap = hp.synfast(Cl_tot, nside=512, lmax=3*nside-1, mmax=3*nside-1)
        print(f"Map generated for iteration {i}, applying mask")
        return i, "success"
    except Exception as e:
        print(f"Error in iteration {i}: {e}")
        return i, "error"


with ProcessPoolExecutor() as executor:
    # Assuming Cl_tot and mask are defined and ready to be used
    futures = [executor.submit(gradually_process_iteration, i, Cl_tot, mask) for i in range(50)]
    for future in tqdm(as_completed(futures), total=len(futures)):
        i, result = future.result()
        print(f"Result from iteration {i}: {result}")
0

There are 0 best solutions below