About the parallel execution issue in Ray

16 Views Asked by At

I tried to use ray to speed up my program. But here is some question. Here is the example code,it is just used to demonstrate the question.

with open(complete_json_data_path, "r", encoding="utf-8") as comf:
        complete_datas = json.load(comf)
@ray.remote
def test(result):
    print(id(result))
    print("start one process")
    for i in tqdm(range(100)):
        time.sleep(1)
    print("begin first")
    time.sleep(4)
    print("begin second")
    time.sleep(4)
    print("end")
    return [1],[2]
        

cpu_count = 4
ray.init(num_cpus=cpu_count)
list_of_incomplete_datas = split_list_into_n_parts(incomplete_datas, cpu_count)
result_ids = [test.remote(complete_datas) for _ in range(4)]

result = []
not_find_result = []
for result_id in result_ids:
    result_, not_find_result_ = ray.get(result_id)
    result.extend(result_)
    not_find_result.extend(not_find_result_)

In this demo,when i run it four processes will execute sequentially in series, with the same PID why?

I want to know what caused this result. If I put the code to open the file into the remote function, the program will run in parallel normally.

0

There are 0 best solutions below