ray.exceptions.RayActorError: The actor died unexpectedly before finishing this task

62 Views Asked by At

i was running a project from github which use ray. Then i met this error and have no idea how to fix it
output:
(pid=3451) 2024-03-18 04:05:30.102745: W tensorflow/core/framework/allocator.cc:124] Allocation of 5600000000 exceeds 10% of system memory. 2024-03-18 04:05:40,734 ERROR worker.py:1752 -- A worker died or was killed while executing task 00000000e8e30890e0a1c55cabe41e1c5567a2b4. 2024-03-18 04:05:40,786 ERROR trial_runner.py:426 -- Error processing event. Traceback (most recent call last): File "/usr/local/envs/mopac/lib/python3.6/site-packages/ray/tune/trial_runner.py", line 389, in _process_events result = self.trial_executor.fetch_result(trial) File "/usr/local/envs/mopac/lib/python3.6/site-packages/ray/tune/ray_trial_executor.py", line 252, in fetch_result result = ray.get(trial_future[0]) File "/usr/local/envs/mopac/lib/python3.6/site-packages/ray/worker.py", line 2288, in get raise value ray.exceptions.RayActorError: The actor died unexpectedly before finishing this task.

mycode

trial_runner

ray_trial_executor

worker

i have tried to find some solution in internet but it seem not similar to my error or i just can't clearly understand how to fix it. i think there is no problem about memory because it was 9.4/12.7gb ram when this error occured

0

There are 0 best solutions below