Please see example image below.
On Dask distributed dashboard under "workers" tab, each worker has 23 "num_fds". What does "num_fds" mean? Does this mean # of file descriptors opened up by each worker? If so, what should the file descriptor limit be on the machine hosting the scheduler process?
My machine, hosting the scheduler process, has limit of 65K FD. But it was able to create all the 3000 workers whose sum num_fds was 69000.
In general, I am trying to understand the relationship between "num_fds" of each worker and what that means for the cluster.
Thank you!
