Open
Description
What is the problem?
WARNING import_thread.py:132 -- The remote function 'filaA.parafor' has been exported 100 times. It's possible that this warning is accidental, but this may indicate that the same remote function is being defined repeatedly from within many tasks and exported to all of the workers. This can be a performance issue and can be resolved by defining the remote function on the driver instead. See https://github.com/ray-project/ray/issues/6240 for more discussion.I read the issues#6240. As I am a new user of Ray, I still don't have any idea to avoid the current warning.
Ray version and other system information (Python version, TensorFlow version, OS):
Ray version:1.2.0
python version: 3.7.10
OS: Ubuntu 20
Reproduction (REQUIRED)
In fileA.py
class A:
def __init__(self):
pass
def dataReader(self):
sub_vector_id=[parafor.remote(self) for sub_vector in np.array_split(raw_vector, num_cpus)]
@ray.remote
def parafor(obj):
passIn fileB.py
from fileA import A
import ray
for _ in range(1000):
ray.init(address='auto', log_to_driver=False)
c=A()
c.dataReader()
ray.shutdown()
...
optimizeFunction(c)
...Then I start to run fileB.py, after many iterations, I got the above-mentioned error. I would appreciate if you have a solution. Thank you in advance!