close
The Wayback Machine - https://web.archive.org/web/20201027032114/https://github.com/ray-project/ray/issues/11451
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ray.util.multiprocessing.Pool.starmap does not work with zip #11451

Open
HerculesJack opened this issue Oct 18, 2020 · 1 comment
Open

ray.util.multiprocessing.Pool.starmap does not work with zip #11451

HerculesJack opened this issue Oct 18, 2020 · 1 comment

Comments

@HerculesJack
Copy link

@HerculesJack HerculesJack commented Oct 18, 2020

What is the problem?

It seems that ray.util.multiprocessing.Pool.starmap does not work with iterable created from zip, unless it's explicitly converted to a list etc. For example, this one works:

from ray.util.multiprocessing import Pool as RayPool

with RayPool(4) as pool:
    print(pool.starmap(lambda x, y: x + y, list(zip(range(0, 7, 2), range(1, 8, 2)))))

But this one gives me an error, saying that TypeError: <lambda>() takes 2 positional arguments but 4 were given.

from ray.util.multiprocessing import Pool as RayPool

with RayPool(4) as pool:
    print(pool.starmap(lambda x, y: x + y, zip(range(0, 7, 2), range(1, 8, 2))))

With python's own multiprocessing.Pool I don't need to explicitly convert it to a list, so I think it'd be better if the behavior of ray.util.multiprocessing.Pool can be the same here? I guess this issue would be easy to fix.

Ray version and other system information (Python version, TensorFlow version, OS):

Ray 1.0.0 installed via pip, Python 3.6.11, SUSE 15.

Reproduction (REQUIRED)

Please provide a script that can be run to reproduce the issue. The script should have no external library dependencies (i.e., use fake or mock data / environments):

See above.

  • I have verified my script runs in a clean environment and reproduces the issue.
  • I have verified the issue also occurs with the latest wheels.
@rkooo567
Copy link
Contributor

@rkooo567 rkooo567 commented Oct 19, 2020

cc @edoakes

Thanks for this report :). Btw, feel free to contribute to the fix!! That'll be really helpful

@rkooo567 rkooo567 added good first issue P2 and removed triage labels Oct 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.