Open
Description
What happened + What you expected to happen
Cannot use gym.spaces.Sequence in observation space. We can use ray.rllib.utils.spaces.repeated.Repeated instead.
Reproduced here: https://github.com/tianhuil/ray-broken-spaces with instructions:
Here is the error we get when using gym.spaces.Sequence:
(RolloutWorker pid=78981) File "/Volumes/Workspace/ray-broken-spaces/.venv/lib/python3.11/site-packages/ray/rllib/models/preprocessors.py", line 41, in __init__ [repeated 3x across cluster]
(RolloutWorker pid=78981) self._update_policy_map(policy_dict=self.policy_dict)
(RolloutWorker pid=78981) File "/Volumes/Workspace/ray-broken-spaces/.venv/lib/python3.11/site-packages/ray/rllib/evaluation/rollout_worker.py", line 1714, in _update_policy_map
(RolloutWorker pid=78981) updated_policy_dict = self._get_complete_policy_specs_dict(policy_dict)
(RolloutWorker pid=78981) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutWorker pid=78981) File "/Volumes/Workspace/ray-broken-spaces/.venv/lib/python3.11/site-packages/ray/rllib/evaluation/rollout_worker.py", line 1792, in _get_complete_policy_specs_dict
(RolloutWorker pid=78981) preprocessor = ModelCatalog.get_preprocessor_for_space(
(RolloutWorker pid=78981) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [repeated 2x across cluster]
(RolloutWorker pid=78981) File "/Volumes/Workspace/ray-broken-spaces/.venv/lib/python3.11/site-packages/ray/rllib/models/catalog.py", line 895, in get_preprocessor_for_space
(RolloutWorker pid=78981) prep = cls(observation_space, options)
(RolloutWorker pid=78981) self.shape = self._init_shape(obs_space, self._options)
(RolloutWorker pid=78981) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutWorker pid=78981) File "/Volumes/Workspace/ray-broken-spaces/.venv/lib/python3.11/site-packages/ray/rllib/models/preprocessors.py", line 314, in _init_shape
(RolloutWorker pid=78981) preprocessor = preprocessor_class(space, self._options)
(RolloutWorker pid=78981) self._size = int(np.product(self.shape))
(RolloutWorker pid=78981) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(RolloutWorker pid=78981) TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'
I expect it to just work with Sequence -- if that's not possible, can we produce a better error in preprocessors.py than this very cryptic error listed above?
Versions / Dependencies
Reproduction script
Reproduced here: https://github.com/tianhuil/ray-broken-spaces with instructions:
Issue Severity
Medium: It is a significant difficulty but I can work around it.