Open
Description
What happened + What you expected to happen
It is possible to pass a custom environment by defining a class but when extra configuration parameters are passed to the custom class then it is giving an error. In the reproduction script when configuration are defined within the custom class CustomCartPole and only this class is passed to PPO it works. However when a separate configuration is defined env_config = {"env_name":'CartPole-v1'} and it is passed along with the class then it gives an error.
I saw related issue with tune that mentions passing environment configuration is an issue. Any suggestion or work around would be appreciated.
Versions / Dependencies
Gymnasium : 0.29.1
Ray: 2.9.1
Reproduction script
import gymnasium as gym
class CustomCartPole(gym.Env):
def __init__(self, env_config):
if "env_name" in env_config:
env_name = env_config["env_name"]
else:
env_name = 'CartPole-v1'
self.env = gym.make(env_name)
self.action_space = self.env.action_space
self.observation_space = self.env.observation_space
def reset(self, *, seed=None, options=None):
return self.env.reset()
def step(self, action):
obs, reward, done, truncated, info = self.env.step(action)
return obs, reward, done, truncated, info
if __name__ == "__main__":
import ray
from ray.rllib.algorithms.ppo import PPO, PPOConfig
print(ray.__version__)
print(gym.__version__)
config = PPOConfig()
config = config.training(gamma=0.9, lr=0.01, kl_coeff=0.3, train_batch_size=128)
config = config.resources(num_gpus=0)
config = config.rollouts(num_rollout_workers=1)
env_config = {"env_name":'CartPole-v1'}
# agent = PPO(config=config, env=CustomCartPole)
agent = PPO(config=config, env=CustomCartPole(env_config=env_config))
results = []
episode_data = []
episode_json = []
for n in range(5):
result = agent.train()
results.append(result)
print(results)
Error message
024-01-26 17:21:28,003 WARNING deprecation.py:50 -- DeprecationWarning: `algo = Algorithm(env='<CustomCartPole instance>', ...)` has been deprecated. Use `algo = AlgorithmConfig().environment('<CustomCartPole instance>').build()` instead. This will raise an error in the future!
Traceback (most recent call last):
File "/....../RLLIB_MultiAgent/test_customenv_argpass.py", line 35, in <module>
agent = PPO(config=config, env=CustomCartPole(env_config=env_config))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/ray/rllib/algorithms/algorithm.py", line 442, in __init__
self._env_id, self.env_creator = self._get_env_id_and_creator(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/ray/rllib/algorithms/algorithm.py", line 2548, in _get_env_id_and_creator
raise ValueError(
ValueError: <CustomCartPole instance> is an invalid env specifier. You can specify a custom env as either a class (e.g., YourEnvCls) or
5E75
a registered env id (e.g., "your_env").
Issue Severity
High: It blocks me from completing my task.