Hi,

I've been running the MAgent Tiger-Deer environment with 2 different algorithms: a RandomLearner and rllib's PPO. I'm also currently using rllib's PettingZooEnv. It seems both of the algorithms work for some number of iterations, but then error-out in this line https://github.com/ray-project/ray/blob/master/rllib/env/pettingzoo_env.py#L161.

The issue is that the agent being selected, deer_92, is not in the action_dict. I checked the self.aec_env.dones dict, however, and the agent is there. I added a snippet of the output, below. I printed out relevant info (shown after each == start step ==) when entering the step() function. Furthermore, it also appears all steps prior to this error only select deer_0 as the agent. I've re-ran the experiment several times and it always has the same result (e.g., deer_0 is always chosen and then it errors-out once any other agent is chosen).

I'm not sure if this is an issue with rllib, the Tiger-Deer env, or my own config.

(pid=34152) =============== start step =====================
(pid=34152) self.aec_env.agent_selection --> deer_0
(pid=34152) stepped_agents --> set()
(pid=34152) list(action_dict) --> ['deer_0', 'deer_1', 'deer_2', 'deer_3', 'deer_4', 'deer_5', 'deer_6', 'deer_7', 'deer_8', 'deer_9', 'deer_10', 'deer_11', 'deer_12', 'deer_13', 'deer_14', 'deer_15', 'deer_16', 'deer_17', 'deer_18', 'deer_19', 'deer_20', 'deer_21', 'deer_22', 'deer_23', 'deer_24', 'deer_25', 'deer_26', 'deer_27', 'deer_28', 'deer_29', 'deer_30', 'deer_31', 'deer_32', 'deer_33', 'deer_34', 'deer_35', 'deer_36', 'deer_37', 'deer_38', 'deer_39', 'deer_40', 'deer_41', 'deer_42', 'deer_43', 'deer_44', 'deer_45', 'deer_46', 'deer_47', 'deer_48', 'deer_49', 'deer_50', 'deer_51', 'deer_52', 'deer_53', 'deer_54', 'deer_55', 'deer_56', 'deer_57', 'deer_58', 'deer_59', 'deer_60', 'deer_61', 'deer_62', 'deer_63', 'deer_64', 'deer_65', 'deer_66', 'deer_67', 'deer_68', 'deer_69', 'deer_70', 'deer_71', 'deer_72', 'deer_73', 'deer_74', 'deer_75', 'deer_76', 'deer_77', 'deer_78', 'deer_79', 'deer_80', 'deer_81', 'deer_82', 'deer_83', 'deer_84', 'deer_85', 'deer_86', 'deer_87', 'deer_88', 'deer_89', 'deer_90', 'deer_91', 'deer_92', 'deer_93', 'deer_94', 'deer_95', 'deer_96', 'deer_97', 'deer_98', 'deer_99', 'deer_100', 'tiger_0', 'tiger_1', 'tiger_2', 'tiger_3', 'tiger_4', 'tiger_5', 'tiger_6', 'tiger_7', 'tiger_8', 'tiger_9', 'tiger_10', 'tiger_11', 'tiger_12', 'tiger_13', 'tiger_14', 'tiger_15', 'tiger_16', 'tiger_17', 'tiger_18', 'tiger_19']
(pid=34152) agent in action_dict -->  True
(pid=34152) agent in self.aec_env.dones --> False
(pid=34152) =============== start step =====================
(pid=34152) self.aec_env.agent_selection --> deer_0
(pid=34152) stepped_agents --> set()
(pid=34152) list(action_dict) --> ['deer_0', 'deer_1', 'deer_2', 'deer_3', 'deer_4', 'deer_5', 'deer_6', 'deer_7', 'deer_8', 'deer_9', 'deer_10', 'deer_11', 'deer_12', 'deer_13', 'deer_14', 'deer_15', 'deer_16', 'deer_17', 'deer_18', 'deer_19', 'deer_20', 'deer_21', 'deer_22', 'deer_23', 'deer_24', 'deer_25', 'deer_26', 'deer_27', 'deer_28', 'deer_29', 'deer_30', 'deer_31', 'deer_32', 'deer_33', 'deer_34', 'deer_35', 'deer_36', 'deer_37', 'deer_38', 'deer_39', 'deer_40', 'deer_41', 'deer_42', 'deer_43', 'deer_44', 'deer_45', 'deer_46', 'deer_47', 'deer_48', 'deer_49', 'deer_50', 'deer_51', 'deer_52', 'deer_53', 'deer_54', 'deer_55', 'deer_56', 'deer_57', 'deer_58', 'deer_59', 'deer_60', 'deer_61', 'deer_62', 'deer_63', 'deer_64', 'deer_65', 'deer_66', 'deer_67', 'deer_68', 'deer_69', 'deer_70', 'deer_71', 'deer_72', 'deer_73', 'deer_74', 'deer_75', 'deer_76', 'deer_77', 'deer_78', 'deer_79', 'deer_80', 'deer_81', 'deer_82', 'deer_83', 'deer_84', 'deer_85', 'deer_86', 'deer_87', 'deer_88', 'deer_89', 'deer_90', 'deer_91', 'deer_92', 'deer_93', 'deer_94', 'deer_95', 'deer_96', 'deer_97', 'deer_98', 'deer_99', 'deer_100', 'tiger_0', 'tiger_1', 'tiger_2', 'tiger_3', 'tiger_4', 'tiger_5', 'tiger_6', 'tiger_7', 'tiger_8', 'tiger_9', 'tiger_10', 'tiger_11', 'tiger_12', 'tiger_13', 'tiger_14', 'tiger_15', 'tiger_16', 'tiger_17', 'tiger_18', 'tiger_19']
(pid=34152) agent in action_dict -->  True
(pid=34152) agent in self.aec_env.dones -->  False
(pid=34152) =============== start step =====================
(pid=34152) self.aec_env.agent_selection --> deer_92
(pid=34152) stepped_agents --> set()
(pid=34152) list(action_dict) --> ['deer_0', 'deer_1', 'deer_2', 'deer_3', 'deer_4', 'deer_5', 'deer_6', 'deer_7', 'deer_8', 'deer_9', 'deer_10', 'deer_11', 'deer_12', 'deer_13', 'deer_14', 'deer_15', 'deer_16', 'deer_17', 'deer_18', 'deer_19', 'deer_20', 'deer_21', 'deer_22', 'deer_23', 'deer_24', 'deer_25', 'deer_26', 'deer_27', 'deer_28', 'deer_29', 'deer_30', 'deer_31', 'deer_32', 'deer_33', 'deer_34', 'deer_35', 'deer_36', 'deer_37', 'deer_38', 'deer_39', 'deer_40', 'deer_41', 'deer_42', 'deer_43', 'deer_44', 'deer_45', 'deer_46', 'deer_47', 'deer_48', 'deer_49', 'deer_50', 'deer_51', 'deer_52', 'deer_53', 'deer_54', 'deer_55', 'deer_56', 'deer_57', 'deer_58', 'deer_59', 'deer_60', 'deer_61', 'deer_62', 'deer_63', 'deer_64', 'deer_65', 'deer_66', 'deer_67', 'deer_68', 'deer_69', 'deer_70', 'deer_71', 'deer_72', 'deer_73', 'deer_74', 'deer_75', 'deer_76', 'deer_77', 'deer_78', 'deer_79', 'deer_80', 'deer_81', 'deer_82', 'deer_83', 'deer_84', 'deer_85', 'deer_86', 'deer_87', 'deer_88', 'deer_89', 'deer_90', 'deer_91', 'deer_93', 'deer_94', 'deer_95', 'deer_96', 'deer_97', 'deer_98', 'deer_99', 'deer_100', 'tiger_0', 'tiger_1', 'tiger_2', 'tiger_3', 'tiger_4', 'tiger_5', 'tiger_6', 'tiger_7', 'tiger_8', 'tiger_9', 'tiger_10', 'tiger_11', 'tiger_12', 'tiger_13', 'tiger_14', 'tiger_15', 'tiger_16', 'tiger_17', 'tiger_18', 'tiger_19']
(pid=34152) agent in action_dict -->  False
(pid=34152) agent in self.aec_env.dones -->  True
== Status ==
Memory usage on this node: 24.8/377.6 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/80 CPUs, 0/2 GPUs, 0.0/252.88 GiB heap, 0.0/77.54 GiB objects (0/1.0 GPUType:V100)
Result logdir: /home/ray_results/Campaign_Tiger-Deer-v1
Number of trials: 1 (1 ERROR)
+------------------------------------------+----------+-------+--------+------------------+------+----------+
| Trial name                               | status   | loc   |   iter |   total time (s) |   ts |   reward |
|------------------------------------------+----------+-------+--------+------------------+------+----------|
| PS_PPO_Trainer_Tiger-Deer-v1_41b65_00000 | ERROR    |       |      3 |          3576.35 | 3672 | 0.166667 |
+------------------------------------------+----------+-------+--------+------------------+------+----------+
Number of errored trials: 1

If I use the PettingZooEnv in version in ray==0.87, the error is https://github.com/ray-project/ray/blob/releases/0.8.7/rllib/env/pettingzoo_env.py#L165.

Lastly, I also applied the following SuperSuit wrappers: pad_observations_v0, pad_action_space_v0, agent_indicator_v0, and flatten_v0, and I'm running PettingZoo==1.3.3 and SuperSuit==2.1.0.

Thanks.

I've added support for continuous actions in MPE as discussed in #249 through the continuous_actions=False argument in the environment config.

I've tested my changes on all environments using RLLib MADDPG, and run ./release_test.sh. The latter needed some updates as well.

All environments are working except simple_world_comm_v2, which seems to have a supersuit-related bug.

• Made rps sequential with argument max_cycles.
• Merged rpsls in rps. Choose between both env with argument lizard_spock

I am not sure if the rps version needs to be bumped or not. Also, should the name of the env still be called rps despite having merged rpsls in the same env?

Description

This PR moves the Pyright checks from CI only to pre-commit (both local + CI), fixes some typing issues, and add also pydocstyle to pre-commit.

Type of change

• Refactoring/maintenance

Checklist:

• [x] I have run the pre-commit checks with pre-commit run --all-files (see CONTRIBUTING.md instructions to set it up)
• [x] I have run pytest -v and no errors are present.
• [x] I have commented my code, particularly in hard-to-understand areas
• [x] I have made corresponding changes to the documentation
• [x] I solved any possible warnings that pytest -v has generated that are related to my code to the best of my knowledge.
• [x] New and existing unit tests pass locally with my changes

I'm running into an error with this long stack trace when I try to run the 13 line tutorial:

/Users/erick/.local/share/virtualenvs/rl-0i49mzF7/lib/python3.9/site-packages/torch/utils/tensorboard/__init__.py:4: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if not hasattr(tensorboard, '__version__') or LooseVersion(tensorboard.__version__) < LooseVersion('1.15'):
/Users/erick/.local/share/virtualenvs/rl-0i49mzF7/lib/python3.9/site-packages/torch/utils/tensorboard/__init__.py:4: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if not hasattr(tensorboard, '__version__') or LooseVersion(tensorboard.__version__) < LooseVersion('1.15'):
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/usr/local/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 125, in _main
    prepare(preparation_data)
  File "/usr/local/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/usr/local/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "/usr/local/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 268, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/usr/local/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/local/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/erick/dev/rl/main_pettingzoo.py", line 22, in <module>
    env = ss.concat_vec_envs_v1(env, 8, num_cpus=4, base_class="stable_baselines3")
  File "/Users/erick/.local/share/virtualenvs/rl-0i49mzF7/lib/python3.9/site-packages/supersuit/vector/vector_constructors.py", line 60, in concat_vec_envs_v1
    vec_env = MakeCPUAsyncConstructor(num_cpus)(*vec_env_args(vec_env, num_vec_envs))
  File "/Users/erick/.local/share/virtualenvs/rl-0i49mzF7/lib/python3.9/site-packages/supersuit/vector/constructors.py", line 38, in constructor
    return ProcConcatVec(
  File "/Users/erick/.local/share/virtualenvs/rl-0i49mzF7/lib/python3.9/site-packages/supersuit/vector/multiproc_vec.py", line 144, in __init__
    proc.start()
  File "/usr/local/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/usr/local/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/usr/local/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/usr/local/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/usr/local/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/local/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 42, in _launch
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "/usr/local/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "/usr/local/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.
Exception ignored in: <function ProcConcatVec.__del__ at 0x112cf1310>
Traceback (most recent call last):
  File "/Users/erick/.local/share/virtualenvs/rl-0i49mzF7/lib/python3.9/site-packages/supersuit/vector/multiproc_vec.py", line 210, in __del__
    for pipe in self.pipes:
AttributeError: 'ProcConcatVec' object has no attribute 'pipes'

Maybe the issue here is some mismatch in library versioning, but I found no reference to which supersuit version is supposed to run with the tutorial (or with the rest of the code).

I am running python 3.9 with supersuit 3.4 and pettingzoo 1.18.1

Hello,

I am new to MARL and want to create custom parallel env in PettingZoo and wanted to play with it. I am getting the following import error when attempting to get the environments.

from pettingzoo.butterfly import pistonball_v4 env = pistonball_v4.env()

` ImportError: cannot import name 'pistonball_v4' from 'pettingzoo.butterfly' PS C:\Users\mvadr\Desktop\gym>

`

• Hanabi, integrated from official repo as git submodule.

• See hanabi/README.md for how to pull and setup git submodule.

• See documentation within class for full information.

• Debugged and E2E tested with unittest, including api test as part of the unit tests.

• CHANGE REQUEST: Add typing annotation in utils/env.py, see last commit.

If you are submitting a bug report, please fill in the following details and use the tag [bug].

Describe the bug I try to change the environment based on this script. The origin script used waterworld which has bug so I change it to magent.battle. The output show below:

Traceback (most recent call last):
  File "rlpe.py", line 4, in <module>
    from pettingzoo.magent import battle_v4
  File "/anaconda3/envs/PettingZoo/lib/python3.7/site-packages/pettingzoo/magent/__init__.py", line 5, in __getattr__
    return deprecated_handler(env_name, __path__, __name__)
  File "/anaconda3/envs/PettingZoo/lib/python3.7/site-packages/pettingzoo/utils/deprecated_module.py", line 52, in deprecated_handler
    spec.loader.exec_module(module)
  File "/anaconda3/envs/PettingZoo/lib/python3.7/site-packages/pettingzoo/magent/battle_v4.py", line 1, in <module>
    from .battle.battle import env, parallel_env, raw_env  # noqa: F401
  File "/anaconda3/envs/PettingZoo/lib/python3.7/site-packages/pettingzoo/magent/battle/battle.py", line 7, in <module>
    from pettingzoo.magent.magent_env import magent_parallel_env, make_env
  File "/anaconda3/envs/PettingZoo/lib/python3.7/site-packages/pettingzoo/magent/magent_env.py", line 4, in <module>
    from magent import Renderer
ImportError: cannot import name 'Renderer' from 'magent' (/anaconda3/envs/PettingZoo/lib/python3.7/site-packages/magent/__init__.py)

Code example

from ray import tune
from ray.tune.registry import register_env
from ray.rllib.env.wrappers.pettingzoo_env import PettingZooEnv
from pettingzoo.magent import battle_v4

# Based on code from github.com/parametersharingmadrl/parametersharingmadrl

if __name__ == "__main__":
    # RDQN - Rainbow DQN
    # ADQN - Apex DQN
    def env_creator(args):
        return PettingZooEnv(battle_v4.env(map_size=45, minimap_mode=False, step_reward=-0.005,
dead_penalty=-0.1, attack_penalty=-0.1, attack_opponent_reward=0.2,
max_cycles=1000, extra_features=False))

    env = env_creator({})
    register_env("battle", env_creator)

    tune.run(
        "APEX_DDPG",
        stop={"episodes_total": 60000},
        checkpoint_freq=10,
        config={
            # Enviroment specific
            "env": "battle",
            # General
            "num_gpus": 1,
            "num_workers": 2,
            # Method specific
            "multiagent": {
                "policies": set(env.agents),
                "policy_mapping_fn": (lambda agent_id, episode, **kwargs: agent_id),
            },
        },
    )

System Info Describe the characteristic of your environment: Linux with conda and pip Python 3.7.9 PettingZoo 1.20.1 supersuit 3.5.0 gym 0.22.0 Additional context Add any other context about the problem here.

Checklist

• [x] I have checked that there is no similar issue in the repo (required)

Description

This PR fixes issue https://github.com/Farama-Foundation/PettingZoo/issues/719. The main changes are the following:

• Removed release_test.sh
• Made according changes to the docs
• Bumped GH actions versions
• Updated the tests section in setup.py

Type of change

• Refactoring
• This change requires a documentation update

Checklist:

• [x] I have run the pre-commit checks with pre-commit run --all-files (see CONTRIBUTING.md instructions to set it up)
• [x] I have run ./release_test.sh and no errors are present.
• [x] I have commented my code, particularly in hard-to-understand areas
• [x] I have made corresponding changes to the documentation
• [x] I solved any possible warnings that ./release_test.sh has generated that are related to my code to the best of my knowledge.

Hi, I have a custom pettingzoo Parallel environment and want to train a shared policy on multiple agents. I want to be able to have a few of my agents inactive during a training-episode under certain conditions. I also want to bring back agents during an episode. I call the petting_zoo_env_to_vec_env_v1 and concat_vec_envs_v1 on my environment to allow for shared policy and training on copies of the environment.

I also use the wrappers black_death_v3, pad_observations_v0 and pad_action_space_v0 to allow for agent death but I still get an error message "environment has agent death. Not allowed for pettingzoo_env_to_vec_env_v1 unless black_death is true". How do I set this? So in principle is it possible to have self.possible_agents as a sort of agent pool to take agents from and then have self.agents as the active agents during an episode?

best regards

Axel

I was trying to read through and understand the chess environment before attempting to create my own environment for MARL problem. I was reading through the step function and think that the environment is archiving a trailing history.

 def step(self, action):
        if self.dones[self.agent_selection]:
            return self._was_done_step(action)
        current_agent = self.agent_selection
        current_index = self.agents.index(current_agent)
        next_board = chess_utils.get_observation(self.board, current_agent)
        self.board_history = np.dstack((next_board[:, :, 7:], self.board_history[:, :, :-13]))
        ....

When the step function is first called by the white pieces, the starting board is added to the board_history as it is run prior to any move being made. When the observe function is then run by the black pieces, the observation would fail to include the first move made by white pieces. Is there anything I am missing?

Describe the bug

When creating a wrapper around an environment and overriding only the seed method, when I try to call the .last() method of the wrapped environment I get: AttributeError: accessing private attribute '_cumulative_rewards' is prohibited. It seems the error only happens if the environment has an observation space with spaces.Sequence in it, as this does not happen on other pettinzoo environments and spaces that I tested.

I managed to re-create the issue on this colab notebook, I have also attached the same code below.

Code example

from typing import Optional, Any, Dict, List

from gymnasium import Space, spaces
from pettingzoo import AECEnv
from pettingzoo.utils import BaseWrapper


class TestEnv(AECEnv):
    metadata = {'render.modes': ['ansi'], 'name': 'test_env'}

    def __init__(self, seed: Optional[int] = None):
        super().__init__()
        self.n_cards = 52
        self.cards = []

        self.possible_agents: List[str] = ["player_0"]
        self.agents = self.possible_agents.copy()

        self.action_spaces = {agent: spaces.Discrete(self.n_cards) for agent in self.agents}
        # The bug seems to be when using spaces.Sequence
        self.observation_spaces = {agent: spaces.Sequence(spaces.Discrete(self.n_cards)) for agent in self.agents}
        self.infos = {i: {} for i in self.agents}

        self.reset(seed)

    def reset(self, seed: Optional[int] = None, return_info: bool = False, options: Optional[Dict] = None) -> None:
        self.cards = []

        self.agents = self.possible_agents.copy()
        self.rewards = {agent: 0 for agent in self.agents}
        self._cumulative_rewards = {agent: 0 for agent in self.agents}
        self.truncations = {i: False for i in self.agents}

    def seed(self, seed: Optional[int] = None) -> None:
        pass

    def observation_space(self, agent: str) -> Space:
        return self.observation_spaces[agent]

    def action_space(self, agent: str) -> Space:
        return self.action_spaces[agent]

    @property
    def agent_selection(self) -> str:
        return self.agents[0]

    @property
    def terminations(self) -> Dict[str, bool]:
        return dict([(agent, False) for agent in self.agents])

    def observe(self, agent: str) -> List[Any]:
        return self.cards

    def step(self, action: int) -> None:
        assert action in self.action_spaces[self.agent_selection]
        self.cards.append(action)

    def render(self) -> str:
        return self.cards.__repr__()

    def state(self):
        pass

    def close(self):
        super().close()


class TestEnvWrapper(BaseWrapper):
    def seed(self, seed: Optional[int] = None) -> None:
        pass



# The env works
print(TestEnv().last())

# The wrapper has an error
TestEnvWrapper(TestEnv()).last()

System info

Pettingzoo version: 1.22.3

Additional context

No response

Checklist

• [X] I have checked that there is no similar issue in the repo

Description

As title, fix typo of BaseParallelWrapper (previously BaseParallelWraper). Linked to its SuperSuit counterpart https://github.com/Farama-Foundation/SuperSuit/pull/204.

Type of change

• Bug fix (non-breaking change which fixes an issue)

Checklist:

• [x] I have run the pre-commit checks with pre-commit run --all-files (see CONTRIBUTING.md instructions to set it up)
• [x] I have run pytest -v and no errors are present.
• [x] I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I solved any possible warnings that pytest -v has generated that are related to my code to the best of my knowledge.
• [ ] I have added tests that prove my fix is effective or that my feature works
• [ ] New and existing unit tests pass locally with my changes

Description

Fixes #864 by drawing on pygame screen before extracting the image for rgb_array mode. Only calls pygame.surfarray.pixels3d when needed.

Not sure if a new unit test is needed, can add if necessary !

Type of change

• Bug fix (non-breaking change which fixes an issue)

Screenshots

Please attach before and after screenshots of the change if applicable.

Before:

After:

Checklist:

• [x] I have run the pre-commit checks with pre-commit run --all-files (see CONTRIBUTING.md instructions to set it up)
• [x] I have run pytest -v and no errors are present.
• [ ] I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I solved any possible warnings that pytest -v has generated that are related to my code to the best of my knowledge.
• [ ] I have added tests that prove my fix is effective or that my feature works
• [ ] New and existing unit tests pass locally with my changes

Proposal

support python3.11

Motivation

3.11 is the latest stable python release,

Gymnasium already supports it

we need it for Gymnasium-Robotics/MaMuJoCo to support 3.11

Pitch

No response

Alternatives

No response

Additional context

No response

Checklist

• [X] I have checked that there is no similar issue in the repo

Proposal

When using this wrapper in a piece of code, I noticed that it only works for Discrete spaces from gym. However, extending it to MultiDiscrete spaces too is extremely straightforward: everything needed is to add a case to the assert check on line 11 in assert_out_of_bounds.py:

`

    assert all(

        (isinstance(self.action_space(agent), Discrete) or isinstance(self.action_space(agent), MultiDiscrete))

        for agent in getattr(self, "possible_agents", [])

    ), "should only use AssertOutOfBoundsWrapper for Discrete spaces"

`

MultiDiscrete spaces has the same contains() method of Discrete ones, used in the step() method, so no further adjustment would be needed.

Motivation

MultiDiscrete spaces are supported by many stable-baselines algorithms, and are useful in many situations.

Pitch

The wrapper is useful in many situation, and having it to work with this kind of space as well would be really good.

Alternatives

No response

Additional context

No response

Checklist

• [X] I have checked that there is no similar issue in the repo