See Issue #1663

This is a bit of a ride. The base change is getting rid of the custom seeding utils of gym, and instead using np.random.default_rng() as is recommended with modern versions of NumPy. I kept the gym.utils.seeding.np_random interface and changed it to basically being a synonym for default_rng (with some API difference, consistent with the old np_random)

Because the API (then RandomState, now Generator) changed a bit, np_random.randint calls were replaced with np_random.integers, np_random.rand -> np_random.random, np_random.randn -> np_random.standard_normal. This is all in accordance with the recommended NumPy conversion.

My doubts in order of (subjective) importance

Doubt number 1:

In gym/utils/seeding.py#L18 I'm accessing a "protected" variable seed_seq. This serves accessing the random seed that was automatically generated under the hood when the passed seed is None. (it also gives the correct value if the passed seed is an integer) An alternative solution would be restoring the whole create_seed machinery which generates a random initial seed from os.urandom. I was unable to find another way to get the initial seed of a default Generator (default_rng(None)) instance.

Doubt number 2:

In gym/spaces/multi_discrete.py#L64. Turns out that a Generator doesn't directly support get_state and set_state. The same functionality seems to be achievable by accesing the bit generator and modifying its state directly (without getters/setters).

Doubt number 3:

As mentioned earlier, this version maintains the gym.utils.seeding file with just a single function. Functionally, I don't think that's a problem, but might feel a bit redundant from a stylistic point of view. This could be replaced by changing something like 17 calls of this function that occur in the codebase, but at the moment I think it'd be a bad move. The reason is that the function passes through the seed that's generated if the passed seed is None (see Doubt 1), which has to be obtained through mildly sketchy means, so it's better to keep it contained within the function. I don't think passing the seed extremely necessary, but that would somewhat significantly change the actual external API, I tend to be hesitant about changes like this if there's no good reason. Overall I think it's good to keep the seeding.np_random function to keep some consistency with previous versions. The alternative is just completely removing the concept of "gym seeding", and using NumPy. (right now "gym seeding" is basically an interface for NumPy seeding)

Doubt number 4:

Pinging @araffin as there's a possibility this will (again) break some old pickled spaces in certain cases, and I know this was an issue with SB3 and the model zoo. Specifically, if you create a Space with the current master branch, run space.sample() at least once, and then pickle it, it will be pickled with a RandomState instance, which is now considered a legacy generator in NumPy. If you unpickle it using new gym code (i.e. this PR), space.np_random will still point to a RandomState, but the rest of the code expects space.np_random to be a Generator instance, which has a few API changes (see the beginning of this post).

Overall I don't know how important it is for the internal details of gym objects to remain the same - which is more or less necessary for old objects to be unpicklable in new gym versions. There's probably a way of a custom unpickling protocol as a compatibility layer - I'm not sufficiently familiar with this to do it, but I imagine it should be doable on the user side? (i.e. not in gym)

Doubt number 2137: (very low importance)

This doesn't technically solve #2210. IMHO this is absolutely acceptable, because np.random.seed is part of the legacy seeding mechanism, and is officially discouraged by NumPy. Proper usage yields expected results:

import numpy as np
from gym.utils import seeding

user_given_seed = None
np_random, seed = seeding.np_random(user_given_seed)

# since in some places np.random.randn or similar is used we want to seed the global numpy random generator as well
rng = np.random.default_rng(seed)

tl;dr

This definitely needs another set of eyes on it because I'm not confident enough about the nitty-gritty details of NumPy RNG. There are a few things I'm not 100% happy with from a stylistic point of view, but as far as my understanding goes, the functionality is what it should be. There's also the question of supporting old pickled objects, which I think is a whole different topic that needs discussing now that gym is maintained again.

Hello!

It's probably some silly mistake on my side, but i wasn't able to fix by random lever pulling, as usual.

Installing Box2d as in instuctions (using pip install -e .[all]) will throw error when trying to use some of Box2D examples.

Code that reproduces the issue:

import gym
atari = gym.make('LunarLander-v0')
atari.reset()

[2016-05-16 02:14:25,430] Making new env: LunarLander-v0

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-1-f89e78f4410b> in <module>()
      1 import gym
----> 2 atari = gym.make('LunarLander-v0')
      3 atari.reset()
      4 #plt.imshow(atari.render('rgb_array'))

/home/jheuristic/yozhik/gym/gym/envs/registration.pyc in make(self, id)
     77         logger.info('Making new env: %s', id)
     78         spec = self.spec(id)
---> 79         return spec.make()
     80 
     81     def all(self):

/home/jheuristic/yozhik/gym/gym/envs/registration.pyc in make(self)
     52             raise error.Error('Attempting to make deprecated env {}. (HINT: is there a newer registered version of this env?)'.format(self.id))
     53 
---> 54         cls = load(self._entry_point)
     55         env = cls(**self._kwargs)
     56 

/home/jheuristic/yozhik/gym/gym/envs/registration.pyc in load(name)
     11 def load(name):
     12     entry_point = pkg_resources.EntryPoint.parse('x={}'.format(name))
---> 13     result = entry_point.load(False)
     14     return result
     15 

/home/jheuristic/thenv/local/lib/python2.7/site-packages/pkg_resources/__init__.pyc in load(self, require, *args, **kwargs)
   2378         if require:
   2379             self.require(*args, **kwargs)
-> 2380         return self.resolve()
   2381 
   2382     def resolve(self):

/home/jheuristic/thenv/local/lib/python2.7/site-packages/pkg_resources/__init__.pyc in resolve(self)
   2384         Resolve the entry point from its module and attrs.
   2385         """
-> 2386         module = __import__(self.module_name, fromlist=['__name__'], level=0)
   2387         try:
   2388             return functools.reduce(getattr, self.attrs, module)

/home/jheuristic/yozhik/gym/gym/envs/box2d/__init__.py in <module>()
----> 1 from gym.envs.box2d.lunar_lander import LunarLander
      2 from gym.envs.box2d.bipedal_walker import BipedalWalker, BipedalWalkerHardcore

/home/jheuristic/yozhik/gym/gym/envs/box2d/lunar_lander.py in <module>()
      3 from six.moves import xrange
      4 
----> 5 import Box2D
      6 from Box2D.b2 import (edgeShape, circleShape, fixtureDef, polygonShape, revoluteJointDef, contactListener)
      7 

/home/jheuristic/thenv/local/lib/python2.7/site-packages/Box2D/__init__.py in <module>()
     18 # 3. This notice may not be removed or altered from any source distribution.
     19 #
---> 20 from .Box2D import *
     21 __author__ = '$Date$'
     22 __version__ = '2.3.1'

/home/jheuristic/thenv/local/lib/python2.7/site-packages/Box2D/Box2D.py in <module>()
    433     return _Box2D.b2CheckPolygon(shape, additional_checks)
    434 
--> 435 _Box2D.RAND_LIMIT_swigconstant(_Box2D)
    436 RAND_LIMIT = _Box2D.RAND_LIMIT
    437 

AttributeError: 'module' object has no attribute 'RAND_LIMIT_swigconstant'

What didn't help:

pip uninstall gym
apt-get install -y python-numpy python-dev cmake zlib1g-dev libjpeg-dev xvfb libav-tools xorg-dev python-opengl
git clone https://github.com/openai/gym
cd gym
pip install -e .[all] --upgrade

The OS is Ubuntu 14.04 Server x64 It may be a clue that i am running the thing from inside python2 virtualenv (with all numpys, etc. installed)

Description

step method is changed to return five items instead of four.

Old API - done=True if episode ends in any way.

New API - terminated=True if environment terminates (eg. due to task completion, failure etc.) truncated=True if episode truncates due to a time limit or a reason that is not defined as part of the task MDP

Link to docs - https://github.com/Farama-Foundation/gym-docs/pull/115 (To be updated with latest changes)

Changes

1. All existing environment implementations are changed to new API without direct support for old API. However gym.make for any environment will default to old API through a compatibility wrapper.
2. Vector env implementations are changed to new API, with backward compatibility for old API, defaulting to old API. New API can set by a newly added argument new_step_api=True in constructor.
3. All wrapper implementations are changed to new API, and have backward compatibility and default to old API (can be switched to new API with new_step_api=True).
4. Some changes in phrasing - terminal_reward, terminal_observation etc. is replaced with final_reward, final_observation etc. The intention is to reserve the 'termination' word for only if terminated=True. (for some motivation, Sutton and Barto uses terminal states to specifically refer to special states whose values are 0, states at the end of the MDP. This is not true for a truncation where the value of the final state need not be 0. So the current usage of terminal_obs etc. would be incorrect if we adopt this definition)
5. All tests are continued to be performed for old API (since the default is old API for now). A single exception for when the test env is unwrapped and so the compatibility wrapper doesn't apply. Also, special tests are added just for testing new API.
6. new_step_api argument is used in different places. It's meaning is taken to be "whether this function / class should output step values in new API or not". Eg. self.new_step_api in a wrapper signifies whether the wrapper's step method outputs items in new API (the wrapper itself might have been written in new or old API, but through compatibility code it will output according to self.new_step_api)
7. play.py alone is retained in old API due to the difficulty in having it be compatible for both APIs simultaneously, and being slightly lower priority.

StepAPICompatibility Wrapper

1. This wrapper is added to support conversion from old to new API and vice versa.
2. Takes new_step_api argument in __init__. False (old API) by default.
3. Wrapper applied at make with new_step_api=False by default. It can be changed during make like gym.make("CartPole-v1", new_step_api=True). The order of wrappers applied at make is as follows - core env -> PassiveEnvChecker -> StepAPICompatibility -> other wrappers

step_api_compatibility function

This function is similar to the wrapper, it is used for backward compatibility in wrappers, vector envs. It is used at interfaces between env / wrapper / vector / outside code. Example usage,

# wrapper's step method
def step(self, action):

    # here self.env.step is made to return in new API, since the wrapper is written in new API
    obs, rew, terminated, truncated, info = step_api_compatibility(self.env.step(action), new_step_api=True) 

    if terminated or truncated:
        print("Episode end")
    ### more wrapper code

    # here the wrapper is made to return in API specified by self.new_step_api, that is set to False by default, and can be changed according to the situation
    return step_api_compatibility((obs, rew, terminated, truncated, info), new_step_api=self.new_step_api) 

TimeLimit

1. In the current implementation of the timelimit wrapper, existence of 'TimeLimit.truncated' key in info means that truncation has occurred. The boolean value it is set to refers to whether the core environment has already ended. So, info['TimeLimit.truncated']=False, means the core environment has already terminated. We can infer terminated=True, truncated=True from this case.
2. To change old API to new, the compatibility function first checks info. If there is nothing in info, it returns terminated=done and truncated=False as there is no better information available. If TimeLimit info is available, it accordingly sets the two bools.

Backward Compatibility

The PR attempts to achieve almost complete backward compatibility. However, there are cases which haven't been included. Environments directly imported eg. from gym.envs.classic_control import CartPoleEnv would not be backward compatible as these are rewritten in new API. StepAPICompatibility wrapper would need to be used manually in this case. Envs made through gym.make all default to old API. Vector and wrappers also default to old API. These should all continue to work without problems. But due to the scale of the change, bugs are expected.

Warning Details

Warnings are raised at the following locations:

1. gym.Wrapper constructor - warning raised if self.new_step_api==False. This means any wrapper that does not explicitly pass new_step_api=True into super() will raise the warning since self.new_step_api=False by default. This is taken care of by wrappers written inside gym. Third party wrappers will face a problem in a specific situation - if the wrapper is not impacted by step API. eg. a wrapper subclassing ActionWrapper. This would work without any change for both APIs, however to avoid the warning, they still need to pass new_step_api=True into super(). The thinking is - "If your wrapper supports new step API, you need to pass new_step_api=True to avoid the warning".
2. PassiveEnvChecker, passive_env_step_check function - if step return has 4 items a warning is raised. This happens only once since this function is only run once after env initialization. Since PassiveEnvChecker is wrapped first before step compatibility in make, this will raise a warning based on the core env implementation's API.
3. gym.VectorEnv constructor - warning raised if self.new_step_api==False.
4. StepAPICompatibility wrapper constructor - the wrapper that is applied by default at make. If new_step_api=False, a warning is raised. This is independent of whether the core env is implemented in new or old api and only depends on the new_step_api argument.

• [x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [x] This change requires a documentation update

Checklist:

• [x] I have run the pre-commit checks with pre-commit run --all-files
• [x] I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation (need to update with latest changes)
• [x] My changes generate no new warnings (only intentional warnings)
• [ ] I have added tests that prove my fix is effective or that my feature works (added two tests but maybe needs more to be comprehensive)
• [x] New and existing unit tests pass locally with my changes
• [x] Locally runs with atari / pybullet envs

Changes made:

• Create Viewer() class to render window in "human" mode with dm_viewer and glfw
• Modified the default viewer_setup() method for all mujoco_environments (only for v3 envs)

New render API

Following this discussion: #2540

This PR extends the render() method, allowing the user to specify render_mode during the instantiation of the environment. The default value of render_mode is None; in that case, the user can call render with the preferred mode as before. In this way, these changes are backwards compatible. In case render_mode != None, the argument mode of .render() is ignored. In case render_mode = "human", the rendering is handled by the environment without needing to call .render(). With other render modes, .render() returns the results as before. We also introduce _list mode that returns a List with all the renders since the last .reset()or .render(). For example, with render_mode = "rgb_array_list", .render() returns a List of np.ndarray, while with render_mode = "ansi" a List[str].

TODO

• [x] Add deprecation warnings to mode arg in .render() and VideoRecorder

Examples

import gym

env = gym.make('FrozenLake-v1', render_mode="human")
env.reset()
for _ in range(100):
    env.step(env.action_space.sample())
    # env renders automatically, no needs to call .render()
    
env.render()
> None

import gym

env = gym.make('CartPole-v1', render_mode="rgb_array_list")
env.reset()

for _ in range(10):
    env.step(env.action_space.sample())

frames = env.render()
type(frames)
> <class 'list'>
len(frames)
> 11
len(env.render()) # expect 0 because frames are popped by previous .render() call
> 0

env.reset()
len(env.render())
> 1

Example of backward compatibility:

import gym

env = gym.make('FrozenLake-v1')  # default render_mode=None
env.reset()
for _ in range(100):
    # no rendering handled by the environment since render_mode = None
    env.step(env.action_space.sample()) 
    env.render()  # render with human mode (default)
   

I'm using MacOS. Since the python script finished, it will print such errors:

It's a script will cause this problem:

import gym
env = gym.make('SpaceInvaders-v0')
env.reset()
env.render()

And after executing it, the error occurs:

~/G/qlearning $ python atari.py
Exception ignored in: <bound method SimpleImageViewer.__del__ of <gym.envs.classic_control.rendering.SimpleImageViewer object at 0x1059ab400>>
Traceback (most recent call last):
  File "/Users/louchenyao/anaconda3/lib/python3.6/site-packages/gym/envs/classic_control/rendering.py", line 347, in __del__
  File "/Users/louchenyao/anaconda3/lib/python3.6/site-packages/gym/envs/classic_control/rendering.py", line 343, in close
  File "/Users/louchenyao/anaconda3/lib/python3.6/site-packages/pyglet/window/cocoa/__init__.py", line 281, in close
  File "/Users/louchenyao/anaconda3/lib/python3.6/site-packages/pyglet/window/__init__.py", line 770, in close
ImportError: sys.meta_path is None, Python is likely shutting down

It doesn't affect the environment running. It just a little annoying.

Configuration:

Dell XPS15 Anaconda 3.6 Python 3.5 NVIDIA GTX 1050

I installed open ai gym through pip. When I run the below code, I can execute steps in the environment which returns all information of the specific environment, but the render() method just gives me a blank screen. When I exit python the blank screen closes in a normal way.

Code:

import gym
env = gym.make('CartPole-v0')
env.reset()
env.render()
for i in range(1000):
    env.step(env.action_space.sample())

After hours of google searching I think the issue might have something to do with pyglet, the package used for rendering, and possibly a conflict with my nvidia graphics card? All help is welcome. Thanks!

Hi,

I am a beginner with gym. After I render CartPole

env = gym.make('CartPole-v0') env.reset() env.render()

Window is launched from Jupyter notebook but it hangs immediately. Then the notebook is dead. I am using Python 3.5.4 on OSX 10.11.6. What could be the problem here?

Thanks.

In order to use MuJoCo on a recent mac, you need to be using MuJoCo 1.5: https://github.com/openai/mujoco-py/issues/36 Otherwise you get:

>>> import mujoco_py
ERROR: Could not open disk

This is because MuJoCo before 1.5 doesn't support NVMe disks.

Gym depends on MuJoCo via mujoco-py, which just released support for MuJoCo 1.5.

It looks like maybe this is something you're already working on? Or would it be useful for me to look into fixing it?

Given DeepMinds acquisition of MuJoCo and past discussions about replacing MuJoCo environments in Gym, I would like to clarify plans going forward after meeting with the Brax/PyBullet/TDS team at Google and the MuJoCo team at DeepMind.

1. We are going to be replacing the documented MuJoCo environments of the "MuJoCo" class with Brax based environments in the "Phys3D" class, add a deprecation warning to the "MuJoCo" environments and move them to a separate deprecated repo some months later. This raises several questions -"Why do the MuJoCo environments have to be replaced?" Despite MuJoCo being free, right now, the Gym environments have numerous bugs in simulation configuration and have code in a state that we are not able to maintain them. Moreover, they all depend on MuJoCo-Py, which is now fully deprecated and cannot be reasonably maintained. Given this, to use the environments with the more updated free versions of MuJoCo, to fix bugs and to be able to continue do basic maintenance like using new Python versions, the environments would have to be very nearly rewritten from scratch. This means that a serious discussion of a change of simulator is appropriate. -"Of all the simulators available, why Brax?" First lets list the possible widely used options: PyBullet, MuJoCo, TDS and Brax. PyBullet, which was originally is the obvious choice, no longer seriously maintained in favor of TDS and Brax. Each simulators have pros and cons. TDS has full differentiability, Brax has accelerator support (the environments run on GPUs or TPUs allowing training to go orders of magnitude faster- e.g. full training in minutes), and PyBullet and MuJoCo are more physically accurate. For the "MuJoCo" environment class, this high level of physical accuracy is not necessary. Accordingly, picking newer simulators with extra feature of use to researchers (differentiability or hardware acceleration support) is likely the preferable option. I personally believe that hardware accelerator support is more important, hence choosing Brax. -"How long will this take?" We hope to have a release with the Brax based Phys3D environments within the next 5 weeks and a lot of progress has already been made, but it a definite date is difficult to say. For the most recent updates, see https://github.com/google/brax/issues/49

2. The "Robotics" environments are being moved out of Gym. This in turn raises several questions: -"Why can't they be maintained as is?" These environments have the same problems with being unmaintainable and having serious bugs as the others in the "MuJoCo" class with hopper and so on do. -"Why can't these be rewritten in Brax like the others?" Brax not physically accurate enough to support such complex simulations well, and while they hope to support this in the future it will take a very long time. -"I use the Robotics environments, were are they going?" ~Into a repo maintained by @Rohan138 , unless someone who is capable of maintaining them to a higher level and wants to reaches out to me. They will still be maintained as best as is reasonably possible in their state, be installable, and be listed as third party environments in Gym.~ https://github.com/Farama-Foundation/gym-robotics -"Shouldn't Gym have robotics environments like this though? Why not rewrite them in a manner that's suitable?" Because I don't think Gym inherently should have them and because we can't. My goal is to make all the environments inside Gym itself good general purpose benchmarks suitable that someone new to the field of reinforcement learning can look at and say "okay, there's are the things everyone uses that I should play with and understand." The robotics environments after many years have never filled this role and have become niche environments specifically for HER and similar research, and while I cannot speak personally to this matter, the robotics researchers I've spoken to say that these environments are no longer widely used in this space, and that forks of them are used instead, which further means these should not live in Gym proper. Regarding why we can't, these would literally have to be rewritten in the new version of MuJoCo (as Pybullet is no longer extensively maintained) and it's new coming Python bindings (which will not be released publicly for many months, likely with Python bindings following later), and that's not something anyone I'm aware of is willing to do due to the utterly extraordinary amount of work required, including the MuJoCo team at Deepmind. -"When will this happen?" Whenever the next release of Gym comes out.

3. The Box2D environments will be rewritten in Brax in a new Phys2D environment class, and the Box2D environments will be deprecated and then removed, similar to the MuJoCo environments. In this process, the duplicate versions of lunar lander and bipedal walker will be consolidated into one environment, with the different modes as arguments on creation. To answer the natural questions about this here as well: -"Why do they need to be rewritten?" This is discussed in https://github.com/openai/gym/issues/2358, but in a nutshell the physics engine they're using using (Box2D) has Python bindings that have not been maintained for years, meaning that they'll stop supporting new Python versions, architectures, and other basic maintenance things. After many discussions over months, I cannot get these bindings maintained by basically anyone. Additionally, using pyglet for rendering has been a source of continual problems for Gym and it does not reasonably support headless rendering (an essential feature). -"Why Brax?" Originally I was planning to use the other major 2D physics library (chipmunk, which has well maintained Python bindings), but Brax is orders of magnitude faster as it can run or accelerators and the Brax team is kind enough to be willing to do the replacements for us. -"When will this happen?" Probably a month after the Phys3D environments are merged at the current rate, but that's not a timeline people have committed to or anything.

4. General questions: -"These Brax environments can still run on my CPU too, right?" Yep! -"Can Brax environments run on AMD GPUs?" With some effort, yes. Brax uses Jax, which uses XLA, which has solid beta support for most AMD GPUs. -"Why are you having Gym so heavily depend on Brax?" Because I think that it's the best option for environments that already need to be rewritten, and because I think that letting the benchmark environments run orders of magnitude faster via accelerators is of profound value to the community and to beginners in the field. -"Is Brax going to be maintained for the long term?" As long as we can realistically expect, yes. All software stands risk of deprecation, e.g. PyBullet, the Box2D Python bindings (and arguably Box2D itself), PIL (what came before pillow), and so on. Given what I've seen that Google is using it for internally, I'm very confident it will be maintained for at least 5 years or so if not longer, which I think is the best we can reasonably plan for. -"Are you going to make other environments hardware accelerated so they can similarly run orders of magnitude faster?" Hopefully! This could be done with the toy text environments and the classic control environments pretty easily through Jax. I have no concrete plans or timeline for this.

Please let me know if anyone has additional questions about the transition here.

>>> import gym
>>> env = gym.make('Copy-v0')
Traceback (most recent call last):
  File "<pyshell#5>", line 1, in <module>
    env = gym.make('Copy-v0')
AttributeError: module 'gym' has no attribute 'make'
>>> 

I wanna Why @jonasschneider

An error appears when I try to run riverride in atari. The remaining atari games do not run with the same error:

[ My code : import gym env = gym.make("Riverraid-v4",render_mode = 'human') env.reset() env.render() ]

[error: File c:\Users\Jang min seock\AppData\Local\Programs\Python\Python38\lib\site-packages\gym\envs\registration.py:640, in make(id, max_episode_steps, autoreset, apply_api_compatibility, disable_env_checker, kwargs) 637 render_mode = None 639 try: --> 640 env = env_creator(_kwargs) 641 except TypeError as e: 642 if ( 643 str(e).find("got an unexpected keyword argument 'render_mode'") >= 0 644 and apply_human_rendering 645 ):

File c:\Users\Jang min seock\AppData\Local\Programs\Python\Python38\lib\site-packages\ale_py\env\gym.py:155, in AtariEnv.init(self, game, mode, difficulty, obs_type, frameskip, repeat_action_probability, full_action_space, max_num_frames_per_episode, render_mode) 152 self.ale.setBool("sound", True) 154 # Seed + Load --> 155 self.seed() ... ---> 13 import libtorrent as lt 15 from typing import Dict 16 from collections import namedtuple

ImportError: DLL load failed while importing libtorrent: The specified module was not found.. ]

my gym version is 0.26.2

Describe the bug When using a newer version of numpy, this DeprecationWarning is shown:

DeprecationWarning: `np.bool8` is a deprecated alias for `np.bool_`.  (Deprecated NumPy 1.24)

Code example Using numpy 1.24:

import gym

env = gym.make('CartPole-v1')
env.reset()
env.step(0)

System Info Describe the characteristic of your environment:

• gym installed via pip install
• OS: MacOS 12.6
• Python: 3.10.8

Checklist

• [X] I have checked that there is no similar issue in the repo (required)

Describe the bug Atari environments do return none on accessing the render_mode attribute in an environment object. This only happens when

Code example

import gym
env = gym.make('ALE/Pong-v5', render_mode='rgb_array')
assert env.render_mode == 'rgb_array'  # this fails because env.render_mode returns None

env = gym.make('CartPole-v0', render_mode='rgb_array')
assert env.render_mode == 'rgb_array'  # passes assertion

System Info M1 Mac, Environment created via conda (no Rosetta emulation, native arm64 wheels)

conda create -n gym python=3.10 -y
conda activate gym
pip install 'gym[all]'

Checklist

• [x] I have checked that there is no similar issue in the repo (required)

Hello,

I have question regarding "gym.vector.SyncVectorEnv", in the documentation there are this explanation : "where the different copies of the environment are executed sequentially".

My question is what do you mean by sequentially?, do you mean with every episode we excecute one env?

Maybe to clarifiy my question, I explain what Iam trying to do:

Iam trying to use DDPG (stable baseline3) to solve a problem.

I would like to know, how can we change the env sampled values with every episode "and it should be reproducible"

for example, assume we have an env where we harvest energy, we assume that the harvested energy is normally distributed, and then in every episode, I will sample DIFFERENT Values of my harvested energy.I would just like to emphasize again, that I would like that the different values of my harvested energy to be reproducible, so I can compare the RL method to other methods.

PS: the customer env is already created where can I change the sampled value with every episode (using seed which gets as input the episode "i"), now my problem how can I fit it to the code using gym and stable baseline.

I think "gym.vector.SyncVectorEnv" is my solution but Iam not sure.

Thank you Best regards

Question

My gym version is 0.17.0. I create an Hopper-v2 environment. When I set seed of 10 using env.action_space.seed(seed), I get the following output:

env.action_space.seed(10)
[10]
env.action_space.sample()
array([ 0.18566807,  0.43384928, -0.9764525 ], dtype=float32)
env.action_space.sample()
array([0.5614053 , 0.0720133 , 0.56162614], dtype=float32)
env.action_space.sample()
array([-0.02573838,  0.19284548, -0.83452713], dtype=float32)

Similarly, when I set seed of 10 using env.action_space.np_random.seed(seed), I get the following output:

env.action_space.np_random.seed(10)
env.action_space.sample()
array([ 0.5426413 , -0.9584961 ,  0.26729646], dtype=float32)
env.action_space.sample()
array([ 0.49760777, -0.00298598, -0.5504067 ], dtype=float32)
env.action_space.sample()
array([-0.60387427,  0.5210614 , -0.66177833], dtype=float32)

Why is there a difference between the action samples?