Hi

I'm on Ubuntu 16.04 and following the guidelines from https://github.com/deepmind/lab/blob/master/docs/build.md . I'm unable to run any of the examples

Here are the major library versions:

• Bazel version: 0.4.3
• Lua: 5.1.5
• Python: 2.7
• OpenGL version: 4.5.0
• GCC: 5.4.0

Error while trying to run a random agent

lab$ bazel run :game -- --level_script tests/demo_map --verbose_failures
WARNING: Output base '/home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b' is on NFS. This may lead to surprising failures and undetermined behavior.
INFO: Found 1 target...
ERROR: /home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b/external/jpeg_archive/BUILD:74:1: Executing genrule @jpeg_archive//:configure failed: linux-sandbox failed: error executing command /home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b/execroot/lab/_bin/linux-sandbox ... (remaining 5 argument(s) skipped).
src/main/tools/linux-sandbox-pid1.cc:398: "remount(NULL, /home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b/bazel-sandbox/905e0eef-0788-42e6-8852-7b444149d38c-13/tmp/home/ndg/projects, NULL, 2101281, NULL)": No such file or directory
Target //:game failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 2.569s, Critical Path: 1.20s
ERROR: Build failed. Not running target.

On trying to build the Python interface to DeepMind Lab with OpenGL

lab$ bazel build :deepmind_lab.so --define headless=glx --verbose_failures
WARNING: Output base '/home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b' is on NFS. This may lead to surprising failures and undetermined behavior.
INFO: Found 1 target...
ERROR: /home/ml/hsatij/code/libs/lab/BUILD:972:1: C++ compilation of rule '//:dmlablib' failed: linux-sandbox failed: error executing command 
  (cd /home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b/bazel-sandbox/ff30fcd2-9759-4ca4-8fa3-b82956431988-1/execroot/lab && \
  exec env - \
  /home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b/execroot/lab/_bin/linux-sandbox @/home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b/bazel-sandbox/ff30fcd2-9759-4ca4-8fa3-b82956431988-1/linux-sandbox.params -- /usr/bin/gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -Wl,-z,-relro,-z,now -B/usr/bin -B/usr/bin -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-canonical-system-headers -fno-omit-frame-pointer '-std=c++0x' -MD -MF bazel-out/local-fastbuild/bin/_objs/dmlablib/public/dmlab_so_loader.pic.d '-frandom-seed=bazel-out/local-fastbuild/bin/_objs/dmlablib/public/dmlab_so_loader.pic.o' -fPIC -iquote . -iquote bazel-out/local-fastbuild/genfiles -iquote external/bazel_tools -iquote bazel-out/local-fastbuild/genfiles/external/bazel_tools -isystem external/bazel_tools/tools/cpp/gcc3 '-DDMLAB_SO_LOCATION="libdmlab.so"' -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c public/dmlab_so_loader.cc -o bazel-out/local-fastbuild/bin/_objs/dmlablib/public/dmlab_so_loader.pic.o).
src/main/tools/linux-sandbox-pid1.cc:398: "remount(NULL, /home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b/bazel-sandbox/ff30fcd2-9759-4ca4-8fa3-b82956431988-1/tmp/home/ndg/projects, NULL, 2101281, NULL)": No such file or directory
Target //:deepmind_lab.so failed to build
INFO: Elapsed time: 2.294s, Critical Path: 1.11s

I'm new to bazel and unfortunately the error logs are too cryptic for me. Any help to resolve this will be appreciated !

Hi,

I built lab on Ubuntu 14.04. The python module tests all pass. I'm a little confused about importing deepmind_lab and running experiments with it in python. I apologize if this is due to my lack of general python/bazel knowledge. My understanding is that the process of experimentation with deepmind_lab is:

1. create a python file experiment.py. import deepmind_lab and use it in the experiment.
2. add a py_binary entry in the BUILD file for bazel named "experiment"
3. perform bazel run :experiment

Is this correct? And, is there any way to instead run experiments directly with python experiment.py?

I have implemented A3C with multiprocessing (+ pytorch) as opposed to using threads, however bazel run seems to break silently and clearly without any visible trace. This is what I do:

$ bazel run :struct_runner --define headless=false
[...]
$ echo $?  # this is the error code of the previous process
8

struct_runner.py initialises a lab environment, then creates a bunch of processes in which more envs are created. In particular, the silent crash happens when I create a process p and do p.start() - it also appears to be non-deterministic with respect to the number of processes I manage to spawn before bazel kills them and quits.

I know that @miyosuda has implemented A3C using threads here, however multiprocessing is supported very well by pytorch and it would be a shame to have to deal with threads management.

Hey on Ubuntu 18.04, Bazel 0.20.0, Tensorflow 1.2 GPU I am trying to run the random agent and I am getting the following error. Can you please help me on how to solve this error? Thanks

bazel run :python_random_agent --define graphics=sdl -- \

           --length=10000 --width=640 --height=480

Starting local Bazel server and connecting to it... INFO: Invocation ID: 69a115a9-45b9-4642-87c8-8fc2170651ac INFO: SHA256 (https://github.com/abseil/abseil-cpp/archive/master.zip) = d3bb4e5578f06ddf3e0e21def6aabf3b4ae81d68b909f06cfcb727734584cab3 INFO: Analysed target //:python_random_agent (53 packages loaded, 3598 targets configured). INFO: Found 1 target... ERROR: /home/neuwong/lab/BUILD:790:1: C++ compilation of rule '//:game_lib_sdl' failed (Exit 1) gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -MD -MF bazel-out/k8-fastbuild/bin/_objs/game_lib_sdl/sdl_input.pic.d ... (remaining 103 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox engine/code/sdl/sdl_input.c:26:11: fatal error: SDL.h: No such file or directory

include <SDL.h>

       ^~~~~~~

compilation terminated. Target //:python_random_agent failed to build Use --verbose_failures to see the command lines of failed build steps. INFO: Elapsed time: 33.266s, Critical Path: 7.03s INFO: 69 processes: 69 linux-sandbox. FAILED: Build did NOT complete successfully FAILED: Build did NOT complete successfully

Informed by #32, #92 and #52 I tried to configure Bazel on a conda environment, with python 3. bazel build //:deepmind_lab.so appears to run successfully, however bazel run //:python_random_agent is not able to import deepmind_lab

INFO: Running command line: bazel-out/k8-py3-fastbuild/bin/python_random_agent
ImportError: numpy.core.multiarray failed to import
Traceback (most recent call last):
  File "/home/florin/.cache/bazel/_bazel_florin/329274937dbc60c7c6c49b959689f873/execroot/org_deepmind_lab/bazel-out/k8-py3-fastbuild/bin/python_random_agent.runfiles/org_deepmind_lab/python/random_agent.py", line 26, in <module>
    import deepmind_lab
ImportError: numpy.core.multiarray failed to import
ERROR: Non-zero return code '1' from command: Process exited with status 1

You can find the changes bellow. @tkoeppe Does this seem right? Also, are there any log files I can provide you with in order to help with the debugging?

diff --git a/BUILD b/BUILD
index 9e274c5..3972e07 100644
--- a/BUILD
+++ b/BUILD
@@ -984,6 +984,7 @@ py_binary(
     data = [":deepmind_lab.so"],
     main = "python/random_agent.py",
     visibility = ["//python/tests:__subpackages__"],
+    default_python_version = "PY3"
 )
 
 LOAD_TEST_SCRIPTS = [
diff --git a/WORKSPACE b/WORKSPACE
index fa8da47..967c712 100644
--- a/WORKSPACE
+++ b/WORKSPACE
@@ -91,5 +91,5 @@ new_local_repository(
 new_local_repository(
     name = "python_system",
     build_file = "python.BUILD",
-    path = "/usr",
+    path = "/home/florin/Tools/miniconda3/envs/torch30",
 )
diff --git a/python.BUILD b/python.BUILD
index f0b3f9a..b6a6fa8 100644
--- a/python.BUILD
+++ b/python.BUILD
@@ -5,7 +5,11 @@
 
 cc_library(
     name = "python",
-    hdrs = glob(["include/python2.7/*.h"]),
-    includes = ["include/python2.7"],
+    hdrs = glob(["include/python3.6m/*.h",
+                 "lib/python3.6/site-packages/numpy/core/include/**/*.h"
+    ]),
+    includes = ["include/python3.6m",
+                "lib/python3.6/site-packages/numpy/core/include"
+    ],
     visibility = ["//visibility:public"],
 )

Hi,

I am having some problems when trying to build/install Deepmind Lab as a pip package with Python 3. Currently I am capable of generating the package .whl file (DeepMind_Lab-1.0-py3-none-any.whl) and installing in a conda environment, but I am getting the following error when importing the deepmind_lab module:

deepmind_lab.so: undefined symbol: PyCObject_Type

Any suggestions about how to fix this?

Thanks,

bazel build -c opt //:deepmind_lab.so ERROR: /home/ubuntu/zz/projects/lab/WORKSPACE:146:1: //external:tree_archive: no such attribute 'repo_mapping' in 'http_archive' rule ERROR: Skipping '//:deepmind_lab.so': error loading package 'external': Package 'external' contains errors WARNING: Target pattern parsing failed. ERROR: error loading package 'external': Package 'external' contains errors

[email protected]:~$ cd lab [email protected]:~/lab$ sudo bazel run :game -- --level_script=tests/empty_room_test --level_setting=logToStdErr=true --sandbox_debug Extracting Bazel installation... Starting local Bazel server and connecting to it... INFO: SHA256 (https://github.com/abseil/abseil-cpp/archive/master.zip) = f63fa171a79bfd38f995b899989e74144255fcea57ad74079792385841db64dd DEBUG: Rule 'com_google_absl' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "f63fa171a79bfd38f995b899989e74144255fcea57ad74079792385841db64dd" INFO: Analysed target //:game (51 packages loaded, 3679 targets configured). INFO: Found 1 target... ERROR: /home/user/.cache/bazel/_bazel_root/851aceb1df3dfff730804dc58954a97b/external/glib_archive/BUILD.bazel:1:1: Executing genrule @glib_archive//:gen_configure failed (Exit 1) bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox configure: error: *** You must have either have gettext support in your C library, or use the *** GNU gettext library. (http://www.gnu.org/software/gettext/gettext.html)

Target //:game failed to build Use --verbose_failures to see the command lines of failed build steps. INFO: Elapsed time: 254.198s, Critical Path: 46.81s INFO: 154 processes: 154 processwrapper-sandbox. FAILED: Build did NOT complete successfully FAILED: Build did NOT complete successfully

Heya,

I'm currently re-implementing the vector-based navigation architecture of deep-mind's nature paper for tensorflow 2. However, it seems there is an issue regarding the environment of the python API when used with the Keras model API. In particular, I do something like:

...
observations = ['RGB', 'DEBUG.POS.ROT', 'DEBUG.POS.TRANS']
env = deepmind_lab.Lab(level, observations,
                           config=lab_config, renderer='hardware')
env.reset()
...
    def generate_batch(env, batch_size=10):
        while True:
            ...
            obs = env.observations()
            ...
            x, y = ... some computation 
            env.reset()
            ...
            yield x, y


model.fit_generator(
generate_batch(env, batch_size=10),
steps_per_epoch=1000,
verbose=1,
use_multiprocessing=False
)

depending whether i use env.reset() or not it sometimes drops error (related to: https://github.com/deepmind/lab/issues/134). Even if it does not explicitely complains, it stops always within the first batch. Since Tensorflow 2 seems to convert the generate_batch function into some eager-execution/autograph stuff, I am quite confused whether the env object gets either copied or something else nasty going on under the hood. I hope someone has more clue than I do :)

The agent I am using is performing random actions (so it should not be the same state-sequence all the time, especially because the rnd seed is not statically set)

Hi,

After running bazel build -c opt //:deepmind_lab.so, I run bazel run :python_random_agent --define headless=osmesa and get the following output:

Failed to find function dmlab_connect in library!
Traceback (most recent call last):
  File "/network/home/alversaf/.cache/bazel/_bazel_alversaf/c78c672c9c47c1e9c2d9a08f489f56c4/execroot/org_deepmind_lab/bazel-out/k8-fastbuild/bin/python_random_agent.runfiles/org_deepmind_lab/python/random_agent.py", line 206, in <module>
    args.record, args.demo, args.demofiles, args.video)
  File "/network/home/alversaf/.cache/bazel/_bazel_alversaf/c78c672c9c47c1e9c2d9a08f489f56c4/execroot/org_deepmind_lab/bazel-out/k8-fastbuild/bin/python_random_agent.runfiles/org_deepmind_lab/python/random_agent.py", line 155, in run
    env = deepmind_lab.Lab(level, ['RGB_INTERLEAVED'], config=config)
RuntimeError: Failed to connect RL API

What should I do? Thanks! (By the way, I am running bazel through a singularity container.)

Walking through the examples, I haven't understood how can I build the code, and after build have an easy, gym-like environment I could instantiate / reset / step / etc? Everything that I managed was building and starting stuff directly through bazel, similar to python_random_agent, but I don't see how to call step/reset outside of bazel. Am I missing something or there is no description of how to build+run such an environment?

Not an issue, just thought I'd share a full installation script for the DMLab Python package because it took a while to put all the pieces together and might be helpful for others. Tested on Ubuntu 20.04, might also work on newer versions:

#!/bin/sh
set -eu

# Dependencies
apt-get update && apt-get install -y \
    build-essential curl freeglut3 gettext git libffi-dev libglu1-mesa \
    libglu1-mesa-dev libjpeg-dev liblua5.1-0-dev libosmesa6-dev \
    libsdl2-dev lua5.1 pkg-config python-setuptools python3-dev \
    software-properties-common unzip zip zlib1g-dev g++
pip3 install numpy

# Bazel
apt-get install -y apt-transport-https curl gnupg
curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor > bazel.gpg
mv bazel.gpg /etc/apt/trusted.gpg.d/
echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | tee /etc/apt/sources.list.d/bazel.list
apt-get update && apt-get install -y bazel

# Build
git clone https://github.com/deepmind/lab.git
cd lab
echo 'build --cxxopt=-std=c++17' > .bazelrc
bazel build -c opt //python/pip_package:build_pip_package
./bazel-bin/python/pip_package/build_pip_package /tmp/dmlab_pkg
pip3 install --force-reinstall /tmp/dmlab_pkg/deepmind_lab-*.whl
cd ..
rm -rf lab

# Stimuli
mkdir dmlab_data
cd dmlab_data
pip3 install Pillow
curl https://bradylab.ucsd.edu/stimuli/ObjectsAll.zip -o ObjectsAll.zip
unzip ObjectsAll.zip
cd OBJECTSALL
python3 << EOM
import os
from PIL import Image
files = [f for f in os.listdir('.') if f.lower().endswith('jpg')]
for i, file in enumerate(sorted(files)):
  print(file)
  im = Image.open(file)
  im.save('../%04d.png' % (i+1))
EOM
cd ..
rm -rf __MACOSX OBJECTSALL ObjectsAll.zip

apt-get clean

Hi, I'm wondering if it is possible to modify the speed in which the agent walks/runs dynamically/programmatically ? For example, speed up/down according to a function while the agent is interacting in the environment.

Thank you very much for your support,

Is it possible to disable velocity kinematics/dynamics of the agent? In other words, if a "forward" action is taken, the agent moves forward by a fixed amount and stops immediately with no residual velocity carrying it forward?

Hello, Is there a way to get either access to a point-cloud observation or access to the camera params (extrinsics, intrinsics, near/far plane) such that I can calculate the point-cloud from z-buffer observation? Thanks!