Noesis is a software toolbox for applying deep reinforcement learning to robotics problems.
This pre-release version currently is tested under Ubuntu 20.04 LTS, and supports C++14 and Python 3.7.
The toolbox consists of the following main components:
noesis: A CMake package providing the following C++ libraries:framework: Computational and operational core libraries for graphs, tensors, logging, system access etc.mdp: A generalized interface for defining Markov Decision Processes (MDP) (i.e. environments).gym: Interfaces to physics engines, and implementations of various environmentsrl: Interfaces to DRL algorithms and relevant components.
noesis_py: Apippackage providing the Python back-end for building computation-graphs using TensorFlow.examples: Example programs for exploring the functionality and components of the library.
Lastly, all C++ components are built using CMake, but also support catkin.
The latter is typically the tool of choice in the robotics community due to the ubiquity of ROS. For
python, we use virtualenv and virtualenvwrapper for package management.
Maintainer: Vassilios Tsounis
Affiliation: Robotic Systems Lab, ETH Zurich
Contact: vastsoun@gmail.com
Please see these instructions on how to install Noesis and all relevant dependencies.
Building noesis can be performed anywhere in the user's home directory using CMake. We provide an example in
the form of a CMake project template in the utils/workspace directory. We recommend using this
for getting started on own projects. See directory for details.
For building with Catkin, please refer to this resource on how to install, configure and use catkin.
mkdir -p catkin_ws/src
cd catkin_ws
catkin init
catkin config --cmake-args -DCMAKE_BUILD_TYPE=Release
catkin build noesis_examples
Now we can test the build by executing any of the examples. Let's try the framework_hello example:
user@ubuntu:~$ ./build/noesis/noesis_examples/framework_hello
Noesis has apt, pip and source dependencies.
All apt dependencies are installed by the install.sh script (see installation instructions below):
- Python 3.7: Current default version of Python 3.7 provided by the dead-snakes repos.
- GCC 9.4: GNU C/C++ Compiler (GCC) v9 provided by the advanced Ubuntu toolchain repository.
- CMake: CMake is an open-source, cross-platform family of tools designed to build, test and package software.
- OpenMP 4.5: OpenMP 4.5 provided by GCC 9.
- Boost: Free peer-reviewed portable C++ source libraries.
- TinyXML: TinyXML is a simple, small, minimal, C++ XML parser that can be easily integrating into other programs.
- SDL2: SDL is a cross-platform development library providing abstractions based on OpenGL.
- SFML: Simple and Fast Multimedia Library used for drawing 2D graphics.
- STB: STB is a set of single-file public domain libraries for C/C++. Only the parts for image processing are used.
Note: STB is already provided in-source in noesis/include/stb and no action needs to be taken for it's installation.
All pip dependencies are automatically installed when by the noesis_py package via the install.sh script:
- TensorFlow 1.15: Computation using data flow graphs for scalable machine learning.
Noesis has a single source dependency on TensorFlow-Cpp, a CMake-based re-packaging of the C/C++ interface to TensorFlow, and can be retrieved from the respective
tensorflow-cpp GitHub repository. The aforementioned carries with it the following packages:
- Eigen3: Eigen is a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms.
- TensorFlow-Cpp: CMake package providing the headers and libraries for the C/C++ API of TensorFlow.
Please report bugs and request features using the Issue Tracker.
If you use Noesis, please cite as:
@phdthesis{tsounis2023thesis,
year = {2023},
type = {Doctoral Thesis},
institution = {ETH Zurich},
publisher = {ETH Zurich},
school = {ETH Zurich},
author = {Vassilios Tsounis},
title = {Applications of Reinforcement Learning to Motion Planning & Control of Quadrupedal Robots},
copyright = {In Copyright - Non-Commercial Use Permitted}
}
See AUTHORS.md for the complete list of authors and contributors and ACKNOWLEDGEMENTS.md for affiliations and funding accreditation.