In complementary to the official Reality Fuison repository, this repository conatins a Unity project that contains a package for teleoperation turtlebot 3 burger robot in immersive virtual reality environment.
This project uses a modified version of the turtlebot, which replaces the rapsberry pi microcontroller with a Jetson TX2 embeded system. The turtle bot is also connected to a stereoscopic camera (ZED Mini).
Full Code Coming Soon...
User could control the turtle bot movement via VR joystick
User could set the target location for the robot to navigate to in VR.
User could control the turtlebot robot with seeing the real-time 3D point cloud stream from the robot's perspective
User could teleoperate the turtlebot by fusing 3D gaussian splatting with real-time point clouds. For reality fusion, please check out the Reality Fuison repository.
- Reality Fusion Robot Setup: Documentation for setting up the robot, including guides in installing the required packages on Jetson TX2.
- Robot Startup Guide: Documentation for starting the modified turtlebot for teleoperation in the real world, after following the Robot Setup.
- Development Tutorial: Documentation of the development process of this toolkit.
If this toolkit helps your research, please consider citing the following work:
@INPROCEEDINGS{realityFusion,
author={Li, Ke and Bacher, Reinhard and Schmidt, Susanne and Leemans, Wim and Steinicke, Frank},
booktitle={2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
title={Reality Fusion: Robust Real-time Immersive Mobile Robot Teleoperation with Volumetric Visual Data Fusion},
year={2024},
volume={},
number={},
pages={8982-8989},
keywords={Visualization;Three-dimensional displays;Telepresence;Robot control;Virtual reality;Robot sensing systems;Rendering (computer graphics);Spatial resolution;Streams;Research and development},
doi={10.1109/IROS58592.2024.10802431}}
Contact: keli95566@gmail.com
Link to paper: Reality Fusion: Robust Real-time Immersive Mobile Robot Teleoperation with Volumetric Visual Data Fusion
This work was supported by DASHH (Data Science in Hamburg - HELMHOLTZ Graduate School for the Structure of Matter) with the Grant-No. HIDSS-0002.
The MIT License in this project does not apply to the turtlebot urdf models in used.