8000 GitHub - uhhhci/turtlebot-xr: unity project containing several example scenes for teleoperation of turtle bot in immersive XR environments.
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
8000

unity project containing several example scenes for teleoperation of turtle bot in immersive XR environments.

License

Notifications You must be signed in to change notification settings

uhhhci/turtlebot-xr

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Turtlebot XR Teleoperation Framework

In complementary to the official Reality Fuison repository, this repository conatins a Unity project that contains a package for teleoperation turtlebot 3 burger robot in immersive virtual reality environment.

This project uses a modified version of the turtlebot, which replaces the rapsberry pi microcontroller with a Jetson TX2 embeded system. The turtle bot is also connected to a stereoscopic camera (ZED Mini).

Installation( Full Code Coming Soon...)

Full Code Coming Soon...

Feature 1: Basic Teleoperation in VR

User could control the turtle bot movement via VR joystick turtlebot3 URDF Unity

Feature 2: Go-to navigation planning

User could set the target location for the robot to navigate to in VR.

turtlebot3 URDF Unity

Feature 3: Fusing 2D lidar with real-time 3D point cloud

User could control the turtlebot robot with seeing the real-time 3D point cloud stream from the robot's perspective

turtlebot3 URDF Unity

Feature 4: Reality Fusion

User could teleoperate the turtlebot by fusing 3D gaussian splatting with real-time point clouds. For reality fusion, please check out the Reality Fuison repository.

turtlebot3 URDF Unity

Documentations

  1. Reality Fusion Robot Setup: Documentation for setting up the robot, including guides in installing the required packages on Jetson TX2.
  2. Robot Startup Guide: Documentation for starting the modified turtlebot for teleoperation in the real world, after following the Robot Setup.
  3. Development Tutorial: Documentation of the development process of this toolkit.

Citation

If this toolkit helps your research, please consider citing the following work:

@INPROCEEDINGS{realityFusion,
  author={Li, Ke and Bacher, Reinhard and Schmidt, Susanne and Leemans, Wim and Steinicke, Frank},
  booktitle={2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, 
  title={Reality Fusion: Robust Real-time Immersive Mobile Robot Teleoperation with Volumetric Visual Data Fusion}, 
  year={2024},
  volume={},
  number={},
  pages={8982-8989},
  keywords={Visualization;Three-dimensional displays;Telepresence;Robot control;Virtual reality;Robot sensing systems;Rendering (computer graphics);Spatial resolution;Streams;Research and development},
  doi={10.1109/IROS58592.2024.10802431}}

Contact: keli95566@gmail.com

Link to paper: Reality Fusion: Robust Real-time Immersive Mobile Robot Teleoperation with Volumetric Visual Data Fusion

Acknowledgment

This work was supported by DASHH (Data Science in Hamburg - HELMHOLTZ Graduate School for the Structure of Matter) with the Grant-No. HIDSS-0002.

License

The MIT License in this project does not apply to the turtlebot urdf models in used.

About

unity project containing several example scenes for teleoperation of turtle bot in immersive XR environments.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published
0