Real robot interface and utilities for asynchronous real-time deployment.
Previously developed to connect with ManiSkill2.
Runs camera capturing and visualization as separate processes to make it closer to using ROS.
From PyPI
python3 -m pip install -U real_robotFor different functionalities, you can install the optional dependencies as follows:
real_robot:SharedObjectsupport only, please only use submodulereal_robot.utils.multiprocessing.real_robot[vis]: support for most visualization, use submodulereal_robot.utils.visualization.real_robot[vis_all]: include support for URDF.real_robot[all]: all optional dependencies (e.g.,pyrealsense2,sapien, etc.).
If you need to install the optional xarm dependency, you must install from GitHub repo
python3 -m pip install -U real_robot[all,xarm]@git+https://github.com/KolinGuo/RealRobot.gitRun unittests with
python3 -m pip install -U pytest
python3 -m pytest [--collect-only]Calibrated camera poses are stored in real_robot/assets/hec_camera_poses/ and loaded in real_robot/sensors/camera.py.
-
When using
real_robot.utils.multiprocessing.SharedObject, you might encounter the followingUserWarningwhenCtrl-Cyour program:/usr/lib/python3.12/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 10 leaked shared_memory objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d 'This is because of the undefined order of processes (main and child processes) shutdown. You can safely ignore it as long as there's no any shared memory object that's created by you under
/dev/shmleft. -
If you see warning log messages similar to the following
Implicitly overwriting data of <SharedObject: name=start_rs_front_camera, data_type=<class 'bool'>, nbytes=10>
this can mean one of the following:
- the shared_memory objects created by previous processes are not properly cleaned up.
Runto clean them all up.find /dev/shm -user $USER -exec rm -v {} \;
- there's a bug in your usage of
SharedObject.
By design,SharedObjectcannot mutate its data type (and data size) after created.
So you should first create a newSharedObjectin process A withSharedObject("test", data=np.ones(10))to initialize it.
Then in process B, you can create it withSharedObject("test")and then use the.fetch()/.assign()methods.
The warning will appear if the order is reversed.
To control the order, you can use the helper methodsstart_and_wait_for_process()andsignal_process_ready()underreal_robot.utils.multiprocessingto synchronize the processes.
- the shared_memory objects created by previous processes are not properly cleaned up.
- Capture Color/Depth/IR images from RS camera, do
python3 -m real_robot.tools.rs_capture
- Get detailed device information of connected RealSense device
(e.g., supported stream configs, sensor intrinsics, extrinsics between sensors), do
python3 -m real_robot.tools.enumerate_rs_devices > info.log - Convert recorded ROS bag file and save rgb & depth images into .npz, do
python3 -m real_robot.tools.convert_rosbag_to_frames <bag_path>
- Add type support for
SharedObject-
complex -
bytearray -
tuple -
list -
set -
dict- [Preliminary] Support only 1
np.ndarrayas values
- [Preliminary] Support only 1
-
- Add string length? A 8-bytes length of strings is
2**(8*8-1)-1 = 8 exa bytes, which should be enough. - Ensure compound data types works with recursive compound structure
- Add prefixes for all
SharedMemorycreated. - Proper clean up zombie
SharedMemorybefore or after multiprocessing code
To fix an unindexed rosbag recorded from RSDevice, do
apt install -y python3-rosbag
rosbag reindex <bag_path>- When controlling xArm7 with
motion_mode="cartesian_online"andwait=True, sometimes the robot will be stuck indefinitely inwait_move()due toget_state()returns wrong state (i.e., returnsstate=1thinking it's still in motion). Simple solution can be just to control briefly via UFACTORY Studio to get it unstuck.
0.1.0
- Added
SharedObjectto create/mount objects stored inSharedMemory - Enabled
RSDeviceto run as a separate process (nowCamerawill createRSDeviceas a separate process) - Enabled
RSDeviceto record camera streams as a rosbag file - Enabled
XArm7to run as a separate process (for streaming robot states) - Enabled
CV2VisualizerandO3DGUIVisualizerto run as separate processes (for visualization) - Added a default
FileHandlerto all Logger created throughreal_robot.utils.logger.get_logger - Allow enabling selected camera streams from
RSDeviceandsensors.camera.Camera - Added
sensors.simsense_depth.SimsenseDepthclass to generate depth image from stereo IR images
real_robot.agents.xarm- Change
XArm7parameters for clarity (safety_boundary=>safety_boundary_mm,boundary_clip_eps=>boundary_clip_mm) - Add
get_gripper_position()to get gripper opening width in mm or m - Add
gripper_speedparameter toset_action()to control gripper speed
- Change
real_robot.utils.visualization.visualizer- Rename
Visualizermethodshow_observation()=>show_obs()
- Rename
real_robot.sensors.cameraCameraConfignow accepts aconfigparameter- Rename
CameraConfigparameterparent_pose_fn=>parent_pose_so_name
real_robot.utils.realsenseRSDevicenow acceptsdevice_sninstead of anrs.deviceRSDevicenow acceptsconfigas parameter (width,height,fps) instead ofrs.config
- Switch from
gymtogymnasium - Rename all
control_modeby removingpd_prefix for clarity. No PD controller is used. real_robot.agents.xarmXArm7will not clear "Safety Boundary Limit" error automatically inset_action()- For
motion_mode == "position", switch from usingset_tool_position()toset_position() - Enable gripper and set to maximum speed in
reset()
- Remove all Loggers created as global variables (they will be created
at import, which might not be saved under
REAL_ROBOT_LOG_DIR) - Bugfix in xArm-Python-SDK: enable
wait=Truefor modes other than position mode
0.0.2
- Added motion_mode to XArm7 agent
- Added several control_mode:
pd_ee_pos,pd_ee_pose_axangle,pd_ee_delta_pose_axangle,pd_ee_pose_quat,pd_ee_delta_pose_quat