Building ROS-O #4

v4hn · 2023-01-23T10:17:45Z

v4hn
Jan 23, 2023
Maintainer

There are multiple (partially) independent efforts to build ROS-O. Here's the list I know of:

https://github.com/v4hn/ros-o-builder
- targets auto-building ros-one-* deb files from source ROS packages for Ubuntu 22.04 (Debian targeted later)
https://github.com/jspricke/ros-one-packages
- build ros-* packages on top of Debian's existing packages for ROS components
https://github.com/lucasw/ros_from_src
- build all ros-related packages (including the build/workspace management tools) from source
https://github.com/lianghongzhuo/ws_ros_src
- build ROS workspace in anaconda for updated python dependencies

lucasw · 2023-02-18T20:48:04Z

lucasw
Feb 18, 2023
Maintainer

build ros-* packages on top of Debian's existing packages for ROS components

I've mostly switched to building on top of debian packages in https://github.com/lucasw/ros_from_src/tree/robot_state_publisher/ubuntu_2204 (and I'm on 22.10 now) - it looks like I should see what ros-one-packages is doing differently. I have a large number of forks, maybe packages ros-one-packages could be using that it isn't yet (though many of the PRs from the forks have been upstreamed, those that aren't probably should go into ros-o for better visibility but the large number is daunting, it's very quick to fork and patch and move on).

3 replies

v4hn Feb 20, 2023
Maintainer Author

it's very quick to fork and patch and move on

just fork into ros-o then. Feel encouraged to do so more ;).
In multiple such cases I found someone else already did it there independently :-)

though many of the PRs from the forks have been upstreamed, those that aren't probably should go into ros-o for better visibility but the large number is daunting,

Cleaning up merged PR branches and migrating those you need in private repositories to this organization would be a very good idea. I hear you about the large number. I still have various uncommitted patches (probably a similar list as you have) which I did not push yet to build on debian bookworm.

My personal plan is to get the current set of ros-o-builder repositories to compile including jsk_rviz_plugins and then setup a similar system building on ros-debian packages for debian testing (which will break in many more details).

lucasw Mar 7, 2023
Maintainer

My personal plan is to get the current set of ros-o-builder repositories to compile including jsk_rviz_plugins

Does https://github.com/lucasw/jsk_visualization/tree/boost_placeholders (did I forget to make a PR for any of that?) work for you or are there other things broken?

v4hn Mar 8, 2023
Maintainer Author

I don't use your forks there, but I will test and migrate them to ros-o if I do (unless you did it by then).

did I forget to make a PR for any of that?

At least there is no PR and no recent commits from you in upstream...

work for you or are there other things broken?

The dependency list for the jsk stacks is quite heavy-weight and I did not work through all of them yet.
Additionally a number of exec_dependency's or broken even on 20.04 and need to be cleaned.
Feel free to look through the reported build errors and point out fixes.

lucasw · 2023-03-07T17:01:23Z

lucasw
Mar 7, 2023
Maintainer

How about a consolidated repos.yaml to be maintained in this repository targeting ubuntu and debian (testing/unstable?) systems, built nightly and maybe even for every push in a github action like https://github.com/jspricke/ros-deb-builder-action or similar. It would nice to have a single source of truth (defined as agreed upon between a handful of us for now) of what all the needed packages are to build all of ros-o that are not yet available via apt, PRs and commits would highlight an addition of a newly supported package (like ros_numpy), or a change from an external branch (like one of mine) to an 'official' ros-o one.

It could be like https://github.com/lucasw/ros_from_src/blob/robot_state_publisher/ubuntu_2204/base_repos.yaml (possibly minus some alterations I made to ros_comm and depth_image_proc, though if anyone is interested those could be put into the ros-o) or https://github.com/jspricke/ros-one-packages/blob/main/sources.repos.

I think there would have to be overlay yamls to support some versions of Ubuntu (for me I'll likely roll forward from 22.10 to 23.04, but the action build could make sure the older versions aren't broken). I'm not that familiar with it but I think Ubuntu is missing debian released packages that are in debian-unstable, so would need an additional repo yaml with all of those, maybe even a per Ubuntu release overlay repos.yaml.

Fast and loose with repo and branch name yaml entries is okay for now but probably that'll cause all kinds of breaks because any triggered build may be pulling in new breaking changes especially for uncontrolled repos outside of ros-o, maybe there could be a stable variant with per-repo commit shas instead of just branches.

If it takes hours to build maybe there could be a set of layers where the lower down packages are built less frequently but their build artifacts can be pulled into the more regular builds.

6 replies

lucasw Mar 7, 2023
Maintainer

Any reason not to move it into ros-o?

jspricke Mar 7, 2023
Maintainer

https://github.com/v4hn/ros-o-builder/issues

v4hn Mar 8, 2023
Maintainer Author

I still consider the ros-o-builder my personal trial&error field, but I got quite far with it now and plan to move it here soon.

Aside from build errors and open issues, one important aspect that still needs to be addressed is web storage for the packages. github enforces a hard 100MB limit on the size of files managed in git and I know of exactly one package for now that violates this limit (eigenpy, a dependency for moveit). As a result we cannot directly add MoveIt yet without tricks. Prominent "community" registries have rather low limits too and I expect we would run into them soon if we tried to use them.
I got in touch with multiple potential academic hosters, but there is no concrete solution yet.

lucasw Mar 14, 2023
Maintainer

Have you looked at https://cloudsmith.com/ or https://packagecloud.io/? It sounds like they have free open source options of so many GB and maybe an (easy?) process for getting it upgraded if needed, was reading about them in rclone/rclone#2153

gavanderhoorn Nov 6, 2023

@v4hn wrote:

I got in touch with multiple potential academic hosters, but there is no concrete solution yet.

I did ask, but sadly there was no interest in hosting this.

I'm glad @rhaschke had more success.

rhaschke · 2023-09-10T21:03:39Z

rhaschke
Sep 10, 2023
Maintainer

I still consider the ros-o-builder my personal trial&error field, but I got quite far with it now and plan to move it here soon.

Thanks for this great initial work. Building on your work, @v4hn and @jspricke, I pushed the GHA-based ros-builder further:
https://github.com/ubi-agni/ros-builder-action

Aside from build errors and open issues, one important aspect that still needs to be addressed is web storage for the packages. github enforces a hard 100MB limit and I know of a package that violates this limit (eigenpy, a dependency for moveit).

I cannot confirm that. For me, eigenpy.deb has just 1.1M. I have build a ROS1 distro comprising MoveIt and MTC successfully:
https://github.com/ubi-agni/ros-repo/tree/jammy-one. The largest .deb was moveit-resources (24M).

I'm also looking into hosting the debian repository outside of github. I've got a web server from our faculty and need to set it up the next days...

I would be interested to get involved in the ros-o organization. As a first step, my ros-builder-action could be migrated here.
But, I would also like to add some ROS packages, which require ros-one fixes. What is your policy on that process?

2 replies

v4hn Sep 11, 2023
Maintainer Author

Great progress! I see you build ros2 compatible packages for jammy.

I cannot confirm that.

According to @jspricke, the eigenpy dbgsym package was that big. I see you do not deploy those packages.
That's an option if you provide support for debuginfod I guess, but no debug-symbols at all seems problematic to me.

I've got a web server from our faculty and need to set it up the next days...

You are in a good position for that I guess. Sadly, I did not get any further followups from the two others I talked with. Admittedly I did not push this much further either in favor of a research project.
A non-github deployment location would be highly appreciated!

I would be interested to get involved in the ros-o organization.

I just invited you into the organization.

I would also like to add some ROS packages, which require ros-one fixes. What is your policy on that process?

If you need patches to build packages on more current distributions, fork the repository to ros-o, ideally have it contain only a single branch (at the moment they are mostly called obese-devel, but it does not matter a lot) and push your fixes there.
If at all possible the codebase should stay backward compatible.
Importantly, also create a pull-request to ask the repository owners to merge the fix, possibly labelled [ROS-O].
This is something you can definitely do wherever needed.

As a first step, my ros-builder-action could be migrated here.

I think that should be the second step. :)
I guess this is at least the fourth build setup for ros-o, probably with its own advantages/disadvantages.
We definitely want an official github builder with a well-defined deployment policy, even if just for validating the current state.
@jspricke could you have a look at it too? I will be extremely busy for the next two weeks at least.

rhaschke Nov 6, 2023
Maintainer

Sorry for not responding yet. I was distracted by other tasks... Thanks for inviting me to the ros-o organization.
Meanwhile, I pushed several updates to ros-o packages. Most importantly, the repo server is available now: https://ros.packages.techfak.net

I also further improved my ros-build-action.

furushchev · 2024-10-31T16:50:30Z

furushchev
Oct 31, 2024
Maintainer

Hello!

I also tried installing ROS-O on Ubuntu 24.04, and it's pretty straightforward! Thank you all for your effort!
Let me report on my installation process here. (Or if there is a place to write something like this, I would be happy to contribute)

Install dependencies

sudo apt update
sudo apt upgrade
sudo apt install python3-full pipx git ssh cmake build-essential
sudo apt install python3-empy 
pipx ensurepath
pipx install vcstool rosinstall_generator catkin_tools
sudo pipx ensurepath
sudo pipx install rosdep

Homebrew gazebo

sudo apt install libfreeimage-dev libtar-dev libqwt-headers

cd ~/workspace/gazebo

# ignition-cmake2
git clone -b ignition-cmake2_2.15.0 https://github.com/gazebosim/gz-cmake
mkdir -p gz-cmake/build && cd gz-cmake/build
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo .. && sudo make install -j

# ignition-math
git clone -b ignition-math6_6.15.1 https://github.com/gazebosim/gz-math
mkdir -p gz-math/build && cd gz-math/build
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo .. && sudo make install -j

# ignition-common
# git clone -b ignition-common3_3.14.2 https://github.com/gazebosim/gz-common
git clone -b obese-devel https://github.com/furushchev/gz-common.git
mkdir -p gz-common/build && cd gz-common/build
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo .. && sudo make install -j

# sdformat
# git clone -b sdformat9_9.10.1 https://github.com/gazebosim/sdformat
git clone -b obese-devel https://github.com/furushchev/sdformat
mkdir -p sdformat/build && cd sdformat/build
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo .. && sudo make install -j

# ignition-msgs
git clone -b ignition-msgs5_5.10.0 https://github.com/gazebosim/gz-msgs
mkdir -p gz-msgs/build && cd gz-msgs/build
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo .. && sudo make install -j

# gazebo
git clone -b gazebo11 https://github.com/osrf/gazebo
mkdir -p gazebo/build && cd gazebo/build
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo .. && sudo make install -j

Import ROS-O repositories and install dependencies using rosdep

mkdir -p ~/workspace/catkin_ws/src
cd ~/workspace/catkin_ws/src
~/obesify.py --output ros-o.repos   # [1]
vcs import --recursive <ros-o.repos

# As of 10/30, the gennodejs package version is incorrect.
# We need to import it manually
rosinstall_generator --rosdistro noetic --upstream-development gennodejs | vcs import

sudo -i rosdep init
rosdep update
rosdep install --from-paths . -i -r -n -y --rosdistro noetic

[1] https://gist.github.com/furushchev/a8d126aed8f160379665cafe632b74e1

Build workspace

cd ~/workspace/catkin_ws
catkin config  --init --merge-devel --install --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo
catkin build

So far, I tested topic pub/sub, service call, Rviz, RQT and Gazebo. All just works well.
(Note: For GUI applications, you will need to set export QT_QPA_PLATFORM=xcb to make them work)

I hope more active users join and the ROS-O become more convenient!

0 replies

mikaelarguedas · 2024-11-09T18:52:19Z

mikaelarguedas
Nov 9, 2024

Would it be useful to have docker images with the ros-o built debian packages installed similar to the official ROS docker images that have the various ROS metapackages installed ?

4 replies

rhaschke Nov 10, 2024
Maintainer

We are building such docker images daily: https://github.com/ubi-agni/ros-builder-action/actions/runs/11753973847
and provide them via dockerhub: https://hub.docker.com/r/ubiagni/ros/tags

mikaelarguedas Nov 11, 2024

great ! thankd @rhaschke

Is there interest / plan for providing some for Ubuntu Noble as well ? and for other arches (e.g. aarch64) ?

Related link: ubi-agni/ros-builder-action#15

mikaelarguedas Nov 11, 2024

Basically what I'm thinking is to provide a base image with ROS-One and ROS Jazzy installed (and maybe a ros1_bridge) so that people can build a hybrid system and a custom bridge. This is under the assumption that I'm not alone in having projects still using Noetic and being willing to migrate to ROS2 but not all at once 😅

gavanderhoorn Nov 12, 2024

Basically what I'm thinking is to provide a base image with ROS-One and ROS Jazzy installed (and maybe a ros1_bridge) so that people can build a hybrid system and a custom bridge.

this is one of the things I'm using this for.

I was even contemplating contributing to ros2/ros1_bridge to get it out of its unmaintained state, but I'm not sure I can dedicate sufficient time to it.

We have been using a custom build -- based on a one-humble combo -- for some time now and it works as expected. The fact it doesn't require any apt tweaks makes things work quite nicely.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Building ROS-O #4

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 5 comments 15 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

Building ROS-O #4

v4hn Jan 23, 2023 Maintainer

Replies: 5 comments · 15 replies

lucasw Feb 18, 2023 Maintainer

v4hn Feb 20, 2023 Maintainer Author

lucasw Mar 7, 2023 Maintainer

v4hn Mar 8, 2023 Maintainer Author

lucasw Mar 7, 2023 Maintainer

lucasw Mar 7, 2023 Maintainer

jspricke Mar 7, 2023 Maintainer

v4hn Mar 8, 2023 Maintainer Author

lucasw Mar 14, 2023 Maintainer

gavanderhoorn Nov 6, 2023

rhaschke Sep 10, 2023 Maintainer

v4hn Sep 11, 2023 Maintainer Author

rhaschke Nov 6, 2023 Maintainer

furushchev Oct 31, 2024 Maintainer

mikaelarguedas Nov 9, 2024

rhaschke Nov 10, 2024 Maintainer

mikaelarguedas Nov 11, 2024

mikaelarguedas Nov 11, 2024

gavanderhoorn Nov 12, 2024

v4hn
Jan 23, 2023
Maintainer

Replies: 5 comments 15 replies

lucasw
Feb 18, 2023
Maintainer

v4hn Feb 20, 2023
Maintainer Author

lucasw Mar 7, 2023
Maintainer

v4hn Mar 8, 2023
Maintainer Author

lucasw
Mar 7, 2023
Maintainer

lucasw Mar 7, 2023
Maintainer

jspricke Mar 7, 2023
Maintainer

v4hn Mar 8, 2023
Maintainer Author

lucasw Mar 14, 2023
Maintainer

rhaschke
Sep 10, 2023
Maintainer

v4hn Sep 11, 2023
Maintainer Author

rhaschke Nov 6, 2023
Maintainer

furushchev
Oct 31, 2024
Maintainer

mikaelarguedas
Nov 9, 2024

rhaschke Nov 10, 2024
Maintainer