-
-
Notifications
You must be signed in to change notification settings - Fork 355
Migrate Dockerfiles from external repository #7303
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Two questions that haven't been addressed in the previous PRs/discussions:
|
this is used in used for two things: - selecting the base contrib image version - selecting the branch from the OpenMS repository
So I've made good progress on most of the items outlined so far but the Singularity conversion isn't as simple as I was hoping. Though Singularity also has this concept, and |
… update workflow accordingly
Update on the singularity issue mentioned above: instead of converting the I am not 100% sure if this is equivalent - this needs some testing. If this does not work then I see two choices:
|
There is no way to have build args or environment vars for singularity. And yes apparently no way to run a single stage. I'm a bit hesitant of converting the ready images. It was a bit of the reason why we build them here. It sometimes failed. Not sure if it was because of on-the-fly conversions at the user and their environments but could also be due to incompatibilitied in the docker layers. I'm not sure.. might work. We could try.. An alternative could be to have a "from-scratch" singularity def file template that can be configured with jinja templates or something (for both args and stages). In this case, we should move the build steps to small scripts that will be mounted and executed, such that we do not have to maintain both files. |
Thank you for the context - I'll see if I can put something together, and if it doesn't work out maybe we can scale back the optimization ambitions and just do a direct migration, or you/others can take a stab at it. Also, I should mention that I don't have much experience with singularity and don't understand deeply enough why converting an already built docker image might be flaky (and what the pitfalls are..), so I trust your experience of course. I did try and convert the current images though, and can verify that it does work or at least, I encounter no errors when doing the following: docker run --rm -v $PWD:/go quay.io/singularity/singularity:v3.11.5 build executables.sif docker://ghcr.io/openms/openms-executables
docker run --rm -v $PWD:/go -it --privileged quay.io/singularity/singularity:v3.11.5 run executables.sif IsobaricAnalyzer --help (yes, I'm running singularity in docker lol) Is there a chance that newer versions of singularity are better in this regard? It does seem that docker/OCI compatibility has been a priority for them. |
I guess we can try with the converted containers and come up with a solution if they fail. Apparently, according to your external issue, they are pretty confident about the conversion process. |
I tried to learn a bit more about how it works, I think the layer flattening should be pretty robust. There's also an open issue with a kinda old PR for building directly SIF from Dockerfiles. Basically I think if we have issues it'd also be worth raising them with the Singularity project. I would like to test the conversion process in a GitHub action though, in my fork. One thing that I can think of that would lead to sporadic failures would be worker diskspace.. I've hit this a few times when building larger containers and had to either slim them down or spread them across different jobs. EDIT: The conversion/build works, and takes ~7 minutes: https://github.com/radusuciu/OpenMS/actions/runs/7587340359/job/20667464502 |
NOTE: in github actions this is set to 2
Yep I think the disk space and permissions on clusters is what made us build them here for other people so they don't have to do it on the fly. I guess if we can convert in a controlled environment, such as here it should really be fine. I'm very happy to merge. Happy to see that it works out on your fork as well. |
Alright, cool! I'm now focusing on seeing if there's any fat to trim from the images. For example, the main repository .git and THIRDPARTY submodules (note: everything needed for the container is already copied to
That's out of a total of 5.3GB so that's a pretty big chunk:
|
Before PR is ready for review, I had a few questions for you (or any other maintainers):
|
hmm I think this is not easily done because installing tests is a bit non-standard for cmake projects. What others have done: they take one or few of the tests (preferably of a stable tool, maybe OpenMSInfo) and execute it in the container to check if loading of libraries etc. works. Edit: fixed sentence |
You can just copy the test executables and run them. Some of them will have hard coded paths to compare outputs to expected outputs. Those files must be copied over to the same path. Not sure if those were relative paths. Other tests will work out of the box. You can also copy over all ctest related files and with Cmake 3.20 run I wouldn't say our tests are non-standard at all. |
Thanks for the responses, I think I've got it! I've reduced the tools/executables runtime to below 1GB.
The relevant stage: FROM ubuntu:22.04 AS tools-slim
ARG OPENMS_LIBRARY_DIR
ARG OPENMS_LIBRARY_BUILD_DIR
ARG CONTRIB_BUILD_DIR
ARG OPENMS_LIBRARY_DIR
RUN apt-get update \
&& apt-get install -y --no-install-recommends --no-install-suggests \
libqt5opengl5 \
libsvm3 \
libzip4 \
zlib1g \
libbz2-1.0 \
libgomp1 \
libqt5svg5 \
libxerces-c3.2 \
libboost-date-time1.74-dev \
libboost-iostreams1.74-dev \
libboost-regex1.74-dev \
libboost-math1.74-dev \
libboost-random1.74-dev \
&& rm -rf /var/lib/apt/lists/*
COPY --from=tools ${CONTRIB_BUILD_DIR}/bin ${CONTRIB_BUILD_DIR}/bin
COPY --from=tools /thirdparty /thirdparty
COPY --from=tools ${OPENMS_LIBRARY_BUILD_DIR}/bin ${OPENMS_LIBRARY_BUILD_DIR}/bin
COPY --from=tools ${OPENMS_LIBRARY_BUILD_DIR}/lib ${OPENMS_LIBRARY_BUILD_DIR}/lib
COPY --from=tools ${OPENMS_LIBRARY_DIR}/share ${OPENMS_LIBRARY_DIR}/share What's missing is the root@US320024WSD:/openms-build# ctest -R TOPP_
[snipped stdout]
100% tests passed, 0 tests failed out of 1278 😃 I'll update PR today/tomorrow, but I think this is very promising and IMO a worthwhile optimization if it holds. |
Whaat? That's awesome! |
a few other modifications: - added dockerignore file - clone thirdparty repo instead of using submodule since I've added .git to the dockergignore
Okay, so the slimmed tools image is at 1.3GB, I accidently the
Notes on the above:
All binaries/files are in the original build folders still, I'll see if I can tidy that up, but that's almost an aesthetic concern. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added some quick comments
Regarding runtime binaries. |
it is sufficient for caching purposes to load it
runners have varying amounts of free space, and these images can be quite big and sometimes juuuust fail to build if the runner happens to have less free space.
Thanks for keeping the PR up to date. It's pretty much ready for merge but I would recommend a final review. I've been using a similar image that I build here https://github.com/radusuciu/docker-openms/ without problems. There are two things I'd call your attention to:
|
looks very good to me in general. |
Hey, sorry for delay, I just got back from a trip. This was mainly for easy testing prior to OpenMS/contrib#136 being merged. It might still be helpful to have as a stage in the same Dockerfile as opposed to a separate Dockerfile in the sub-module when how changes to the contrib packgages affect builds of the images here.. but this also introduces the risk of some confusion for future maintenance. I don't feel super-strongly, but I'm inclined to say let's remove it. |
@jpfeuffer what is your call here? |
If you want to avoid contrib containers you can also build remaining dependencies in the corresponding action (CI, pyopenms, this docker action) |
Ah I misremembered the end of the thread here. I think you can remove. (For which we do not need any contrib base containers anymore) |
I've moved the unused contrib stage from the Thanks! |
Is this ready to merge? I'm trying to set up devcontainer image that we can use for development and came across this (which is going to be used as a basis for my setup) |
Quoting myself
This is minor, and well covered in the README I included in the containing directory: https://github.com/OpenMS/OpenMS/blob/54a90800c60e9ffd9a1ec250671480277a25bb74/dockerfiles/README.md
I just tagged a fake release, let's see if it runs https://github.com/radusuciu/OpenMS/actions/runs/10724213483 |
Not necessarily related to the PR but looks like we failed two tests
and
|
So the error is due to a change in the command line params to ThermoRawFileParser we just need to make sure to that the test cmake file matches the correct version of the executable. |
@poshul what changed?
Deployment fails currently |
fixed in #7673 |
Description
This PR migrates the Dockerfiles currently housed in the OpenMS/dockerfiles as initially outlined in OpenMS/dockerfiles#28. At the same time I'd like where possible optimize the images and related workflows to achieve minimal image size and build times
Some notes, outstanding questions, and issues. This is basically what I want to finish before this is ready for review/testing (though you're welcome to test and note issues ahead of time).
OPENMS_BRANCH
andOPENMS_TAG
pyopenms target isn't building due to missingpython-dev
package. I copied this straight from the original dockerfile so I assume that that must've been broken as wellLABEL
for each stage.dockerignore
Checklist