8000 feat: replace Crane usage for Image pull and Push with ORAS by AustinAbro321 · Pull Request #3559 · zarf-dev/zarf · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat: replace Crane usage for Image pull and Push with ORAS #3559

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 119 commits into from
Apr 1, 2025

Conversation

AustinAbro321
Copy link
Contributor
@AustinAbro321 AustinAbro321 commented Mar 7, 2025

Description

This replaces the Crane library uses for image pulls and pushes with ORAS. This does not change zarf tools crane operations.

Proposal - ZEP-0019: Refactor Image operations #20

This PR also enables the Zarf cache during E2E tests which helps verify that #3194 is truly getting solved.

Crane has special behavior for localhost. If localhost is used for image pulls or pushes it will try to connect over https, and fallback to http if that fails. This behavior was brought into Zarf since ORAS does not have it.

Release notes:
Image pulls and pushes now respect the --oci-concurrency flag to determine the number of layers to pull/push in parallel. In previous versions of Zarf every image layer was pulled and pushed concurrently and the --oci-concurrency flag only applied to zarf package publish and zarf package pull. Additionally, the default value of --oci-concurrency changed from three to six.

Zarf will no longer pull images in parallel, you may see a slowdown if you have a fast internet connection. We changed this because concurrent image pulls were causing reliability issues for some users. Generally image pulls are bottlenecked by the network so too many concurrent pulls can degrade performance.

The --plain-http flag is now respected when connected to registries for image pulls and pushes.

Docker v1 images are no longer supported. Dockerhub completely stopped serving v1 images in June 2019, it's unlikely you will be effected by this, but if you are using a particularly old registry you may want to double check.

The format of the cache is changing. You are encouraged to run zarf tools clear-cache as existing image layers will not do anything.

Related Issue

Fixes #3434
Fixes #3194
Relates to #3113
Relates to #2104

Checklist before merging

Copy link
netlify bot commented Mar 7, 2025

Deploy Preview for zarf-docs canceled.

Name Link
🔨 Latest commit 0eff3fe
🔍 Latest deploy log https://app.netlify.com/sites/zarf-docs/deploys/67ec01221b990300084dc282

Copy link
codecov bot commented Mar 7, 2025

Codecov Report

Attention: Patch coverage is 45.38088% with 337 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/internal/packager/images/pull.go 51.34% 109 Missing and 18 partials ⚠️
src/internal/packager/images/common.go 32.94% 53 Missing and 4 partials ⚠️
src/internal/packager/images/push.go 50.50% 35 Missing and 14 partials ⚠️
src/internal/packager/images/cache/cache.go 49.35% 31 Missing and 8 partials ⚠️
src/internal/packager2/mirror.go 0.00% 16 Missing ⚠️
src/pkg/packager/creator/normal.go 0.00% 10 Missing ⚠️
src/pkg/cluster/tunnel.go 0.00% 9 Missing ⚠️
src/pkg/utils/image.go 0.00% 9 Missing ⚠️
src/internal/packager2/layout/create.go 0.00% 7 Missing ⚠️
src/pkg/packager/deploy.go 0.00% 5 Missing ⚠️
... and 4 more
Files with missing lines Coverage Δ
src/cmd/viper.go 53.08% <100.00%> (ø)
src/internal/dns/dns.go 88.46% <100.00%> (+14.54%) ⬆️
src/internal/packager2/layout/layout.go 62.50% <ø> (ø)
src/internal/packager2/create.go 0.00% <0.00%> (ø)
src/internal/packager2/layout/package.go 38.37% <0.00%> (-2.14%) ⬇️
src/cmd/package.go 40.91% <0.00%> (-0.18%) ⬇️
src/internal/packager2/layout/sbom.go 51.02% <0.00%> (+0.62%) ⬆️
src/pkg/packager/deploy.go 5.41% <0.00%> (ø)
src/internal/packager2/layout/create.go 41.73% <0.00%> (+0.05%) ⬆️
src/pkg/cluster/tunnel.go 10.10% <0.00%> (-0.26%) ⬇️
... and 7 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@RothAndrew
Copy link
Contributor

let's gooooooo! so excited for this lol

AustinAbro321 and others added 27 commits March 12, 2025 16:33
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
…3550)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
git push

Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
mkcp
mkcp previously approved these changes Mar 29, 2025
Copy link
Contributor
@mkcp mkcp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice work! License matches and the included attribution should be sufficient.

Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
AustinAbro321 and others added 4 commits April 1, 2025 07:34
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Copy link
Contributor
@mkcp mkcp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, discussed final review synced up

@AustinAbro321 AustinAbro321 added this pull request to the merge queue Apr 1, 2025
Merged via the queue into main with commit a2dd6c1 Apr 1, 2025
38 checks passed
@AustinAbro321 AustinAbro321 deleted the move-image-operations-to-oras branch April 1, 2025 16:58
@Noxsios Noxsios mentioned this pull request Apr 28, 2025
2 tasks
nevinaragam pushed a commit to nevinaragam/zarf that referenced this pull request May 20, 2025
…#3559)

Signed-off-by: Austin Abro <AustinAbro321@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Austin Abro <37223396+AustinAbro321@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: razzle <razzle@defenseunicorns.com>
Signed-off-by: NevinAragam <nevin.aragam@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Migrate off of crane for pulling and pushing OCI images 3F73 image pull fail from corrupted Crane cache
6 participants
0