8000 migrate from container registry to artifact registry · Issue #4626 · ossf/scorecard · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

migrate from container registry to artifact registry #4626

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
spencerschrock opened this issue Apr 11, 2025 · 6 comments
Closed

migrate from container registry to artifact registry #4626

spencerschrock opened this issue Apr 11, 2025 · 6 comments

Comments

@spencerschrock
Copy link
Member

https://cloud.google.com/container-registry/docs/deprecations/container-registry-deprecation

The shutdown is scheduled as follows:

Effective March 18, 2025, writing images to Container Registry is unavailable.
After May 20, 2025, reading images from Container Registry is unavailable. Existing images in Container Registry are inaccessible.
Effective July 17, 2025, if you have completed the steps to set up gcr.io repositories in Artifact Registry, then pushing to gcr.io endpoints stores images in Artifact Registry and all gcr.io URLs are served from Artifact Registry. Any images in Container Registry that haven't been copied to Artifact Registry are unavailable.

@spencerschrock
Copy link
Member Author

Despite saying writes are unavailable, it actually seems to be working intermittently. So this needs to be fixed soon, but the deploy to deal with ossf/scorecard-webapp#774 seems to have worked for now.

@spencerschrock spencerschrock transferred this issue from ossf/scorecard-webapp May 14, 2025
@spencerschrock
Copy link
Member Author
spencerschrock commented May 14, 2025

One option of this migration is --recent-images

If you only want to copy images pulled from Container Registry in the last 30 to 180 days, you can include the --recent-images=DAYS flag. Replace DAYS with the number of days, between 30 and 180, that the tool should check for pulls within.

Currently we generate ~8 docker images on every commit push, which over 4 years is a lot. We don't need that majority of these images anymore.

Consumer facing images:

  • gcr.io/openssf/scorecard
  • gcr.io/openssf/scorecard-action
  • gcr.io/openssf/scorecard-attestor (maybe?)

Our own infrastructure (no need to necessarily be public)

  • gcr.io/openssf/scorecard-batch-controller
  • gcr.io/openssf/scorecard-batch-worker
  • gcr.io/openssf/scorecard-bq-transfer
  • gcr.io/openssf/scorecard-cii-worker
  • gcr.io/openssf/scorecard-github-server
  • gcr.io/openssf/scorecard-webhook-releasetest

However, @justaugustus mentioned we should save any semantically tagged image (which should include gcr.io/openssf/scorecard and gcr.io/openssf/scorecard-action, which we can enforce by writing a script to pull all of them manually so they have recent pulls.

@spencerschrock
Copy link
Member Author

In terms of avoiding this bloat going forward:

Moving the Scorecard cron infra to GHCR would require more thought, since we need to be able to modify which images are tagged as stable

@spencerschrock
Copy link
Member Author

Here are all the currently tagged Scorecard images

gcloud container images list-tags gcr.io/openssf/scorecard --format=json > tags.json
jq ".[].tags | select(length > 0)" tags.json | grep -vE "[a-f0-9]{40}" | grep "\""

gcr.io/openssf/scorecard

  • latest
  • stable
  • v5.1.1
  • v5.1.0
  • v5.0.0
  • v5.0.0-rc2
  • v5.0.0-rc1
  • v4.13.1
  • v4.13.0
  • v4.12.0
  • v4.11.0
  • v4.10.5
  • v4.10.4
  • v4.10.3
  • v4.10.2
  • v4.10.1
  • v4.9.1
  • v4.10.0
  • v4.9.0
  • v4.8.0
  • v4.7.0
  • v4.6.0
  • v4.5.0
  • v4.4.0
  • v4.3.1
  • v4.3.0
  • v4.2.0
  • v4.1.0
  • v4.0.1
  • v4.0.0
  • v3.2.1
  • v3.1.1
  • v3.1.0
  • v3.0.1
  • v3.0.0
  • v2.2.8
  • v2.2.7
  • v2.2.6
  • v2.2.5
  • v2.2.4
  • v2.2.3
  • v2.1.3
  • v2.1.2
  • v2.1.1
  • tag1,
  • tag2
  • v2.1.0
  • v2.0.0
  • v1.0.0
  • v1

gcr.io/openssf/scorecard-action

  • latest
  • v2.4.1
  • v2.4.0
  • v2.3.3
  • v2.3.2
  • v2.3.1
  • v2.3.0
  • v2.2.0
  • v2.1.3
  • v2.1.2
  • v2.1.1
  • v2.1.0
  • test
  • v2.0.6
  • v2.0.4
  • v2.0.3
  • v2.0.2
  • v2.0.1
  • v2.0.0
  • v2.0.0-alpha.2
  • v2.0.0-alpha.1
  • v2.0.0-beta.1

gcr.io/openssf/scorecard-attestor

  • latest

@spencerschrock
Copy link
Member Author

Manually pulled all of these tags with a simple bash script (shown for scorecard, but done for all 3):

versions=(
    latest
    stable
    v5.1.1
    v5.1.0
    v5.0.0
    v5.0.0-rc2
    v5.0.0-rc1
    v4.13.1
    v4.13.0
    v4.12.0
    v4.11.0
    v4.10.5
    v4.10.4
    v4.10.3
    v4.10.2
    v4.10.1
    v4.9.1
    v4.10.0
    v4.9.0
    v4.8.0
    v4.7.0
    v4.6.0
    v4.5.0
    v4.4.0
    v4.3.1
    v4.3.0
    v4.2.0
    v4.1.0
    v4.0.1
    v4.0.0
    v3.2.1
    v3.1.1
    v3.1.0
    v3.0.1
    v3.0.0
    v2.2.8
    v2.2.7
    v2.2.6
    v2.2.5
    v2.2.4
    v2.2.3
    v2.1.3
    v2.1.2
    v2.1.1
    tag1
    tag2
    v2.1.0
    v2.0.0
    v1.0.0
    v1
)

for i in "${versions[@]}"; do
    docker pull gcr.io/openssf/scorecard-action:$i
done

@spencerschrock
Copy link
Member Author

A quick estimation is roughly 18,000 images, at roughly 20MB per image. or 350 GB, which under Artifact Registry pricing would be $35/month.

Bringing over all images for now may be simplest.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

No branches or pull requests

1 participant
0