8000 Supporting changes for new docker client by michaeldwan · Pull Request #2344 · replicate/cog · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Supporting changes for new docker client #2344

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
May 19, 2025
Merged

Conversation

michaeldwan
Copy link
Member

These are changes to the existing codebase extracted from #2327 to make the diff easier to review.

No behavior changes expected. Each commit is roughly one logical change, though the diff might be strange because I did a half ass job of splitting them up to save time. The end result is the same though.

The main things:
Several functions on command.Command didn't actually change between the cli and sdk clients. So I extracted them out to helper functions that operate on a command.Command.

  • docker credential store access has been moved to standalone functions that both clients call
  • CreateTarFile & CreateAptTarFile functions also don't change between clients, so they were extracted out to helpers that operate on a command.Command. I also moved related code to monobase.go file to make it clear what that stuff is for

A few other things:

  • added an integration test for build secrets
  • added helpers to test for known errors generated by different clients and backends. Eg docker cli<>dockerd and docker api<>containerd produce slightly different errors for the same operation
  • added many more docker client integration tests and helpers

Once this is merged I can tidy up #2327 for review 🎉

the credential stuff isn't tied to the docker client. This moves logic to helper functions that both clients can use
The monobase helpers from command.Command were calling docker run, instead have them call the run functions on the client since the logic doesn't depend on client implementation
helpers to map errors from different backends to known types. needed to support both clients
@michaeldwan michaeldwan requested review from a team and Copilot May 16, 2025 23:14
Copy link
Contributor
@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Adds shared helpers and refactors the Docker client implementation and tests to support a new unified client interface, plus an integration test for build secrets.

  • Extracted common tar-creation and credential logic into pkg/docker/monobase.go and pkg/docker/credentials.go
  • Enhanced the registry test helper (StartTestRegistry) with auth options, repo cloning, and image-exists checks
  • Updated unit and integration tests (including a new secrets fixture) to use the new helper functions

Reviewed Changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated no comments.

Show a summary per file
File Description
test-integration/.../predict.py Added sample Predictor for secrets integration tests
test-integration/.../file-secret.txt Added a file-based secret value
test-integration/.../cog.yaml Defined mounts and assertions for file and env secrets
pkg/registry_testhelpers/registry_container.go Made StartTestRegistry configurable, added auth, clone, exists helpers
pkg/docker/monobase.go Extracted CreateTarFile/CreateAptTarFile implementations
pkg/docker/credentials.go Factored out credential loading into standalone functions
pkg/docker/docker_command.go Updated Push and LoadUserInformation to use new helpers
pkg/docker/command/command.go Removed tar helpers from the Command interface
pkg/docker/dockertest/* Refactored test helpers and mocks to drive new client tests
pkg/docker/docker_client_test.go Rewrote client tests to leverage new registry and helper APIs
go.mod Added golang.org/x/crypto dependency
Comments suppressed due to low confidence (5)

test-integration/test_integration/fixtures/secrets-project/cog.yaml:14

  • The second secret mount uses id: env-secret but no corresponding secret source or environment variable is defined in the fixtures; this test will fail at runtime. Consider adding a file for env-secret or specifying an env var to back it.
-        - type: secret

pkg/registry_testhelpers/registry_container.go:142

  • [nitpick] The WithAuth function returns an anonymous func(*options) but the type alias is named Option. Consider changing the signature to func WithAuth(...) Option for clarity.
func WithAuth(username, password string) func(*options) {

pkg/docker/docker_command.go:9

  • After removing the old CreateTarFile and CreateAptTarFile methods, there may be unused imports in this file. Run go mod tidy / goimports to clean them up.
import (

pkg/docker/command/command.go:15

  • The interface no longer declares CreateTarFile or CreateAptTarFile, but implementations have been moved out. Consider removing any dead methods on DockerCommand or elsewhere to keep the interface and implementations in sync.
type Command interface {

pkg/docker/monobase.go:22

  • This function returns only the tar filename, while CreateTarFile returns the full path. It may be more consistent to return the absolute path here as well.
func CreateAptTarball(ctx context.Context, tmpDir string, dockerClient command.Command, packages ...string) (string, error) {

@michaeldwan michaeldwan merged commit b30b0de into main May 19, 2025
21 checks passed
@michaeldwan michaeldwan deleted the md/docker-sdk-client-prep branch May 19, 2025 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
0