8000 Support CNAI OCI artifact models · Issue #9127 · cri-o/cri-o · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Support CNAI OCI artifact models #9127

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
chlins opened this issue Apr 11, 2025 · 7 comments · Fixed by #9131
Closed

Support CNAI OCI artifact models #9127

chlins opened this issue Apr 11, 2025 · 7 comments · Fixed by #9131
Assignees
@saschagrunert
Labels
kind/feature Categorizes issue or PR as related to a new feature. oci-artifacts

Comments

@chlins
Copy link
chlins commented Apr 11, 2025

What happened?

It seems that the CRI-O has supported mount OCI Artifact in the main branch, but I ran into the issue when mount the artifact which bundled the AI models into it. The mount action is success, but cannot list the accurate files in pod.

What did you expect to happen?

Mount the OCI Artifact into the pod and can list the accurate files in the pod mount path.

How can we reproduce it (as minimally and precisely as possible)?

apiVersion: v1
kind: Pod
metadata:
  name: image-volume
spec:
  containers:
  - name: shell
    command: ["sleep", "infinity"]
    image: debian
    volumeMounts:
    - name: volume
      mountPath: /tmp/model
  volumes:
  - name: volume
    image:
      reference: ghcr.io/chlins/qwen:v1
      pullPolicy: Always

Exec into the pod and ls the /tmp/model.

Anything else we need to know?

No response

CRI-O and Kubernetes version

$ crio --version
crio version 1.33.0
   GitCommit:      3e6b6cf8237166a6424a6d319c29a99c22f1e9ed
   GitCommitDate:  2025-03-28T10:58:57Z
   GitTreeState:   clean
   BuildDate:      2025-03-31T12:11:30Z
   GoVersion:      go1.24.1
   Compiler:       gc
   Platform:       linux/amd64
   Linkmode:       dynamic
   BuildTags:
     containers_image_ostree_stub
     seccomp
     selinux
   LDFlags:          unknown
   SeccompEnabled:   true
   AppArmorEnabled:  false
$ kubectl version --output=json
{
  "clientVersion": {
    "major": "1",
    "minor": "31",
    "gitVersion": "v1.31.0",
    "gitCommit": "9edcffcde5595e8a5b1a35f88c421764e575afce",
    "gitTreeState": "clean",
    "buildDate": "2024-08-13T07:37:34Z",
    "goVersion": "go1.22.5",
    "compiler": "gc",
    "platform": "linux/amd64"
  },
  "kustomizeVersion": "v5.4.2",
  "serverVersion": {
    "major": "1",
    "minor": "31",
    "gitVersion": "v1.31.1",
    "gitCommit": "948afe5ca072329a73c8e79ed5938717a5cb3d21",
    "gitTreeState": "clean",
    "buildDate": "2024-09-11T21:22:08Z",
    "goVersion": "go1.22.6",
    "compiler": "gc",
    "platform": "linux/amd64"
  }
}

OS version

# On Linux:
$ cat /etc/os-release
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
$ uname -a
Linux c3-small-x86-01 6.1.0-23-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.99-1 (2024-07-15) x86_64 GNU/Linux

Additional environment details (AWS, VirtualBox, physical, etc.)
@chlins chlins added the kind/bug Categorizes issue or PR as related to a bug. label Apr 11, 2025
@saschagrunert
Copy link
Member

@chlins do you mind making the image ghcr.io/chlins/qwen:v1 public?

@chlins
Copy link
Author
chlins commented Apr 14, 2025

@chlins do you mind making the image ghcr.io/chlins/qwen:v1 public?

Of course, it has already been set to public. I apologize for forgetting that the default is a private project.

@saschagrunert
Copy link
Member

Thanks. Looking at ghcr.io/chlins/qwen:v1, we'll see:

  • mediaType: application/vnd.oci.image.manifest.v1+json
  • config.mediaType: application/vnd.cnai.model.config.v1+json
  • artifactType: application/vnd.cnai.model.manifest.v1+json

Coming from: https://github.com/CloudNativeAI/model-spec/blob/a5d6440a33ac003df300902527e0c7e8fd08c27f/specs-go/v1/mediatype.go#L17-L26

I think the new handling as artifacts is fine, but we do not seem to support the untar operation yet.

This sounds more like an enhancement of the current implementation rather than a bug.

@saschagrunert saschagrunert added kind/feature Categorizes issue or PR as related to a new feature. and removed kind/bug Categorizes issue or PR as related to a bug. labels Apr 14, 2025
@saschagrunert saschagrunert changed the title Failed to mount OCI Artifact Support CNAI OCI artifact models Apr 14, 2025
@saschagrunert saschagrunert moved this to To do in Feature Roadmap Apr 14, 2025
@saschagrunert
Copy link
Member

@cri-o/cri-o-maintainers I assume it would be great for CRI-O to support CNAI artifact models, so that's the feature.

@chlins is the image ghcr.io/chlins/qwen:v1 following the final specification of CNAI or do we expect further changes in the future?

@chlins
Copy link
Author
chlins commented Apr 14, 2025

Thanks. Looking at ghcr.io/chlins/qwen:v1, we'll see:

  • mediaType: application/vnd.oci.image.manifest.v1+json
  • config.mediaType: application/vnd.cnai.model.config.v1+json
  • artifactType: application/vnd.cnai.model.manifest.v1+json

Coming from: https://github.com/CloudNativeAI/model-spec/blob/a5d6440a33ac003df300902527e0c7e8fd08c27f/specs-go/v1/mediatype.go#L17-L26

I think the new handling as artifacts is fine, but we do not seem to support the untar operation yet.

This sounds more like an enhancement of the current implementation rather than a bug.

@saschagrunert Thanks for your confirmation, I've tested another artifact which bundled with raw file instead of tar, but also not work as normal. The raw artifact is: ghcr.io/chlins/qwen:raw.

@chlins
Copy link
Author
chlins commented Apr 14, 2025

@cri-o/cri-o-maintainers I assume it would be great for CRI-O to support CNAI artifact models, so that's the feature.

@chlins is the image ghcr.io/chlins/qwen:v1 following the final specification of CNAI or do we expect further changes in the future?

Yes, this image was packaged by modctl which followed the CNAI model spec, Although more people and projects are still involved in discussing this spec, but I believe the basic version is relatively stable.

@saschagrunert
Copy link
Member

@chlins ah I think I'll have a fix for that!

@saschagrunert saschagrunert self-assigned this Apr 14, 2025
@saschagrunert saschagrunert moved this from To do to In Progress in Feature Roadmap Apr 14, 2025
This was referenced Apr 14, 2025
Merged
Open
@github-project-automation github-project-automation bot moved this from In Progress to Done in Feature Roadmap Apr 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@saschagrunert saschagrunert

Labels
kind/feature Categorizes issue or PR as related to a new feature. oci-artifacts
Projects
Feature Roadmap
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add support for CNAI models saschagrunert/cri-o
2 participants
@saschagrunert @chlins
0