Modify internal base image struct #1712
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
To improve model startup performance, we are providing a set of pre-built image based based on a combination of python, pytorch and CUDA versions. The current version stores all the versions as a flat list and it makes a bit hard to reason about combinations and provide user with feedback on what versions are available. The change here makes Python the top level key and store all the CUDA and PyTorch versions availab 8000 le for each Python version. It would be possible to normalize the struct more to represent the relationship between CUDA, Python but PyTorch but I don't know if that's worth doing at this point. The work here is WIP but sharing it for feedback. Signed-off-by: Marcus Martins <marcus@replicate.com>
Signed-off-by: Marcus Martins <marcus@replicate.com>
Signed-off-by: Marcus Martins <marcus@replicate.com>
Signed-off-by: Marcus Martins <marcus@replicate.com>
dd19205 to
53d2724
Compare
jessie-young
reviewed
Jun 4, 2024
* Base images are now derived from the Torch compatibility matrix * Only include versions greater or equal to minimums * Patch versions are stripped Signed-off-by: andreasjansson <andreas@replicate.ai>
05a4506 to
22d179a
Compare
mattt
reviewed
Jun 18, 2024
pkg/cli/baseimage.go
Outdated
Comment on lines
53
to
70
| func newVersionsMatrix() *cobra.Command { | ||
| var cmd = &cobra.Command{ | ||
| Use: "generate-versions", | ||
| Short: "Generate a nested matrix of python, cuda, pytorch versions (json)", | ||
| RunE: func(cmd *cobra.Command, args []string) error { | ||
| versions := dockerfile.BaseImageConfigurations() | ||
| output, err := json.Marshal(versions) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| fmt.Println(string(output)) | ||
| return nil | ||
| }, | ||
| Args: cobra.MaximumNArgs(0), | ||
| } | ||
| return cmd | ||
| } | ||
|
|
There was a problem hiding this comment.
What's the difference between this new command and the existing generate-matrix command implemented below?
There was a problem hiding this comment.
As far as I can tell, generate-versions was an unused duplicate, so I removed it with 6d6d54c
Signed-off-by: Mattt Zmuda <mattt@replicate.com>
Signed-off-by: Mattt <mattt@replicate.com>
mattt
approved these changes
Jun 18, 2024
Signed-off-by: Mattt Zmuda <mattt@replicate.com>
Signed-off-by: andreasjansson <andreas@replicate.ai>
jessie-young
pushed a commit
that referenced
this pull request
Jun 21, 2024
* Modify base image struct To improve model startup performance, we are providing a set of pre-built image based based on a combination of python, pytorch and CUDA versions. The current version stores all the versions as a flat list and it makes a bit hard to reason about combinations and provide user with feedback on what versions are available. The change here makes Python the top level key and store all the CUDA and PyTorch versions available for each Python version. It would be possible to normalize the struct more to represent the relationship between CUDA, Python but PyTorch but I don't know if that's worth doing at this point. The work here is WIP but sharing it for feedback. Signed-off-by: Marcus Martins <marcus@replicate.com> * Fix ordering Signed-off-by: Marcus Martins <marcus@replicate.com> * Fix the matrix-generation command Signed-off-by: Marcus Martins <marcus@replicate.com> * Generate versions matrix Signed-off-by: Marcus Martins <marcus@replicate.com> * Smaller set of base image combinations with aggressive minimums * Base images are now derived from the Torch compatibility matrix * Only include versions greater or equal to minimums * Patch versions are stripped Signed-off-by: andreasjansson <andreas@replicate.ai> * Remove duplicate / unused generate-versions function Signed-off-by: Mattt Zmuda <mattt@replicate.com> * Update compatibility matrix Signed-off-by: Mattt Zmuda <mattt@replicate.com> * Support non-Torch and non-Torch images Signed-off-by: andreasjansson <andreas@replicate.ai> --------- Signed-off-by: Marcus Martins <marcus@replicate.com> Signed-off-by: andreasjansson <andreas@replicate.ai> Signed-off-by: Mattt Zmuda <mattt@replicate.com> Signed-off-by: Mattt <mattt@replicate.com> Co-authored-by: andreasjansson <andreas@replicate.ai> Co-authored-by: Mattt Zmuda <mattt@replicate.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Modify the internal structure on how we store our base image compatibility matrix.