8000 Add ARM64 arch support for serving runtimes by sivanantha321 · Pull Request #3038 · kserve/kserve · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Add ARM64 arch support for serving runtimes #3038

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 27, 2023
Merged

Add ARM64 arch support for serving runtimes #3038

merged 1 commit into from
Jul 27, 2023

Conversation

sivanantha321
Copy link
Member
@sivanantha321 sivanantha321 commented Jul 19, 2023 •
edited
Loading

What this PR does / why we need it:

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #2989

Type of changes
Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Feature/Issue validation/testing:

Please describe the tests that you ran to verify your changes and relevent result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

  • Test A

  • Test B

  • Logs

Special notes for your reviewer:

  1. Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Checklist:

  • Have you added unit/e2e tests that prove your fix is effective or that this feature works?
  • Has code been commented, particularly in hard-to-understand areas?
  • Have you made corresponding changes to the documentation?

Release note:

Added ARM64 arch support for serving runtimes

@sivanantha321 sivanantha321 force-pushed the architecture-support branch from 2e04ec5 to 3cd4ee4 Compare July 20, 2023 04:57
@rhuss rhuss mentioned this pull request Jul 21, 2023
Open
7 tasks
@rhuss
Copy link
Contributor
rhuss commented Jul 21, 2023

Thanks, the update looks straightforward to me. FWIW, I stumbled upon the same issue when cross-compiling (however, I "just" installed gcc which seemed good enough to compile the native parts in the dependencies, but I think the solution here is cleaner and more comprehensive even when it might add a bit more to the build times).

I wonder whether it would make sense to combine this with #3042 (if this makes sense to add), so that only one version bump would be required?

ckadner
ckadner approved these changes Jul 25, 2023
View reviewed changes
Copy link
Member
@ckadner ckadner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @sivanantha321 -- this is very helpful! The changes look good, but there is a lot of duplicated code in the workflow YAML files and the Dockerfiles.

For the GitHub actions/workflows, maybe we could Reusable Workflows or Composite Actions to wrap the boilerplate and just provide the Dockerfile and image name as parameters/inputs.

https://stackoverflow.com/a/71570847/5601796

Similarly, for the Dockerfiles, maybe create a(nother) common base image that contains all the common layers?

But since this PR is good as it is, those changes could go into a separate PR later. WDYT?

@sivanantha321
Copy link
Member Author

Thanks for the suggestion @ckadner. I agree with you. It would be nice to remove the duplicates. Let's handle it in a separate PR.

@sivanantha321
Copy link
Member Author

@rhuss Thanks for the suggestion. But I think it is better to keep it as a different PR since, both are two different features.

@ckadner ckadner mentioned this pull request Jul 26, 2023
Open
ckadner
ckadner approved these changes Jul 26, 2023
View reviewed changes
Copy link
Member
@ckadner ckadner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Need to resolve merge conflicts still, though.

@sivanantha321
Add ARM64 arch support for serving runtimes
f71f786 
Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>
@sivanantha321 sivanantha321 force-pushed the architecture-support branch from 3cd4ee4 to f71f786 Compare July 27, 2023 11:06
@sivanantha321
Copy link
Member Author
sivanantha321 commented Jul 27, 2023 •
edited
Loading

@yuzisun @ckadner Currently, Paddle does not have support for ARM64.

@ckadner
Copy link
Member
ckadner commented Jul 27, 2023

@yuzisun @ckadner Currently, Paddle does not have support for ARM64.

I think that's okay. We could keep track of that in a separate issue.

ckadner
ckadner approved these changes Jul 27, 2023
View reviewed changes
Copy link
Member
@ckadner ckadner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@kserve-oss-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: ckadner, sivanantha321

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ckadner ckadne 8000 r merged commit 32be746 into kserve:master Jul 27, 2023
@ckadner ckadner mentioned this pull request Jul 27, 2023
Closed
rhuss pushed a commit to rhuss/kserve that referenced this pull request Jul 28, 2023
@sivanantha321 @rhuss
Add ARM64 arch support for serving runtimes (kserve#3038)
d0d1d42 
Except for Paddle, which does not support arm64/aarch64

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>
greenmoon55 pushed a commit to greenmoon55/kserve that referenced this pull request Aug 2, 2023
@sivanantha321 @greenmoon55
Add ARM64 arch support for serving runtimes (kserve#3038)
3998764 
Except for Paddle, which does not support arm64/aarch64

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>
Signed-off-by: Jin Dong <jdong183@bloomberg.net>
yuzisun added a commit that referenced this pull request Aug 7, 2023
@yuzisun @sivanantha321
@ddelange @ckadner @rafvasq @tessapham @andyi2it
Cherry-pick: Release 0.11 branch (#3067)
1529b71 
* Fix configmap docs removed by kustomize build (#3024)

Signed-off-b
8000
y: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>

* Loosen ray version range (#2993)

* Loosen ray range

Signed-off-by: ddelange <14880945+ddelange@users.noreply.github.com>

* Fix import of RayServeHandle

ref ray-project/ray#34714

Signed-off-by: ddelange <14880945+ddelange@users.noreply.github.com>

* Allow pandas 2.0

Signed-off-by: ddelange <14880945+ddelange@users.noreply.github.com>

* Remove exclusion markers

Signed-off-by: ddelange <14880945+ddelange@users.noreply.github.com>

* Run poetry lock --no-update

Signed-off-by: ddelange <14880945+ddelange@users.noreply.github.com>

---------

Signed-off-by: ddelange <14880945+ddelange@users.noreply.github.com>

* Retry on resource conflict error (#3035)

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>

* chore: Fix `no space left on device` error in `predictor-runtime-build` (#3049)

* Create a separate action to free-up disk space on GH action runner
* Move 'free-up-disk-space' step outside 'minikube-setup' action
* Free-up disk space before running 'predictor-runtime-build'

Signed-off-by: Christian Kadner <ckadner@us.ibm.com>

* Update MM version to v0.11.0-rc1 (#3047)

Signed-off-by: Rafael Vasquez <raf.vasquez@ibm.com>

* Bump scikit-learn version to 1.3.0 (#3040)

* Bump scikit-learn version to 1.3.0

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>

* Update test models

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>

* Downgrade ray serve to 2.4

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>

---------

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>

* template out kube-rbac-proxy image (#3048)

* template out kube-rbac-proxy image

Signed-off-by: Tessa Pham <hpham111@bloomberg.net>

* add kserve-addressable-resolver to clusterroles

Signed-off-by: Tessa Pham <hpham111@bloomberg.net>

---------

Signed-off-by: Tessa Pham <hpham111@bloomberg.net>

* Add ARM64 arch support for serving runtimes (#3038)

Except for Paddle, which does not support arm64/aarch64

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>

* Support secret name overrides with annotation (#3034)

* Add annotation for secret name overrides

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Remove the field

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Fix secret name annotation

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Update kserve url storage example docs

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Address review comments

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Update config docs for storage secret

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Fix storage secret name

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Add test for secret name annotation override

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Fix flake8 lint

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

---------

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* release: Update defaultModelMeshVersion to v0.11.0 (#3059)

Signed-off-by: Christian Kadner <ckadner@us.ibm.com>

* chore: Rename ModelMesh validating webhook (#3064)

Resolves #3063

Signed-off-by: Christian Kadner <ckadner@us.ibm.com>

* Release v0.11.0 (#3061)

Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com>

* Update poetry lock files (#3066)

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

---------

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>
Signed-off-by: ddelange <14880945+ddelange@users.noreply.github.com>
Signed-off-by: Christian Kadner <ckadner@us.ibm.com>
Signed-off-by: Rafael Vasquez <raf.vasquez@ibm.com>
Signed-off-by: Tessa Pham <hpham111@bloomberg.net>
Signed-off-by: Dan Sun <dsun20@bloomberg.net>
Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com>
Co-authored-by: Sivanantham <90966311+sivanantha321@users.noreply.github.com>
Co-authored-by: ddelange <14880945+ddelange@users.noreply.github.com>
Co-authored-by: Christian Kadner <ckadner@us.ibm.com>
Co-authored-by: Rafael Vasquez <raf.vasquez@ibm.com>
Co-authored-by: Tessa Pham <hpham111@bloomberg.net>
Co-authored-by: Andrews Arokiam <87992092+andyi2it@users.noreply.github.com>
greenmoon55 pushed a commit to greenmoon55/kserve that referenced this pull request Aug 10, 2023
@sivanantha321 @greenmoon55
Add ARM64 arch support for serving runtimes (kserve#3038)
1c20e58 
Except for Paddle, which does not support arm64/aarch64

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>
Signed-off-by: Jin Dong <jdong183@bloomberg.net>
Iamlovingit pushed a commit to Iamlovingit/kserve that referenced this pull request Oct 1, 2023
@sivanantha321 @Iamlovingit
Add ARM64 arch support for serving runtimes (kserve#3038)
a1df359 
Except for Paddle, which does not support arm64/aarch64

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>
Signed-off-by: iamlovingit <freecode666@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ckadner ckadner ckadner approved these changes

@alexagriffith alexagriffith Awaiting requested review from alexagriffith

@sukumargaonkar sukumargaonkar Awaiting requested review from sukumargaonkar

Assignees

@ckadner ckadner

Labels
approved lgtm
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Unable to fetch image "docker.io/kserve/sklearnserver:v0.10.0" in Minikube/Kind version: v1.30.1 on Darwin 13.4 (arm64)
4 participants
@sivanantha321 @rhuss @ckadner @kserve-oss-bot
0