8000 feat: support serving by kemingy · Pull Request #1228 · tensorchord/envd · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat: support serving #1228

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 23 commits into from
Dec 7, 2022
Merged

feat: support serving #1228

merged 23 commits into from
Dec 7, 2022

Conversation

kemingy
Copy link
Member
@kemingy kemingy commented Nov 25, 2022 •
edited
Loading

Breaking changes:

  • base(image, dev)
  • install.conda()
  • install.python()
  • install.r_lang() not implemented yet
  • install.julia() not implemented yet
  • config.jupyter() move to envdlib (TBD)
  • config.rstudio_server() move to envdlib (TBD)

@kemingy
init base and install
0da9ec6 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
wip
3b351c7 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
fix build
ea937cd 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
work for dev
c8a27c3 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
Copy link
Member Author
kemingy commented Nov 28, 2022 •
edited
Loading
  • conda is required if Python needs to be installed
    • I checked pyenv but it requires a dev env to compile from source
  • CUDA only works if the image + CUDA + CUDNN can be found in docker hub
  • it seems serving env also needs some system packages like wget to install conda, although this can be done by LLB, WDYT
  • I plan to move jupyter and rstudio to envdlib, WDYT

cc @gaocegege @VoVAllen It's not 100% finished, but I think you can take a look

@kemingy
Copy link
Member Author
kemingy commented Nov 28, 2022

To test this PR, you can try:

def build():
    dev_tools()
    install.conda()
    install.python()
    install.python_packages(name=["ipython"])
    install.apt_packages(name=["ripgrep"])

@kemingy kemingy linked an issue Nov 28, 2022 that may be closed by this pull request
feat(lang): v2 language new design #157
Closed
@gaocegege
Copy link
Member

I think we can keep jupyter since it is used frequently. Maybe we could have a new package editor?

@gaocegege
Copy link
Member

We should schedule v0.3.0 for the PR. Since it is a breaking change to user interface. WDYT

gaocegege
gaocegege reviewed Nov 28, 2022
View reviewed changes
@kemingy
Copy link
Member Author
kemingy commented Nov 28, 2022

We should schedule v0.3.0 for the PR. Since it is a breaking change to user interface. WDYT

Yep.

I think we can keep jupyter since it is used frequently. Maybe we could have a new package editor?

  • people should prefer jupyter-lab, if you visit https://jupyter.org/, it shows "JupyterLab: A Next-Generation Notebook Interface"
  • I think they are still daemon processes, we don't need to keep too many specific codes

@VoVAllen
Copy link
Member

I think we can move jupyter to envdlib. If it's not convenient enough for user, we can embed envdlib and include it automatically. Also what's our plan if install.python is called without install.conda?

@kemingy
Copy link
Member Author
kemingy commented Nov 28, 2022

Also what's our plan if install.python is called without install.conda?

For now, it will still install conda. I didn't find a "portable Python". pyenv will download the source code and compile it. Will check miniconda source code later.

@gaocegege
Copy link
Member

I think we can move jupyter to envdlib. If it's not convenient enough for user, we can embed envdlib and include it automatically.

If we embed the envdlib, there is no difference between envdlib and install

@kemingy kemingy mentioned this pull request Nov 29, 2022
Closed
@kemingy
change cuda to a image+cuda+cudnn match
73e7ade 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
install python without conda
d843ff2 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
add cert
0c62b68 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
Copy link
Member Author
kemingy commented Nov 30, 2022
  • Python without conda can be done by
    • copy micromamba
    • create a Python env
    • rm micromamba
  • Shall we install "curl ca-certificates bzip2" for serving? I feel copy the /etc/ssl/certs/ca-certificates.crt is not elegant.
  • Not sure which user should we put in ImageConfig.User
  • Need to remove the hard-code sshd label

@gaocegege
Copy link
Member

Shall we install "curl ca-certificates bzip2" for serving? I feel copy the /etc/ssl/certs/ca-certificates.crt is not elegant.

I think it should be handled by users. And we do not need to copy certs to it.

@gaocegege
Copy link
Member

Not sure which user should we put in ImageConfig.User

Could we not care about the user? It only be added with dev_tools

@kemingy
Copy link
Member Author
kemingy commented Nov 30, 2022

Shall we install "curl ca-certificates bzip2" for serving? I feel copy the /etc/ssl/certs/ca-certificates.crt is not elegant.

I think it should be handled by users. And we do not need to copy certs to it.

No, we have to. Without the certs, mamba cannot download the required libraries to create a Python env. And you can get the certs if you install ca-certificates.

Refer to: https://github.com/mamba-org/micromamba-docker/blob/903975a7cc10f20d8cab684e5a4f9df6485279b4/Dockerfile#L27

Not sure which user should we put in ImageConfig.User

Could we not care about the user? It only be added with dev_tools

Now it hard codes as envd. I will try to find a way to access the DefaultGraph there.

@kemingy
Copy link
Member Author
kemingy commented Nov 30, 2022

Shall we install "curl ca-certificates bzip2" for serving? I feel copy the /etc/ssl/certs/ca-certificates.crt is not elegant.

I think it should be handled by users. And we do not need to copy certs to it.

BTW, without these libraries, we will need to use curlimages/curl:latest or buildpack-deps:stable-curl to download and unpack miniconda.

@kemingy
fix PATH when dev is false
92b8cdf 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
use python without absolute path
ca99fa7 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
install conda with multi stage build
1118225 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
support conda for non-dev env
fcf514f 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
support mamba, fix sudo group and user directory llb
7fd7d6a 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
fix mamba, really buggy
c628f98 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
algin py default version
aebcd82 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
Copy link
Member Author
kemingy commented Nov 30, 2022

Breaking changes:

* [x]  `base()`

* [x]  `dev_tools()`

* [x]  `install.conda()`

* [x]  `install.python()`

* [ ]  `install.r_lang()` not implemented yet

* [ ]  `install.julia()` not implemented yet

* [ ]  `config.jupyter()` move to envdlib (TBD)

* [ ]  `config.rstudio_server()` move to envdlib (TBD)

Check the progress on the top of this PR. I prefer to implement the rest in other PRs.

It's ready to review. cc @gaocegege @VoVAllen

A bug needs your help: the envd user is not added to the sudo group. I don't know why.

You can try it with:

def build():
    dev_tools()
    install.python()
    install.python_packages(name=["via"])
    install.conda()
    install.conda_packages(name=["exa"])


def serving():
    install.conda(use_mamba=True)
    install.python(version="3.9")
    install.python_packages(name=["via"])
    install.conda_packages(name=["exa"])

The serving image needs to run with docker run because it doesn't have envd-sshd.

@kemingy kemingy marked this pull request as ready for review November 30, 2022 15:01
This was referenced Dec 1, 2022
Closed
Merged
@kemingy
fix mount for dev
4d75f23 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
fix vscode merge
faca087 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
fix user group bug
d9bac12 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
use curl image to replace buildpack curl
89ed9de 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@gaocegege gaocegege mentioned this pull request Dec 6, 2022
Closed
@kemingy
base(dev)
f52b445 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
Copy link
Member Author
kemingy commented Dec 6, 2022

Update

Deleted dev_tools(), adding dev as a parameter to base.

def build():
    # dev
    base(dev=True)
    ...

    # serving
    ...

Changed to use curlimages/curl to download miniconda. This one is tiny.

Fixed user group bug and vscode merge bug.

PTAL @gaocegege @VoVAllen

@gaocegege gaocegege mentioned this pull request Dec 7, 2022
Merged
@gaocegege
Copy link
Member

May I ask why e2e test fails?

@kemingy
Copy link
Member Author
kemingy commented Dec 7, 2022

May I ask why e2e test fails?

I haven't changed the envd file under the test folder. So it should always fail.

If you're okay with the code change, I can update all the test files.

@gaocegege
Copy link
Member

#1228 (comment)

Could you please update the latest demo here?

@gaocegege
Copy link
Member

If you're okay with the code change, I can update all the test files.

The code change LGTM.

@kemingy
fix e2e test
34ea0d3 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
fix e2e test and envd init
4aa8d87 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
fix init test
04f922a 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@kemingy
fix e2e conda dev
c5e01b6 
Signed-off-by: Keming <kemingyang@tensorchord.ai>
@gaocegege gaocegege merged commit 8e9cb2e into tensorchord:main Dec 7, 2022
@kemingy kemingy deleted the serving branch December 7, 2022 05:55
This was referenced Dec 7, 2022
Closed
Closed
AlexXi19 pushed a commit to AlexXi19/envd that referenced this pull request Dec 12, 2022
@kemingy @AlexXi19
feat: support serving (tensorchord#1228)
82cd7e4 
* init base and install

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* wip

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* fix build

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* work for dev

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* change cuda to a image+cuda+cudnn match

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* install python without conda

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* add cert

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* fix PATH when dev is false

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* use python without absolute path

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* install conda with multi stage build

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* support conda for non-dev env

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* support mamba, fix sudo group and user directory llb

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* fix mamba, really buggy

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* algin py default version

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* fix mount for dev

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* fix vscode merge

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* fix user group bug

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* use curl image to replace buildpack curl

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* base(dev)

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* fix e2e test

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* fix e2e test and envd init

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* fix init test

Signed-off-by: Keming <kemingyang@tensorchord.ai>

* fix e2e conda dev

Signed-off-by: Keming <kemingyang@tensorchord.ai>

Signed-off-by: Keming <kemingyang@tensorchord.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gaocegege gaocegege gaocegege left review comments

@aseaday aseaday Awaiting requested review from aseaday aseaday is a code owner

@zwpaper zwpaper Awaiting requested review from zwpaper zwpaper is a code owner

@VoVAllen VoVAllen Awaiting requested review from VoVAllen VoVAllen is a code owner

Assignees
No one assigned
Labels
None yet
Projects
No open projects
envd sprint board
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

feat(lang): v2 language new design
3 participants
@kemingy @gaocegege @VoVAllen
0