Add missing integration tests, clean up dockerfiles, and remove Anaconda dependencies #435

dcrankshaw · 2018-03-13T04:02:40Z

This is a catch all maintenance PR that does a few things:

Removes all Docker images on Anaconda. The biggest change to functionality here is the removal of the module dependency analysis that we did to try to automatically track down all Anaconda and pip dependencies. This feature relied on some semi-private Anaconda APIs and unfortunately was fairly buggy as a result. Furthermore, the conda installs in the docker images broke frequently for a variety of reasons.
Cleans up docker images and dramatically reduces the size of many of them. We now use the debian:stretch-slim Docker image as our base image. For the Python-based images that were previously using an Anaconda image as the base image, this hugely reduces image sizes and speeds up Docker build times.
Adds missing integration tests for MXNet, Pytorch, and the PyTorch->ONNX->Caffe2 deployer.
Adds a dev docker image
Consolidates all container entry scripts into a single, parameterized script

AmplabJenkins · 2018-03-13T05:53:17Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1102/
Test FAILed.

dcrankshaw · 2018-03-13T06:51:07Z

@chester-leung Can you look into why the mxnet test is failing?

chester-leung · 2018-03-13T10:38:50Z

Just figured out the problem. I've committed (but not pushed) changes. Will also need to update documentation a little bit. I'll push changes by this week's meeting.

…

On Mon, Mar 12, 2018 at 11:51 PM Dan Crankshaw ***@***.***> wrote: @chester-leung <https://github.com/chester-leung> Can you look into why the mxnet test is failing? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#435 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ARrkCijkppHPYItBv-Pt1VKae2p_qPuHks5td2xggaJpZM4SoDK1> .

AmplabJenkins · 2018-03-17T05:06:29Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1130/
Test FAILed.

AmplabJenkins · 2018-03-17T07:56:08Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1133/
Test FAILed.

AmplabJenkins · 2018-03-23T03:22:37Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1167/
Test FAILed.

AmplabJenkins · 2018-03-23T05:46:42Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1168/
Test FAILed.

…_data, a DataDesc object

AmplabJenkins · 2018-03-30T16:48:56Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1221/
Test FAILed.

…te packages

AmplabJenkins · 2018-03-30T20:06:30Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1225/
Test FAILed.

AmplabJenkins · 2018-03-31T00:05:16Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1227/
Test FAILed.

AmplabJenkins · 2018-03-31T00:57:39Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1229/
Test FAILed.

dcrankshaw · 2018-03-31T01:07:22Z

Jenkins test this please

AmplabJenkins · 2018-03-31T01:16:26Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1230/
Test FAILed.

AmplabJenkins · 2018-04-01T21:04:28Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1239/
Test FAILed.

AmplabJenkins · 2018-04-02T17:19:31Z

8000

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1240/
Test FAILed.

…ocker image

AmplabJenkins · 2018-04-02T18:29:34Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1241/
Test FAILed.

AmplabJenkins · 2018-04-02T19:56:49Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1242/
Test FAILed.

AmplabJenkins · 2018-04-02T21:19:01Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1243/
Test PASSed.

AmplabJenkins · 2018-04-02T22:21:15Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1246/
Test FAILed.

dcrankshaw · 2018-04-02T22:23:37Z

jenkins test this please

AmplabJenkins · 2018-04-02T22:35:26Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1247/
Test FAILed.

AmplabJenkins · 2018-04-02T23:03:43Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1248/
Test FAILed.

shaneknapp · 2018-04-02T23:09:02Z

we're experiencing some network issues from campus to the outside world. i just ran ping github.com from one of the ubuntu workers and we're getting ~30% packet loss. this definitely explains the jobs timing out.

Corey-Zumar

LGTM - just a nit and a question

Corey-Zumar · 2018-04-02T22:00:20Z

dockerfiles/Py2RPCDockerfile

+    numpy==1.14.*
+
+
+# RUN conda install -y pyzmq libsodium


Remove commented code

Corey-Zumar · 2018-04-02T22:01:03Z

dockerfiles/MXNetContainerDockerfile

-      && conda install -y --file /lib/python_container_conda_deps.txt \
-      && pip install mxnet==1.0.0 \
-      && apt-get install -y graphviz \
-      && pip install graphviz


Is this graphviz package no longer required?

Graphviz was never required. It just got spuriously added.

Corey-Zumar · 2018-04-02T22:12:48Z

integration-tests/deploy_mxnet_models.py

@@ -131,7 +126,7 @@ def get_test_point():
            mxnet_model = mx.mod.Module(softmax)
            mxnet_model.fit(data_iter, num_epoch=0)


Nit: data_iter isn't defined anywhere in this example. Consider defining it in the same fashion as the MXNet test case: data_iter = mx.io.CSVIter(data_csv=train_path, data_shape=(785, ), batch_size=1)

Not sure what you're referring to here? data_iter is defined on line 100/95 of this file.

My mistake. I was referring to the "example" subsections of the documentation for the mxnet container. These sections don't define data_iter

Ahh I see. Yeah good call, I'll add that.

Corey-Zumar · 2018-04-02T22:20:15Z

dockerfiles/Caffe2OnnxDockerfile

+RUN cd ~ && python -c 'from caffe2.python import core'
+
+RUN pip install onnx
+RUN python -c "import onnx"


Just confirming that this isn't a piece of test code that should be removed (same for line 16); for debugging purposes, it seems helpful to verify correct import functionality for critical packages when the docker container is being built.

Yeah that's intentional

dcrankshaw · 2018-04-03T04:17:30Z

jenkins test this please

shaneknapp · 2018-04-03T15:03:52Z

test this please

AmplabJenkins · 2018-04-03T17:36:39Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/1254/
Test PASSed.

dcrankshaw force-pushed the fix_missing_integration_tests branch from c6245d4 to ab88806 Compare March 23, 2018 03:00

dcrankshaw and others added 6 commits March 30, 2018 08:33

added missing mxnet, pytorch, and pytorch-onnx-caffe2 integration tests

71c348a 8000

change data_shapes parameter in integration test to data_iter.provide…

ebc8125

…_data, a DataDesc object

updated documentation in deployers/mxnet.py to match changes

7fcda03

typo in file name

ce42507

changed print statements to logging

87375a4

format code

b086b8f

dcrankshaw and others added 3 commits March 30, 2018 12:32

fixed some onnx issues but the container still is not working

dec8ffe

fixed container build issues related to conda segfaults and out of da…

6747b4d

…te packages

reverted .gitignore

7cb6620

dcrankshaw force-pushed the fix_missing_integration_tests branch from bd34214 to 7cb6620 Compare March 30, 2018 19:41

fixed mxnet issues

b6ded28

fixed onnx tests

301a52b

format and remove conda update conda

b38270b

dcrankshaw added 3 commits March 31, 2018 19:22

major overhaul of docker images

cbb292e

Removed conda module deps analyzer

2e106f3

All docker images building

41d4660

fixed onnx caffe docker image

16ed2c0

changed way we handle docker login for k8s docker repo

b3e90dc

check out specific caffe2 commit and install awscli in clippertests d…

149b157

…ocker image

dcrankshaw added 2 commits April 2, 2018 11:45

catch exception in kubernetes integration test

00e421e

install codedepends dependencies

4767661

dcrankshaw added 2 commits April 2, 2018 21:36

check R packages are properly installed

a458bf5

removed conda module analysis code

29a7249

dcrankshaw changed the title ~~Adds missing mxnet, pytorch, and pytorch-onnx-caffe2 integration tests~~ Add missing integration tests, clean up dockerfiles, and fix jenkins issues Apr 2, 2018

dcrankshaw changed the title ~~Add missing integration tests, clean up dockerfiles, and fix jenkins issues~~ Add missing integration tests, clean up dockerfiles, and remove Anaconda dependencies Apr 2, 2018

added maintainer label to all dockerfiles

b31b711

Corey-Zumar and others added 2 commits April 2, 2018 18:38

Add Rscript for installing R container test dependencies

1a3dc89

Added trycatch to r install scripts

d340c39

Corey-Zumar approved these changes Apr 3, 2018

View reviewed changes

E9EB

added data_iter to mxnet example

c81f587

dcrankshaw merged commit 7460291 into ucbrise:develop Apr 3, 2018

This was referenced Apr 4, 2018

Use pip freeze to add dependency tracking back to Clipper #459

Open

Update RPCDockerfile to fix build error #453

Closed

		@@ -131,7 +126,7 @@ def get_test_point():
		mxnet_model = mx.mod.Module(softmax)
		mxnet_model.fit(data_iter, num_epoch=0)

Add missing integration tests, clean up dockerfiles, and remove Anaconda dependencies #435

Add missing integration tests, clean up dockerfiles, and remove Anaconda dependencies #435

Uh oh!

Conversation

Uh oh!

Uh oh!

AmplabJenkins commented Mar 13, 2018

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!