I am using custom docker image to run the vertex ai pipeline. So below are the necessary steps:
- Create repository in Artifact Registry.
- Edit the parameters in docker_build.sh and run ./docker_build.sh to build docker image
- Edit the parameters in docker_push_gcr.sh and run ./docker_push_gcr.sh to build docker image to push docker image to artifact registry.
- Also added requirements file, so make sure docker_build.sh, docker_push_gcr.sh and requirements.txt files are in the same folder.
- we can use this image as our custom docker container to run Vertex AI pipeline.
- Please follow the steps mentioned in sky_assignment.ipynb and run the pipeline.