Hokusai is a Docker + Kubernetes CLI for application developers.
Hokusai "dockerizes" applications and manages their lifecycle throughout development, testing, and release cycles.
Hokusai wraps calls to Kubectl, Docker, Docker-Compose and Git with a CLI, and defines a CI workflow.
Hokusai currently only supports Kubernetes deployments on AWS, configured to pull from ECS container repositories (ECR), although other providers may be added in the future.
At Artsy, as we began working with Kubernetes, while impressed with its design, capabilities, and flexibility, we were in need of tooling we could deliver to agile development teams that addressed the day-to-day tasks of application development, delivery, introspection and maintenance, while providing a clean and uncomplicated interface.
Transitioning teams to the Docker / Kubernetes ecosystem can be intimidating, and comes with a steep learning curve. We set out to create a Heroku-like CLI that would shepherd the application developer into the ecosystems of Docker and Kubernetes, and while introducing new tooling and concepts, outlining a clear practice for dependency management, local development, testing and CI, image repository structure, deployment and orchestration.
We recommend installing via Homebrew:
$ brew update
$ brew tap artsy/formulas
$ brew install hokusai
If you previously installed Hokusai via an alternate installation method, you may need to force the link step.
$ brew link --overwrite hokusai
If you previously installed Hokusai via Pip, you may want to first uninstall it:
$ pip uninstall hokusai
curl -sSL https://raw.githubusercontent.com/artsy/hokusai/main/get-hokusai.sh | sudo bash
Note: This method installs Hokusai to /usr/local/bin/hokusai.
Hokusai can be installed via Pip, on MacOS or Linux. If you do so, please first go through Pyenv, Python, and Virtualenv steps.
Python 3.7+ is required.
pip install hokusai
Note: If Pip fails at upgrading your system Python packages, try:
pip install hokusai --ignore-installed
We also maintain Hokusai Docker images for running Hokusai in Docker.
Release artifacts are available on Github.
Release artifacts are also available in AWS S3. You can use this convenience script or Curl to fetch them.
Hokusai currently supports Python 3.7+ only. The last version that supported Python 2.x was v0.5.18.
We assume that you already have Kubernetes cluster, Git, Docker, and Docker-Compose set up, and that you have an AWS account. Perform the following steps to setup Hokusai:
-
Configure Hokusai
hokusai configure --kubectl-version <kubectl version> --s3-bucket <bucket name> --s3-key <file key>Provide the Kubectl version matching that of your Kubernetes clusters, as well as the S3 bucket name/key where your org's Kubectl config file is stored.
For system administrators: see Administering Hokusai for instructions on preparing AWS and Kubernetes, and on publishing a Kubectl config file.
For Artsy developers: see artsy/README for the current way of installing and configuring hokusai.
-
Enable Bash autocompletion:
eval "$(_HOKUSAI_COMPLETE=source hokusai)"
See Getting Started to start using Hokusai for your project.
A full command reference can be found in Command Reference.
Hokusai can be used to simplify the process of spinning up a "review app" instance of your project, based on a feature branch or pull request.
Full details are in the Review App reference.
To work on Hokusai itself, please set up:
We recommend using Pyenv to install the correct version of Python. For a tutorial of Pyenv, see this guide.
When installing on MacOS, please make sure to use brew-installed openssl and readline libraries, and xcode-installed zlib library. And make sure these libraries are correctly linked. Like so:
brew install openssl readline zlib
echo 'export PATH="/usr/local/opt/openssl@1.1/bin:$PATH"' >> ~/.bash_profile
echo 'export LDFLAGS="-L/usr/local/opt/openssl@1.1/lib"' >> ~/.bash_profile
echo 'export CPPFLAGS="-I/usr/local/opt/openssl@1.1/include"' >> ~/.bash_profile
echo 'export PKG_CONFIG_PATH="/usr/local/opt/openssl@1.1/lib/pkgconfig"' >> ~/.bash_profile
Hokusai is currently tested on Python 3.9.10 so we recommend using that Python version.
If you use Pyenv to install Python, you should see an output similar to this:
pyenv install 3.9.10
python-build: use openssl from homebrew
python-build: use readline from homebrew
Downloading Python-3.9.10.tar.xz...
-> https://www.python.org/ftp/python/3.9.10/Python-3.9.10.tar.xz
Installing Python-3.9.10...
python-build: use tcl-tk from homebrew
python-build: use readline from homebrew
python-build: use zlib from xcode sdk
Installed Python-3.9.10 to $HOME/.pyenv/versions/3.9.10
With the desired Python version installed, activate it globally:
pyenv global 3.9.10
Note: If you want to create a PyInstaller distribution, Python must be installed with development libraries. Use the environment variable PYTHON_CONFIGURE_OPTS="--enable-framework" on Darwin and PYTHON_CONFIGURE_OPTS="--enable-shared" on Linux when running pyenv install.
We recommend using a virtual environment to isolate Hokusai's dependencies from that of other projects on your local environment.
The Pyenv install comes with pyenv-virtualenv which can be used to create virtual environments.
Use Poetry to install Hokusai's dependencies as well as Hokusai itself in editable mode. See this guide for working with Poetry.
Install Poetry:
pip install poetry
Install dependencies and Hokusai in editable mode:
poetry install
To update dependencies:
poetry lock
The version reported by hokusai version is taken from hokusai/_version.py file. The file is dynamically generated by setuptools_scm during CI and it is not version controlled. To have the file in local dev, run:
python -m setuptools_scm
Minikube is used for integration tests.
brew install minikube
minikube start --kubernetes-version=<version of your Kubernetes clusters, example: v1.2.3>
To run all tests:
make tests
To run only unit tests:
make test
Only integration tests:
make integration
Only specific modules, TestClasses, or even methods:
python -m unittest test.unit.test_module.TestClass.test_method
Tip: Set DEBUG=1 environment variable to print boto logging
Merging a branch into main automatically creates a beta version useful for testing.
To install the beta:
$ brew uninstall hokusai
$ brew uninstall hokusai-beta
$ brew update
$ brew tap artsy/formulas
$ brew install hokusai-beta
curl -sSL https://raw.githubusercontent.com/artsy/hokusai/main/get-hokusai.sh | sudo bash -s beta
To create an official release, such as v1.2.3, perform the following:
-
Create a branch named
prepare-v1.2.3and make the following changes:- Bump version in RELEASE_VERSION file.
- Upate CHANGELOG.
- Open a PR to merge into
main. Please see past PRs for example.
-
Open a PR to merge
mainintorelease. Please see past PRs for example.
The project is named for the great Japanese artist Katsushika Hokusai (1760-1849).
This project is the work of engineers at Artsy, the world's leading and largest online art marketplace and platform for discovering art. One of our core Engineering Principles is being Open Source by Default which means we strive to share as many details of our work as possible.
You can learn more about this work from our blog and by following @ArtsyOpenSource or explore our public data by checking out our API. If you're interested in a career at Artsy, read through our job postings!