Popper is a tool for defining and executing container-native workflows in Docker, as well as other container engines. With Popper, you define a workflow in a YAML file, and then execute it with a single command. A workflow file looks like this:
version: '1'
steps:
- uses: docker://byrnedo/alpine-curl:0.1.8
args: [-LO, https://github.com/datasets/co2-fossil-global/raw/master/global.csv]
- uses: docker://python:3.8.1-alpine
args: [scripts/get_mean_by_group.py, global.csv, '5']Assuming the above is stored in a wf.yml file, the workflow gets
executed by running:
popper run -f wf.ymlKeep reading down to find installation instructions. For more information on the YAML syntax, see here.
The high-level goals of this project are to provide:
- Lightweight workflow definition syntax. Defining a workflow is
as simple as writing file in a lightweight YAML syntax and
invoking
popper run(see demo above). If you're familiar with Docker Compose, you can think of Popper as Compose but for workflows instead of services. - An abstraction over container runtimes. In addition to Docker, Popper can seamlessly execute workflows in other runtimes by interacting with distinct container engines. We currently support Singularity, as well as running Docker inside virtual machines (via Vagrant). We are working on adding Podman to the list.
- Continuous integration. Generate configuration files for distinct CI services, allowing users to run the exact same workflows they run locally on Travis, Jenkins, Gitlab, Circle and others.
- Workflow development. Aid in the implementation and debugging of workflows, and provide with an extensive list of example workflows that can serve as a starting point.
This repository contains:
cli/. The codebase of the command-line tool.docs/. General documentation containing guides, CLI documentation and pointers to other resources.gh-pagesbranch. Contents of our landing page.
To run workflows, you need to have Python 3.6+, Git and a container
engine installed (Docker, Singularity and
Vagrant are currently supported). To install
Popper you can use pip. We recommend
to install in a virtual environment (see here for more on
virtualenv). To install:
pip install popperOnce installed, you can get an overview and list of available commands:
popper --helpFor a Quickstart guide on how to use Popper, look here.
Anyone is welcome to contribute to Popper! To get started, take a look at our contributing guidelines, then dive in with our list of good first issues.
Popper adheres to the code of conduct posted in this repository. By participating or contributing to Popper, you're expected to uphold this code. If you encounter unacceptable behavior, please immediately email us.