Using Vagrant:
$ vagrant plugin install vagrant-hostmanager
$ vagrant up
Then head to http://stage1.dev/.
Current code for STAGE1 is prototype quality. It's dirty, and I'm really not proud of it. Actually, if someone working for me produced code like this, I'd fire him. Yeah.
And please don't look at the commit history.
The whole codebase is undergoing drastic refactoring to bring its quality up to an acceptable level.
STAGE1 is quite a big project to grasp, so don't worry about understanding each and every detail (even I don't understand everything that's going on sometimes), and rather focus on the parts you're interesting in contributing to. The Vagrant VM should give you a fully functional dev environment (if not, it's a bug).
There a few CLI commands that you may find interesting:
stage1:user:role:add <username> <role>: add a security role to a user. You might want give your user theROLE_ADMINrolestage1:github -u <username> <path>: a non-interactive CLI github client that uses the Access Token associated with the given user
The directory structure is basically that of a standard Symfony2 project, with a few additions:
docker/holds the Dockerfiles for all STAGE1's base containersfabfile/has the fabric configurationhelp/is the help site, built with jekyllnode/has all node related codepackercontains the packer configuration and support filesservice/has the daemontools services definitions
STAGE1 is composed of the following bricks:
- A Symfony2 app for the web UI
- A nodejs server for websockets (using primusjs)
- A nodejs server to fetch docker containers logs (aldis)
- A nodejs reverse-proxy to route URLs to the right containers (hipache)
- A few RabitMQ consumers
- A MySQL server for all persistent data storage
- A Redis server for volatile or performance-demanding storage
- A very basic code analyzer to determine the build steps (yuhao)
When a new push notification is received (through GitHub's push WebHook), a build order is sent through RabbitMQ to whatever builder is currently available. The builder clones and analyzes the project using yuhao and decides whether it should use the DockerfileStrategy (projects with a Dockerfile) or the DefaultStrategy (all other cases). The DockerfileStrategy is the equivalent of a docker build, with a few options to accomodate STAGE1's requirements. The default strategy creates a build container and uses the build script generated by Yuaho. Once built, the resulting container is commited and executed, a route is added for Hipache in Redis and the post-build listeners are executed.
Build output management depends on the strategy. In the DockerfileStrategy, build output is received directly by the docker-php client, and all messages are sent through RabbitMQ from there. In the StandardStrategy, output is received by Aldis, which in turn forwards everything into RabbitMQ. Two consumers are expected to receive this output:
- the docker-output consumer, that saves all output to a Redis database
- the websocket consumer, that will forward all output via websocket for live output streaming in the browser
List of existing RabbitMQ consumers:
consumer-build, the actual builderconsumer-kill, used to kill buildsconsumer-stop, used to stop already running containersdocker-output, used to save docker containers output to Redisconsumer-project-import, used to import projects
See the service/ folder for more informations.
In addition to the RabbitMQ consumers, a few other services exist:
- aldis is a nodejs server that connects to the docker remote API and forward all containers output to RabbitMQ
- hipache is the reverse proxy used to route URLs to their containers (note: STAGE1 runs a patched version of hipache to allow arbitrary middleware stacking)
websocketis the websocket server, written with primusjs
See the service/ folder for more informations.
Using Packer:
VMware users:
$ packer build - packer/dev.json
$ vagrant box add \
--name ubermuda/stage1-dev \
--box-version 9999 \
--provider vmware_desktop \
packer/boxes/dev.box
VirtualBox users:
$ packer build - packer/dev.json
$ vagrant box add \
--name ubermuda/stage1-dev \
--box-version 9999 \
packer/boxes/dev.box
We use a box version of 9999 to trick Vagrant into using our version rather than the "official" one.