CloudExec is a command-line tool for easily running cloud-based jobs on DigitalOcean. Start, manage and pull the results of jobs from your terminal.
brew tap trailofbits/tools
brew install cloudexecbrew update && brew upgrade cloudexecalternati 8000 vely, you can install from a GitHub release:
Download the latest release for your platform from the releases page.
Releases are signed with sigstore. You can verify using cosign with the following example command:
cosign verify-blob \
--certificate-identity-regexp "https://github.com/crytic/cloudexec.*" \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
--bundle cloudexec-<version>-<os>-<arch>.tar.gz.bundle \
cloudexec-<version>-<os>-<arch>.tar.gztar -xzf cloudexec-<version>-<os>-<arch>.tar.gz
mv cloudexec /usr/local/binRunning the command below will build the CLI tool from source with a binary named cloudexec in a dist folder:
make buildThen, move the resulting binary from ./dist/clouexec into your PATH.
Nix users can run nix build and then nix profile install ./result to install cloudexec. A helper command make nix-install is available which performs these steps for you and also upgrades an existing version of cloudexec that might already be installed.
CloudExec requires DigitalOcean API credentials to manage droplets, and Spaces credentials to store state and job data. The recommended method for storing and providing your credentials securely is by using the 1Password CLI.
CloudExec supports natively integrating with 1Password, allowing you to reference your credentials stored in your 1Password vault. However, you can also choose to provide plaintext credentials using the cloudexec configure command. Additionally, you can override individual values or the entire configuration by setting the corresponding environment variables.
Save the above tokens in your 1Password vault and install the 1password CLI.
brew install --cask 1password/tap/1password-cli # see the link above for installation instructions on other platformsSign in to your 1Password account.
eval $(op signin)Note what your 1Password secret references are and use them in place of your actual secret values during the cloudexec configure or env var setup steps described in the next section.
These references generally follow the format: op://<vault-name>/<item-name>/<field-name>. For example, if you saved your keys to a vault called Private, in an item called DigitalOcean and the api key field is called ApiKey, then the secret reference to use is op://Private/DigitalOcean/ApiKey.
cloudexec configureor set environment variables:
DIGITALOCEAN_API_KEY
DIGITALOCEAN_SPACES_ACCESS_KEY
DIGITALOCEAN_SPACES_SECRET_ACCESS_KEY
DIGITALOCEAN_SPACES_REGIONRemember, if you save secret values to a .env file, never commit it to any version control system. Add such .env files to your project's .gitignore file to help prevent making such mistakes.
Confirm cloudexec has access to DigitalOcean.
cloudexec checkGenerate a cloudexec.toml configuration file in the current directory.
cloudexec launch initUpdate the cloudexec.toml as needed.
# default nyc3 region and c-2 size droplet, using a cloudexec.toml file in the current directory
cloudexec launch
# custom region and droplet size
cloudexec launch --size c-4 --region sfo2cloudexec logsNote that the logs subcommand will continue to stream logs until you stop with ctrl-c, even after the job is finished and stops producing new logs. This is a read-only command and it is safe to kill it at any point.
cloudexec logs --job 1cloudexec attach
# or
ssh -t cloudexec tmux attach -s cloudexecssh cloudexec# show only runnning jobs, and the last completed job
cloudexec status
# show all jobs
cloudexec status --all# pull from the latest successful job
cloudexec pull example/output
# pull from any job ID
cloudexec pull --job 1 example/output
cloudexec cancelcloudexec cleanNote that there is often a delay while deleting files from Digital Ocean Spaces buckets.
Building and uploading a dedicated DigitalOcean image for cloudexec will simplify your launch configuration and improve startup times.
To do so, install packer with brew install packer. If you're using nix and direnv, it's added to your PATH via the flake's dev shell.
To build and upload a docker image, run the following command. Make sure your DigitalOcean API key is either in your env vars or replace it with the actual token.
packer build -var do_api_token=$DIGITALOCEAN_API_KEY cloudexec.pkr.hcl
This will take care of everything and if you visit the DigitalOcean snapshots page, you'll see a snapshot called cloudexec-20230920164605 or similar. cloudexec will search for snapshots starts with a cloudexec- prefix and it will use the one with the most recent timestamp string.
Now, you can remove everything from the setup command in the example launch config or replace it to install additional tools.