A self-hostable, distributed Nix binary cache server with flexible storage strategies
Key Features Installation Configuration Usage Known Limitations Contributing License Acknowledgements
This project provides a solution for distributed Nix binary cache management, developed as part of the bachelor's thesis "Distributed Nix Archive Storage" at Brno University of Technology, Faculty of Information Technology by Radim Mifka (2025).
-
Distributed Cache Network
Uses OpenDHT for decentralized access to cache servers across multiple nodes. -
Cachix-Compatible
Supports the Cachix client for seamless binary pushing, fetching and deploying. -
Flexible Storage Backends
Supports local and S3 storage backends with optional storage strategies for distributing data across multiple backends. -
Declarative Configuration
Simple YAML configuration with per-cache settings, supporting multiple cache instances on a single server.
The simplest way to build the project is using Nix:
# Build the project
nix-build
# Run the server
./result/bin/cache-serverTo set up a development environment:
# Enter development shell
nix-shell
# Run using
./server.sh
# Run Static analysis
mypy cache_server_app/main.pyThe configuration file is located at:
$XDG_CONFIG_HOME/cache-server/config.yaml
server:
database: "node.db" # SQLite database file
hostname: "0.0.0.0" # Bind address
standalone: false # Operate in standalone mode (no DHT)
dht-port: 4222 # Port for DHT communication
dht-bootstrap-host: "other-node" # Bootstrap node for DHT
dht-bootstrap-port: 4223 # Bootstrap node port
server-port: 5001 # HTTP API port
deploy-port: 5002 # Deployment API port
key: "your-secret-key-here" # Authentication key
default-retention: 4 # Default retention period (days)
default-port: 8080 # Default HTTP portDefine multiple cache instances with individual settings:
caches:
- name: "main" # Cache name
retention: 7 # Override default retention (days)
port: 8081 # Override default port
storage-strategy: "split" # Storage strategy (see below)
storages:
- name: "local-fast"
type: "local"
root: "binary-caches" # Local path
split: 40 # Percentage for split strategy
- name: "external-usb"
type: "local"
root: "/Volumes/USB/binary-caches"
split: 60Required Fields:
- name - name of the storage
- type - type of the storage
- root - root path for storage
| Type | Extra Required Fields |
|---|---|
| local | |
| s3 | s3_bucket, s3_region, s3_access_key, s3_secret_key |
Example S3 configuration:
- name: "remote"
type: "s3"
root: "binary-caches" # Folder within bucket
s3_bucket: "cache-1"
s3_region: "eu-central-1"
s3_access_key: "access-key"
s3_secret_key: "secret-key"
โ ๏ธ Strategies only apply if more than one storage backend is defined.
Configure how data is distributed across storage backends:
| Strategy | Description | Config Requirements |
|---|---|---|
| round-robin | Cycles through storages one by one for each new file. | No extra fields |
| in-order | Uses storages in listed order until full before moving to the next. | No extra fields |
| split | Distributes files based on a percentage split between storages. | split field on each storage |
| least-used | Dynamically picks the storage with the least usage. | No extra fields |
Example with split strategy:
storage-strategy: "split"
storages:
- name: "storage-1"
type: "local"
root: "/fast"
split: 70
- name: "storage-2"
type: "s3"
root: "binary-caches"
s3_bucket: "backup"
s3_region: "eu-central-1"
s3_access_key: "key"
s3_secret_key: "secret"
split: 30To enable integration with Cachix Deploy, you can define workspaces and agents in the configuration:
workspaces:
- name: "workspace1"
cache: "first"
agents:
- name: "agent1"
workspace: "workspace1"These settings allow the server to support deployments using Cachix agents and workspaces. For more information on how these components work and how to run agents, see the Cachix Deploy documentation.
# Starting the server
./result/bin/cache-serverUse the Cachix client to push derivations:
# Push a specific derivation
cachix push your-cache-name /nix/store/hash-name
# Push closure of a derivation
nix-store -qR $(which your-program) | cachix push your-cache-nameSee Cachix documentation for more details.
- Legacy Components: Features like workspaces and agents are retained from previous version but remain untested and may not function reliably.
- Lack of Automated Testing: Due to the complexity of the distributed setup, the project currently lacks comprehensive automated tests.
- Monitoring: No unified monitoring or logging system is currently implemented.
- Deprecated CLI Tool: A CLI is no longer maintained, as its functionality has been integrated or made obsolete by configuration-driven design.
Contributions are welcome! Please feel free to submit a Pull Request.
This project is licensed under the MIT License. See the LICENSE file for details.
- Marek Kriลพan, author of the previous version of this project, which served as a foundation for this work
- Marek Rychlรฝ, thesis supervisor, for guidance and support throughout the project
- Attic implementation, which provided inspiration
- OpenDHT for the distributed networking capabilities