┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐
Project A
│ _ │
╱ ╲ .─────. ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─
┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ ╔══╬═══▶▕ ▏═══▶(|||||||)═╬═════╗ Project C │
Ingress Project ║ ╲ ╱ `─────' ║ │ _
│ │ ║ │ ▔ │ ║ ╱ ╲ .─────. │
════▶╔ ═ ═ ╗ ║ Workload ╚════╬═▶▕ ▏═══▶(|||||||)
│ _ │ ║ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘ ╲ ╱ `─────' │
║ ╱#╲ ║ .─────. ║ │ ▔
│ ════▶ ▕###▏ ════▶(|||||||)═╬═════╣ ▲ │
║ ╲#╱ ║ `─────' ║ └ ─ ─║─ ─ ─ ─ ─ ─ ─ ─
│ ▔ Dataset │ ║ ║
════▶╚ ═ ═ ╝ ║ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ╬ ─ ─ ─
│ Boundary │ ║ _ _ ║ │
─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ║ │ ╱ ╲ .─────. ╱ ╲ .─────.
╚══════▶▕ ▏═══▶(|||||||)═══▶▕ ▏═══▶(|||||||) │
│ ╲ ╱ `─────' ╲ ╱ `─────'
▔ ▔ │
│Project B
─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘
Clusterless is a tool for deploying decentralized, scalable, and secure data-processing workloads for continuously arriving data, across clouds.
By leveraging native pay-as-you-go primitives, no runtimes or dedicated services need to be managed.
Zero data arriving means zero costs (other than storage for historical data).
Draft documentation can be found here: https://docs.clusterless.io/
For example scenarios, see
- How To - step-by-step guides
- clusterless-aws-example - simple examples to start with
- aws-s3-log-pipeline - end-to-end sample pipeline for processing AWS S3 access logs