⭐⭐⭐ Kalavai platform is open source, and free to use in both commercial and non-commercial purposes. If you find it useful, consider supporting us by giving a star to our GitHub project, joining our discord channel and follow our Substack.
AI in the cloud is not aligned with you, it's aligned with the company that owns it. Make sure you own your AI
Kalavai is an open source tool that turns any devices into a self-hosted AI platform. It aggregates resources from multiple machines, including cloud, on prem and personal computers, and is compatible with most model engines to make model deployment and orchestration simple and reliable.
Kalavai's goal is to make using self-hosted AI (GenAI models and agents) in real applications accessible and affordable to all. It's a tool that transforms machines into a magic box that integrates all the components required to make AI useful in the age of massive computing, from model deployment and orchestration to Agentic AI.
- Manage multiple devices resources as one. One pool of RAM, CPUs and GPUs
- Deploy open source models seamlessly across devices, wherever they are (cloud, on premises, personal devices)
- Beyond LLMs: not just for large language models, but text-to-speech, speech-to-text, image understanding, coding generation and embedding models.
- The hybrid dream: build on your laptop, move to the cloud (any!) with zero changes
- Auto-discovery: all models are automatically exposed through a single OpenAI-like API and a ChatGPT-like UI playground
- Compatible with most popular model engines
- Easy to expand to custom workloads
Powered by Kalavai
- CoGen AI: A community hosted alternative to OpenAI API for unlimited inference.
- Create your own Free Cursor/Windsurf Clone
- 11 June 2025: Native support for Mac and Raspberry pi devices (ARM).
- 20 February 2025: New shiny GUI interface to control LLM pools and deploy models
- 31 January 2025:
kalavai-clientis now a PyPI package, easier to install than ever!
More news
- 27 January 2025: Support for accessing pools from remote computers
- 9 January 2025: Added support for Aphrodite Engine models
- 8 January 2025: Release of a free, public, shared pool for community LLM deployment
- 24 December 2024: Release of public BOINC pool to donate computing to scientific projects
- 23 December 2024: Release of public petals swarm
- 24 November 2024: Common pools with private user spaces
- 30 October 2024: Release of our public pool platform
We currently support out of the box the following LLM engines:
Coming soon:
Not what you were looking for? Tell us what engines you'd like to see.
Kalavai is at an early stage of its development. We encourage people to use it and give us feedback! Although we are trying to minimise breaking changes, these may occur until we have a stable version (v1.0).
- Get a free Kalavai account and access unlimited AI.
- Full documentation for the project.
- Join our Substack for updates and be part of our community
- Join our discord community
The kalavai-client is the main tool to interact with the Kalavai platform, to create and manage both local and public pools and also to interact with them (e.g. deploy models). Let's go over its installation.
Requirements
For workers sharing resources with the pool:
- A laptop, desktop or Virtual Machine
- Docker engine installed (for linux, Windows and MacOS) with privilege access.
Support for Windows and MacOS workers is experimental: kalavai workers run on docker containers that require access to the host network interfaces, thus systems that do not support containers natively (Windows and MacOS) may have difficulties finding each other.
The client is a python package and can be installed with one command:
pip install kalavai-clientKalavai is free to use, no caps, for both commercial and non-commercial purposes. All you need to get started is one or more computers that can see each other (i.e. within the same network), and you are good to go. If you are interested in join computers in different locations / networks, contact us or book a demo with the founders.
You can create and manage your pools with the new kalavai GUI, which can be started with:
kalavai gui startThis will expose the GUI and the backend services in localhost. By default, the GUI is accessible via http://localhost:3000. In the UI users can create and join LLM pools, monitor devices, deploy LLMs and more.
Check out our getting started guide for next steps.
Check our examples to put your new AI pool to good use! For an end to end tour, check our self-hosted and public LLM pools guides.
If your system is not currently supported, open an issue and request it. We are expanding this list constantly.
Hardware and OS compatibility
Currently seed nodes are supported exclusively on linux machines (x86_64 platform). However Kalavai supports mix-pools, i.e. having Windows and MacOS computers as workers.
Since worker nodes run inside docker, any machine that can run docker should be compatible with Kalavai. Here are instructions for linux, Windows and MacOS.
The kalavai client, which controls and access pools, can be installed on any machine that has python 3.10+.
amd64orx86_64CPU architecture for seed and worker nodes.arm64CPU architecture for worker nodes.- NVIDIA GPU
- Mac M series, AMD and Intel GPUs are currently not supported (interested in helping us test it?)
- Kalavai client on Linux
- [TEMPLATE] Distributed LLM deployment
- Kalavai client on Windows (worker only)
- Kalavai client on Windows WSL2 (seed and worker)
- Self-hosted LLM pools
- Collaborative LLM deployment
- Ray cluster support
- Kalavai client on Mac (worker only)
- Kalavai pools UI
- Support for AMD GPUs
- Support for Mac M GPUs
- Docker install path
Anything missing here? Give us a shout in the discussion board
- PR welcome!
- Join the community and share ideas!
- Report bugs, issues and new features.
- Help improve our compatibility matrix by testing on different operative systems.
- Follow our Substack channel for news, guides and more.
- Community integrations are template jobs built by Kalavai and the community that makes deploying distributed workflows easy for users. Anyone can extend them and contribute to the repo.
Expand
Python version >= 3.6.
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
sudo apt install python3.10 python3.10-dev python3-virtualenv python3-venv
virtualenv -p python3.10 env
source env/bin/activate
sudo apt install python3.10-venv python3.10-dev -y
pip install -U setuptools
pip install -e .[dev]Build python wheels:
bash publish.sh buildTo run the unit tests, use:
python -m unittest