Fluvio is a lightweight high-performance distributed data streaming system written in Rust and Web Assembly.
On your terminal run
curl -fsS https://hub.infinyon.cloud/install/install.sh | bashFollow the instructions and copy/paste the path to the bin directory to your startup script file.
Fluvio version manager will give you the ability to download different versions of Fluvio:
- Including our read-only edge cluster with built-in compression, caching, and mirroring to never lose data even with extended downtimes.
- Or our Developer Preview of Stateful Streaming which we are building using the web assembly component model to support all web assembly compatible languages.
The following command will start a local cluster by default:
fluvio cluster startThe following command will create a topic called hello-fluvio:
fluvio topic create hello-fluvioProduce data to your topic. Run the command first and then type some messages:
fluvio produce hello-fluvio
> hello fluvio
Ok!
> test message
Ok!Consume data from the topic, Run the following command in a different terminal:
fluvio consume hello-fluvio -B -dJust like that! You have a local cluster running.
Now that you have a cluster running you can try building data flows in different paradigms.
Fluvio documentation will provide additional context on how to use the Fluvio clusters, CLI, clients, a development kits.
Fluvio can connect to practically any system that you can think of.
- For first party systems, fluvio clients can integrate with the edge system or application to source data.
- For third party systems fluvio connectors connect at the protocol level and collects data into fluvio topics.
Out of the box Fluvio has native http, webhook, mqtt, kafka inbound connectors. In terms of outbound connectors out of the box Fluvio supports SQL, DuckDB, Graphite, experimental builds of Redis, S3 etc.
Using Connector Development Kit, we built our existing connectors in a matter of few days. Check out the docs and let us know if you need help building any connector.
Fluvio applies wasm based stream processing and data transformations. We call these reusable transformation functions smart modules. Reusable Smart modules are built using Smart Module Development Kit and can be distributed using InfinyOn Cloud hub.
There are some limitations on the amount of polyglot development interface support. While bindings can be generated for wasm compatible languages, there are quirks in that approach. We have a better solution with Stateful Service Development Kit, which we are implementing using the web assembly component model. In the upcoming releases we will be able to natively support all wasm compatible programming languages.
- Stateful Service Development Kit docs- Coming Soon Request Developer Preview Invite
InfinyOn Cloud is Fluvio on the cloud as a managed service. All new users get $3000 worth of credits to build data flows on InfinyOn Cloud.
Language Specifc API docs:
Community Maintained:
If you'd like to contribute to the project, please read our Contributing guide.
Made with contrib.rocks.
This project is licensed under the Apache license.