8000 GitHub - typicalhacker/fluvio: Lean and mean distributed stream processing system written in rust and web assembly. Alternative to Kafka + Flink in one.
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
/ fluvio Public
forked from infinyon/fluvio

Lean and mean distributed stream processing system written in rust and web assembly. Alternative to Kafka + Flink in one.

www.fluvio.io/

License

Apache-2.0 license
0 stars 515 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

typicalhacker/fluvio

BranchesTags
 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2,312 Commits
.github
.github
 
 
actions
actions
 
 
build-scripts
build-scripts
 
 
connector
connector
 
 
crates
crates
 
 
dev-tools
dev-tools
 
 
examples
examples
 
 
k8-util
k8-util
 
 
makefiles
makefiles
 
 
release-tools
release-tools
 
 
rfc
rfc
 
 
smartmodule
smartmodule
 
 
tests
tests
 
 
tls
tls
 
 
.gitignore
 
 
CHANGELOG.md
CHANGELOG.md
 
 
CODE-OF-CONDUCT.md
CODE-OF-CONDUCT.md
 
 
CONTRIBUTING.md
CONTRIBUTING.md
 
 
Cargo.lock
Cargo.lock
 
 
Cargo.toml
Cargo.toml
 
 
DEVELOPER.md
DEVELOPER.md
 
 
LICENSE
LICENSE
 
 
Makefile
Makefile
 
 
README.md
README.md
 
 
RELEASE.md
RELEASE.md
 
 
VERSION
VERSION
 
 
action.yaml
action.yaml
 
 
cliff.toml
cliff.toml
 
 
deny.toml
deny.toml
 
 
rust-toolchain.toml
rust-toolchain.toml
 
 
rustfmt.toml
rustfmt.toml
 
 

Repository files navigation

Fluvio is a lightweight high-performance distributed data streaming system written in Rust and Web Assembly.

Quick Start - Get started with Fluvio in 2 minutes or less!

Step 1. Download Fluvio Version Manager:

On your terminal run

curl -fsS https://hub.infinyon.cloud/install/install.sh | bash

Follow the instructions and copy/paste the path to the bin directory to your startup script file.

Fluvio version manager will give you the ability to download different versions of Fluvio:

  • Including our read-only edge cluster with built-in compression, caching, and mirroring to never lose data even with extended downtimes.
  • Or our Developer Preview of Stateful Streaming which we are building using the web assembly component model to support all web assembly compatible languages.

Step 2. Start local cluster:

The following command will start a local cluster by default:

fluvio cluster start

Step 3. Create Topic:

The following command will create a topic called hello-fluvio:

fluvio topic create hello-fluvio

Step 4. Produce to Topic, Consume From Topic:

Produce data to your topic. Run the command first and then type some messages:

fluvio produce hello-fluvio
> hello fluvio
Ok!
> test message
Ok!

Consume data from the topic, Run the following command in a different terminal:

fluvio consume hello-fluvio -B -d

Just like that! You have a local cluster running.

Using Pre-Build Fluvio Versions

You may want to prefer other Fluvio versions than the latest stable release. You can do so by specifying the version in the VERSION environment variable.

Install Latest Release (as of master branch)

$ curl -fsS https://hub.infinyon.cloud/install/install.sh | VERSION=latest bash

Install Specific Version

$ curl -fsS https://hub.infinyon.cloud/install/install.sh | VERSION=x.y.z bash

Next Steps

Now that you have a cluster running you can try building data flows in different paradigms.

Check Fluvio Core Documentation

Fluvio documentation will provide additional context on how to use the Fluvio clusters, CLI, clients, a development kits.

Learn how to build custom connectors

Fluvio can connect to practically any system that you can think of.

  • For first party systems, fluvio clients can integrate with the edge system or application to source data.
  • For third party systems fluvio connectors connect at the protocol level and collects data into fluvio topics.

Out of the box Fluvio has native http, webhook, mqtt, kafka inbound connectors. In terms of outbound connectors out of the box Fluvio supports SQL, DuckDB, Graphite, experimental builds of Redis, S3 etc.

Using Connector Development Kit, we built our existing connectors in a matter of few days. Check out the docs and let us know if you need help building any connector.

Learn how to build custom smart modules

Fluvio applies wasm based stream processing and data transformations. We call these reusable transformation functions smart modules. Reusable Smart modules are built using Smart Module Development Kit and can be distributed using InfinyOn Cloud hub.

There are some limitations on the amount of polyglot development interface support. While bindings can be generated for wasm compatible languages, there are quirks in that approach. We have a better solution with Stateful Service Development Kit, which we are implementing using the web assembly component model. In the upcoming releases we will be able to natively support all wasm compatible programming languages.

Try workflows on InfinyOn Cloud

InfinyOn Cloud is Fluvio on the cloud as a managed service. All new users get $3000 worth of credits to build data flows on InfinyOn Cloud.

Clients

Language Specifc API docs:

Community Maintained:

Contributing

If you'd like to contribute to the project, please read our Contributing guide.

Contributors are awesome

Made with contrib.rocks.

License

This project is licensed under the Apache license.

About

Lean and mean distributed stream processing system written in rust and web assembly. Alternative to Kafka + Flink in one.

www.fluvio.io/

Resources

Readme

License

Apache-2.0 license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages

  • Rust 93.8%
  • Shell 5.4%
  • Other 0.8%
0