8000 Move exporter config loading out of agent by mheffner · Pull Request #130 · streamfold/rotel · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Move exporter config loading out of agent #130

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 25, 2025
Merged

Move exporter config loading out of agent #130

merged 2 commits into from
Jun 25, 2025

Conversation

mheffner
Copy link
Contributor
@mheffner mheffner commented Jun 23, 2025

This is the second round of refactoring to make it a bit easier to incorporate multiple exporter support. With this PR we decouple the parsing, loading and verification of exporter configuration from the main agent startup. The agent gets back a typed structure with the exporter configuration per each pipeline type: metrics, logs and traces. It then matches on each pipeline and finishes the startup of the configured exporter. This will allow us to return different exporter configs per-pipeline in the future, decoupling the agent code from handling it.

Ideally the core agent code wouldn't need to know about each exporter type, which telemetry types they support, and how to start its exporter. In a perfect world the configs for each exporter type would support a into_exporter() method that transformed the config into a running exporter. However, we aren't at that level just yet and it requires some further work. We may take that up after multiple exporters are in place.

Completes: STR-3470

@mheffner mheffner requested a review from rjenkins June 23, 2025 17:40
Copy link
Contributor
@rjenkins rjenkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks nice!

@@ -55,7 +55,7 @@ pub enum Compression {
Lz4,
}

#[derive(Default)]
#[derive(Clone, Default)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for the Clone, just a config builder but wasn't sure if that slipped in on accident?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's because we now need to duplicate the config for each of the metrics, traces and logs pipelines. Before we'd just initialize the exporters from the shared config. Cloning happens here:

rotel/src/init/config.rs

Lines 133 to 136 in e6aca68

cfg.traces = Some(ExporterConfig::Clickhouse(cfg_builder.clone()));
cfg.metrics = Some(ExporterConfig::Clickhouse(cfg_builder.clone()));
cfg.logs = Some(ExporterConfig::Clickhouse(cfg_builder));

@mheffner mheffner merged commit 642e829 into main Jun 25, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
0