fix(batch-exports): Report Prometheus metrics in new consumer #33739
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
PR Summary
Added missing Prometheus metrics tracking in the new batch exports consumer implementation, specifically for monitoring rows and bytes exported during the transform_batch loop.
- Added rows_exported_counter tracking in the SPMC consumer's transform_batch loop to count processed rows
- Added bytes_exported_counter to measure data volume throughput during export operations
- Implemented proper metrics handling for file metadata chunks during finalization
- Maintained consistency with existing metrics patterns from metrics.py for monitoring batch export performance
1 file reviewed, 1 comment
Edit PR Review Bot Settings | Greptile
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Important
👉 Stay up-to-date with PostHog coding conventions for a smoother review.
Problem
We publish metrics for rows exported and bytes exported but these were missed in the new consumer implementation.
Changes
Update Prometheus counters.
Did you write or update any docs for this change?
How did you test this code?
Relying on existing tests