8000 [beatreceiver] Add otel test framework by khushijain21 · Pull Request #44705 · elastic/beats · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[beatreceiver] Add otel test framework #44705

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 32 commits into from
Closed

[beatreceiver] Add otel test framework #44705

wants to merge 32 commits into from

Conversation

khushijain21
Copy link
Contributor
@khushijain21 khushijain21 commented Jun 9, 2025 •
edited
Loading

Proposed commit message

This PR adds otel test binary that helps us write integration tests close to beats.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

cd x-pack/filebeat/
mage docker:composeUp
go test   -tags integration -run TestFilebeatOTelE2E ./tests/integration/ -v

Related issues

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jun 9, 2025
@github-actions GitHub Actions
Copy link
Contributor
github-actions bot commented Jun 9, 2025

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@mergify Mergify
Copy link
Contributor
mergify bot commented Jun 9, 2025

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @khushijain21? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
  • backport-active-all is the label that automatically backports to all active branches.
  • backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
  • backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

mauri870
mauri870 reviewed Jun 10, 2025
View reviewed changes
mauri870
mauri870 reviewed Jun 10, 2025
View reviewed changes
mauri870
mauri870 reviewed Jun 10, 2025
View reviewed changes
mauri870
mauri870 reviewed Jun 11, 2025
View reviewed changes
mauri870
mauri870 reviewed Jun 11, 2025
View reviewed changes
@khushijain21 khushijain21 changed the title [beatreceiver] Add otel test binary" [beatreceiver] Add otel test binary Jun 12, 2025
@khushijain21 khushijain21 marked this pull request as ready for review June 12, 2025 06:58
@khushijain21 khushijain21 requested a review from a team as a code owner June 12, 2025 06:58
@khushijain21 khushijain21 requested review from belimawr and rdner June 12, 2025 06:58
8000
@khushijain21 khushijain21 requested review from cmacknz, leehinman and VihasMakwana and removed request for belimawr June 12, 2025 07:11
@khushijain21 khushijain21 marked this pull request as draft June 12, 2025 12:44
@belimawr belimawr added the Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team label Jun 12, 2025
mauri870
mauri870 reviewed Jun 13, 2025
View reviewed changes
@khushijain21 khushijain21 marked this pull request as ready for review June 16, 2025 13:19
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

leehinman
leehinman reviewed Jun 16, 2025
View reviewed changes
Copy link
Contributor
@leehinman leehinman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For integration tests it is nice to have a binary that you can run by hand. For example with all the current filebeat integration tests we have filebeat.test. Can we modify the mage file so we get a binary that we could run by hand and use it the same way that we use filebeat.test for the current beat integration tests? It would be nice if this binary could take beat configuration or otel configuration files.

The current setup doesn't just test if the receiver does what it is supposed to do, it also tests the conversion logic for beat config to otel config. That isn't necessarily bad, but it does mean that we are limited by the conversion logic in things we can test.

@khushijain21
Copy link
Contributor Author

it also tests the conversion logic for beat config to otel config. That isn't necessarily bad, but it does mean that we are limited by the conversion logic in things we can test.

the conversion logic just moves everything under filebeat to filebeatreceiver::filebeat section in otel. We wouldn't be limited by this logic as such. I rather like the idea that we don't have to write a full beatreceiver config which is quite verbose.

As for the otel binary - yeah that sounds better than this approach. I raised another PR here #44855. Can you take a look

@mauri870
Copy link
Member

the conversion logic just moves everything under filebeat to filebeatreceiver::filebeat section in otel. We wouldn't be limited by this logic as such. I rather like the idea that we don't have to write a full beatreceiver config which is quite verbose.

Agree partially with you. Yes, it is true that writting the full config is quite verbose, I also don't want to write a full otel config each time, but we have different integration test scenarios that I trully hope this test suite will be able to provide coverage, some of them:

  • Testing FoSB translation and internals using a standard beat config
  • Testing Beats Receivers using the beat config, without having to fully write an entire otel pipeline for basic tests.
  • Testing Beats Receivers using a full otel configuration for more complex test scenarios, such as custom exporter configuration, otel processors, multiple receivers/exporters, etc

@khushijain21 khushijain21 changed the title [beatreceiver] Add otel test binary [beatreceiver] Add otel test framework Jun 17, 2025
@mauri870
Copy link
Member
mauri870 commented Jun 18, 2025 •
edited
Loading

Hey @khushijain21, I don’t think you meant to close this, did you?

@khushijain21
Copy link
Contributor Author
khushijain21 commented Jun 19, 2025 •
edited
Loading

Hello @mauri870, yes I did mean to close it. We wouldn't need this PR after #44855

And as for your above points of support, the first two are supported. And for the last one - we can raise a PR for otel binary to accept a full otel config as well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@belimawr belimawr belimawr left review comments

@mauri870 mauri870 mauri870 left review comments

@leehinman leehinman leehinman left review comments

@rdner rdner Awaiting requested review from rdner rdner is a code owner automatically assigned from elastic/elastic-agent-data-plane

@cmacknz cmacknz Awaiting requested review from cmacknz

@VihasMakwana VihasMakwana Awaiting requested review from VihasMakwana

Assignees

@khushijain21 khushijain21

Labels
Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@khushijain21 @elasticmachine @mauri870 @belimawr @leehinman
0