8000 Remove socket filter by grcevski · Pull Request #394 · grafana/beyla · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Remove socket filter #394

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Oct 31, 2023
Merged

Remove socket filter #394

merged 14 commits into from
Oct 31, 2023

Conversation

grcevski
Copy link
Contributor
@grcevski grcevski commented Oct 30, 2023

This PR changes how we process kprobe based events to use tcp_sendmsg and tcp_recvmsg instead of the socket filter. I had initially added the probes for tcp_recvmsg/sendmsg to capture the traceparent header field, so it makes sense to actually use these now instead of having duplicate logic with the socket filter.

The PR also has some minor tweaks:

  • I added an option to help us turn off BPF based PID filtering, so that we always rely on the user space filter. This can be helpful during debugging.
  • I started tracking the response sizes in the socket filter, since this is now super easy to do. No code yet to report this.
  • I noticed that our earlier for traceparent capture didn't work with large requests because of how the kernels handle message splitting, so I added some helpers to test large requests in our integration tests.
  • NodeJS clients keep client ports open and recycle only when needed. I added some code so that Beyla ignores ephemeral ports in the ports scanning, so that we don't cause memory growth in Beyla if the watched application is leaking ports.

TODO:

  • Add tests for large requests for Node and Rust, because they show diversity in how they handle the recvmsg struct in the kernel.
  • Manual test with kernels 5.14 and 6.5. Note to self: We need to add VM based tests.

@grcevski grcevski requested a review from mariomac as a code owner October 30, 2023 16:22

bpf_map_update_elem(&active_ssl_handshakes, &id, &s, BPF_ANY);
// Important: We must work here to remember the iovec pointer, since the msghdr structure
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Saving the msghdr pointer didn't work with requests that are split in few parts, i.e. large requests > 4K in size. Therefore, here I save now the iovec_ptr before we end up receiving the message, so we can read it properly when the first 4K chunk is processed.

@@ -0,0 +1,10 @@
with open('data.json', 'w') as f:
f.write('''{
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a small helper program to generate large requests. The earlier version of large_data was manually copy pasted duplicate keys, which prevented me from testing large response sizes, the JSON responder would remove the duplicates :).

@codecov-commenter
Copy link
codecov-commenter commented Oct 30, 2023

Codecov Report

Attention: 7 lines in your changes are missing coverage. Please review.

Comparison is base (949c0ff) 81.10% compared to head (5f729af) 42.87%.

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #394       +/-   ##
===========================================
- Coverage   81.10%   42.87%   -38.23%     
===========================================
  Files          55       53        -2     
  Lines        4318     4191      -127     
===========================================
- Hits         3502     1797     -1705     
- Misses        634     2284     +1650     
+ Partials      182      110       -72     
Flag Coverage Δ
integration-test ?
unittests 42.87% <0.00%> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
pkg/internal/discover/services/criteria.go 63.38% <ø> (-4.23%) ⬇️
pkg/internal/discover/watcher.go 64.00% <0.00%> (-26.91%) ⬇️
pkg/internal/ebpf/httpfltr/httpfltr.go 34.28% <0.00%> (-53.08%) ⬇️

... and 39 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor
@mariomac mariomac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is amazing!

Feel free to merge when the integration tests are fixed

@@ -258,7 +251,7 @@ func (p *Tracer) UProbes() map[string]map[string]ebpfcommon.FunctionPrograms {
}

func (p *Tracer) SocketFilters() []*ebpf.Program {
return []*ebpf.Program{p.bpfObjects.SocketHttpFilter}
return nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we can now remove the SocketFilter method from the ebpf.Tracer interface and the logic that invokes it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah good point!

@mariomac
Copy link
Contributor

Please can we merge first this PR to make sure the DaemonSet scenario still keeps working? #399

@grcevski
Copy link
Contributor Author

Sure! let me resolve the conflicts.

@grcevski grcevski merged commit 324c8d2 into grafana:main Oct 31, 2023
@grcevski grcevski deleted the remove_socket_filter branch October 31, 2023 16:13
@grcevski
Copy link
Contributor Author

Thanks Mario! it's merged now

@mariomac
Copy link
Contributor

Ouch! I meant to merge first the DaemonSet test 😅 before the current PR (my message writing was too ambiguous). I'll rerun the tests to make sure DaemonSet still works

@grcevski
Copy link
Contributor Author

No worries, I can always revert the changes related to pid matching and we know we need something better anyway, which I'm working on next.

8000
@mariomac
Copy link
Contributor

No need to, the tests are passing: #399

mattdurham pushed a commit to mattdurham/beyla that referenced this pull request Jan 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0