Description
While creating a PR to write any lost events to stderr on tracee I came upon a strange phenomena - tracee-ebpf was getting lost events on the lost events channel - but the lost event count was 0.
I continued to test this with the guiding help of @yanivagman and we found that libbpf actually doesn't generate those lost events callback, and that they are generated immediately after tracee-ebpf is killed using ^C.
We verified that by printing the lost events after each callback in libbpf and on tracee and this is the result:
libbpf: Lost events - 176
lost 176 events
libbpf: Lost events - 269
lost 269 events
libbpf: Lost events - 408
lost 408 events
libbpf: Lost events - 72
lost 72 events
^Clibbpf: Lost events - 45
lost 45 events
libbpf: Lost events - 208
lost 208 events
lost 0 events
lost 0 events
lost 0 events
lost 0 events
lost 0 events
lost 0 events
lost 0 events
lost 0 events
lost 0 events
lost 0 events
lost 0 events
lost 0 events
lost 0 events
lost 0 events
lost 0 events
lost 0 events
...
The lost 0 events message carries for quite some time before tracee is killed.
It seems that the lost event callback on libbpfgo somehow gets zeroed values during tracee termination.
I tried to find the cause but was unsuccessful.
@grantseltzer maybe you have an idea?
In anyway I think it's good practice to check if the lost event cnt is greater than 0 before sending it to the channel.
WDYT?