8000 Compute entry leaders by haesbaert · Pull Request #40 · elastic/quark · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Compute entry leaders #40

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 1, 2024
Merged

Compute entry leaders #40

merged 1 commit into from
Jul 1, 2024

Conversation

haesbaert
Copy link
Collaborator
@haesbaert haesbaert commented Jun 7, 2024 •
edited
Loading 
commit 0ff5476ffc8fa9bda7a7349de39bea7a72ba229a (HEAD -> leader, origin/leader)
Author: Christiano Haesbaert <haesbaert@elastic.co>
Date:   Fri Jun 7 23:05:42 2024 +0200

    Compute entry leaders, off by default. Issue #41

    This computes entry_leader and entry_meta.type from ECS:
    https://www.elastic.co/guide/en/ecs/current/ecs-process.html#field-process-entry-meta-type

    In order to properly inherit both fields, we have to traverse the process tree
    "in order of creation", this means building a walking list from pid 1,
    descending to everything else. Cloud defend does this by ordering pids, which is
    not quite right as pids can wrap around, we handle it by traversing processes
    via their actual relationship.

    There is a bug that can be observed with kprobes and docker, basically docker
    forks and exits a bunch of intermediary processes before giving us our
    container, this causes reparenting of processes, meaning their ppid changes. The
    ppid we get is racy since it depends if the parent exited before fork was
    commited or after. The actual bug is that kprobes are not updating ppid after
    fork, this is further described in #43.

    As this is all very ECS specific, it is disabled by default and you must pass
    QQ_ENTRY_LEADER to enable the computation.

    The hooks in raw_event_to_quark_event() might not be completely right,
    recomputing for exec only, maybe we should recompute more.

    At any rate this seems to produce correct values for INIT, SSHD and docker (at least
    on eBPF).
    ```

@haesbaert
Compute entry leaders, off by default. Issue #41
0ff5476 
This computes entry_leader and entry_meta.type from ECS:
https://www.elastic.co/guide/en/ecs/current/ecs-process.html#field-process-entry-meta-type

In order to properly inherit both fields, we have to traverse the process tree
"in order of creation", this means building a walking list from pid 1,
descending to everything else. Cloud defend does this by ordering pids, which is
not quite right as pids can wrap around, we handle it by traversing processes
via their actual relationship.

There is a bug that can be observed with kprobes and docker, basically docker
forks and exits a bunch of intermediary processes before giving us our
container, this causes reparenting of processes, meaning their ppid changes. The
ppid we get is racy since it depends if the parent exited before fork was
commited or after. The actual bug is that kprobes are not updating ppid after
fork, this is further described in #43.

As this is all very ECS specific, it is disabled by default and you must pass
QQ_ENTRY_LEADER to enable the computation.

The hooks in raw_event_to_quark_event() might not be completely right,
recomputing for exec only, maybe we should recompute more.

At any rate this seems to produce correct values for INIT, SSHD and docker (at least
on eBPF).
@haesbaert haesbaert changed the title Hackish POC to show how entry_leader could be Compute entry leaders Jun 26, 2024
@haesbaert haesbaert marked this pull request as ready for review June 26, 2024 11:46
@haesbaert haesbaert merged commit 2995321 into main Jul 1, 2024
@haesbaert haesbaert deleted the leader branch October 15, 2024 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@haesbaert
0