[FEATURE][FIX] New capabilities with rings (replacing old capabilities feature) #2243

rafaeldtinoco · 2022-10-18T03:45:53Z

Initial Checklist

There is an issue describing the need for this PR.
Git log contains summary of the change.
Git log contains motivation and context of the change.

Description (git log)

Reduce tracee capabilities during its execution. Do it through
"protection rings":

Privileged (ring 0): all capabilities are Effective (almost never)
Required (ring 1): only required (by events) capabilities are Effective (during config)
Requested (ring 2): single time requested capabilities are Effective (special needs)
Unprivileged (ring 3): no capabilities are Effective at all (most of the time)

This way, tracee will be most of its time (99%) without any Effective
capability (Unprivileged ring). The Required() ring will set
capabilities needed by Events as Effective until Unprivileged ring is
set.

This change is needed to allow tracee to raise its capabilities for
certain operations (like mounting/umounting cgroupv2 filesystems, for
example).

Documentation provided at docs/deep-drive/capabilities.md.

Fixes: #2242

Type of change

Bug fix (non-breaking change fixing an issue, preferable).
Quick fix (minor non-breaking change requiring no issue, use with care)
Code refactor (code improvement and/or code removal)
New feature (non-breaking change adding functionality).
Breaking change (cause existing functionality not to work as expected).

How Has This Been Tested?

Multiple executions, in different environments and kernel versions, with command lines similar to:

sudo ./dist/tracee-ebpf --containers --output format:json --output option:parse-arguments --output option:detect-syscall --trace container --trace comm=ls --capabilities add="cap_kill" --capabilities add="cap_syslog" --capabilities add="cap_lease" --capabilities drop="cap_bpf"

You may add, remove or skip capabilities dropping feature. By not enabling the cmdline, tracee successfully drops the capabilities as needed.

Final Checklist:

Pick "Bug Fix" or "Feature", delete the other and mark appropriate checks.

I have made corresponding changes to the documentation.
My code follows the style guidelines (C and Go) of this project.
I have performed a self-review of my own code.
I have commented all functions/methods created explaining what they do.
I have commented my code, particularly in hard-to-understand areas.
My changes generate no new warnings.
I have added tests that prove my fix, or feature, is effective.
New and existing unit tests pass locally with my changes.
Any dependent changes have been merged and published before.

Git Log Checklist:

My commits logs have:

Subject starts with "subsystem|file: description".
Do not end the subject line with a period.
Limit the subject line to 50 characters.
Separate subject from body with a blank line.
Use the imperative mood in the subject line.
Wrap the body at 72 characters.
Use the body to explain what and why instead of how.

rafaeldtinoco · 2022-10-18T04:16:20Z

There are some opportunistic style/format fixes/improvements together, but I won't split into new commits just because of that (I believe the opportunistic refactoring is a good practice and want to use more in my PRs).

rafaeldtinoco · 2022-10-18T04:21:05Z

@yanivagman per our recent discussion, this PR will unblock:

#2233

which will be capable of mounting/unmounting cgroupv1/cgroupv2 during startup/exit.

And then unblock:

#2200

it fails because of capabilities bug
it needs cgroupv1/v2 mounting/unmounting

josedonizetti

@rafaeldtinoco Didn't review it yet but took a quick pick, and something popped into my mind, let me know what you think. I really would prefer that moving-in/moving-out from a ring would be done in a block, without the need to remember to run unprivileged again.

Something like:

c.WithCaps(caps.SYSLOG, func() {
 //do your thing
})

c.Privileged(func() {
 // do your thing
})

Those would execute the code and leave the ring as soon as the func returns. Instead of

// c.Require(cap.SYS_ADMIN)
// do stuff
// c.Unprivileged()

wdyt?

cmd/tracee-ebpf/flags/capabilities.go

rafaeldtinoco · 2022-10-18T23:47:12Z

@rafaeldtinoco Didn't review it yet but took a quick pick, and something popped into my mind, let me know what you think. I really would prefer that moving-in/moving-out from a ring would be done in a block, without the need to remember to run unprivileged again.

I can give a try, lets see...

rafaeldtinoco · 2022-10-19T00:20:53Z

Yep, I like it:

I'll make that change @josedonizetti

pkg/capabilities/capabilities.go

cmd/tracee-ebpf/flags/capabilities_test.go

pkg/capabilities/capabilities.go

pkg/ebpf/tracee.go

rafaeldtinoco · 2022-10-20T02:22:19Z

Thanks for the review @NDStrahilevitz, I have made the changes. I thought we should merge this only next week (after this week's "special release" we are going to make with changes arriving tomorrow) BUT it occurred to me that, since we are currently broken regarding capabilities, maybe we should merge this together (since this is also a fix).

WDYT ? (@grantseltzer FYI since you're coordinating this "special" release)

Note, I have also improved debugging:

$ sudo TRACEE_LOGGER_LVL=debug ./dist/tracee-ebpf --containers --output format:json --output option:parse-arguments --output option:detect-syscall --trace container --trace comm=ls --capabilities add="cap_kill" --capabilities add="cap_lease,,," --capabilities drop="cap_syslog" --capabilities bypass=0
{"level":"debug","ts":1666239284.002444,"msg":"osinfo","OSReleaseField":"ARCH","OS_KERNEL_RELEASE":"x86_64"}
{"level":"debug","ts":1666239284.0024936,"msg":"osinfo","OSReleaseField":"ID","OS_KERNEL_RELEASE":"manjaro"}
{"level":"debug","ts":1666239284.0025043,"msg":"osinfo","OSReleaseField":"ID_LIKE","OS_KERNEL_RELEASE":"arch"}
{"level":"debug","ts":1666239284.0025103,"msg":"osinfo","OSReleaseField":"BUILD_ID","OS_KERNEL_RELEASE":"rolling"}
{"level":"debug","ts":1666239284.0025146,"msg":"osinfo","OSReleaseField":"PRETTY_NAME","OS_KERNEL_RELEASE":"\"Manjaro Linux\""}
{"level":"debug","ts":1666239284.0025272,"msg":"osinfo","OSReleaseField":"KERNEL_RELEASE","OS_KERNEL_RELEASE":"5.15.71-1-MANJARO"}
{"level":"debug","ts":1666239284.0029862,"msg":"osinfo","security_lockdown":"none"}
{"level":"debug","ts":1666239284.0181413,"msg":"capabilities change","pkg":"capabilities"}
{"level":"debug","ts":1666239284.0184891,"msg":"capabilities change","pkg":"capabilities"}
{"level":"debug","ts":1666239284.0185218,"msg":"enabling","pkg":"capabilities","cap":"cap_syslog"}
{"level":"debug","ts":1666239284.018633,"msg":"capabilities change","pkg":"capabilities"}
{"level":"debug","ts":1666239284.250974,"msg":"capabilities change","pkg":"capabilities"}
{"level":"debug","ts":1666239284.251018,"msg":"enabling","pkg":"capabilities","cap":"cap_bpf"}
{"level":"debug","ts":1666239284.251027,"msg":"enabling","pkg":"capabilities","cap":"cap_sys_resource"}
{"level":"debug","ts":1666239284.2510338,"msg":"enabling","pkg":"capabilities","cap":"cap_perfmon"}
{"level":"debug","ts":1666239284.2510386,"msg":"enabling","pkg":"capabilities","cap":"cap_ipc_lock"}
{"level":"debug","ts":1666239284.2510436,"msg":"enabling","pkg":"capabilities","cap":"cap_kill"}
{"level":"debug","ts":1666239284.2510478,"msg":"enabling","pkg":"capabilities","cap":"cap_lease"}
{"level":"debug","ts":1666239285.246613,"msg":"capabilities change","pkg":"capabilities"}

So it is very clear which capabilities are being enabled each time they are "applied".

rafaeldtinoco · 2022-10-20T17:46:07Z

BTW, test showing fork()/exec() Without any privileges after bound dropping (independently of inheritable):

docs/deep-dive/capabilities.md

cmd/tracee-ebpf/main.go

pkg/capabilities/capabilities.go

rafaeldtinoco · 2022-10-21T01:57:29Z

I'll rebase and address comments from @yanivagman tomorrow, so we can merge this before the next release (if others agree).

josedonizetti

@rafaeldtinoco when booting tracee without any capabilities (not using sudo), we get a segmentation error that is misleading:

./dist/tracee-ebpf 
TIME             UID    COMM             PID     TID     RET              EVENT                ARGS
fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0xffffffc0 pc=0x7fb55469455b]

runtime stack:
runtime.throw({0x171a7f5?, 0x0?})
	/snap/go/9951/src/runtime/panic.go:1047 +0x5d fp=0x7fb5067fb8c8 sp=0x7fb5067fb898 pc=0x94ce1d
runtime.sigpanic()
	/snap/go/9951/src/runtime/signal_unix.go:819 +0x369 fp=0x7fb5067fb918 sp=0x7fb5067fb8c8 pc=0x963e49
...

can we double check tracee is getting called with enough caps, and if not return a nicer error?

rafaeldtinoco · 2022-10-21T18:03:53Z

can we double check tracee is getting called with enough caps, and if not return a nicer error?

$ TRACEE_LOGGER_LVL=debug ./dist/tracee-ebpf --containers --output format:json --output option:parse-arguments --output option:detect-syscall --trace container --trace comm=ls --capabilities add="cap_kill" --capabilities add="cap_lease,,," --capabilities drop="cap_syslog" --capabilities bypass=0
{"level":"debug","ts":1666375394.1403742,"msg":"osinfo","OSReleaseField":"ID","OS_KERNEL_RELEASE":"manjaro"}
{"level":"debug","ts":1666375394.1404386,"msg":"osinfo","OSReleaseField":"ID_LIKE","OS_KERNEL_RELEASE":"arch"}
{"level":"debug","ts":1666375394.1404557,"msg":"osinfo","OSReleaseField":"BUILD_ID","OS_KERNEL_RELEASE":"rolling"}
{"level":"debug","ts":1666375394.1404705,"msg":"osinfo","OSReleaseField":"PRETTY_NAME","OS_KERNEL_RELEASE":"\"Manjaro Linux\""}
{"level":"debug","ts":1666375394.1404862,"msg":"osinfo","OSReleaseField":"KERNEL_RELEASE","OS_KERNEL_RELEASE":"5.15.74-3-MANJARO"}
{"level":"debug","ts":1666375394.1405168,"msg":"osinfo","OSReleaseField":"ARCH","OS_KERNEL_RELEASE":"x86_64"}
{"level":"debug","ts":1666375394.1415193,"msg":"osinfo","security_lockdown":"none"}
{"level":"debug","ts":1666375394.1503236,"msg":"capabilities change","pkg":"capabilities"}
{"level":"debug","ts":1666375394.1505644,"msg":"capabilities change","pkg":"capabilities"}
{"level":"debug","ts":1666375394.150603,"msg":"enabling","pkg":"capabilities","cap":"cap_syslog"}
{"level":"debug","ts":1666375394.150662,"msg":"capabilities change","pkg":"capabilities"}
{"level":"debug","ts":1666375394.3082733,"msg":"capabilities change","pkg":"capabilities"}
{"level":"debug","ts":1666375394.3083105,"msg":"enabling","pkg":"capabilities","cap":"cap_ipc_lock"}
{"level":"debug","ts":1666375394.3083243,"msg":"enabling","pkg":"capabilities","cap":"cap_sys_resource"}
{"level":"debug","ts":1666375394.3083296,"msg":"enabling","pkg":"capabilities","cap":"cap_lease"}
{"level":"debug","ts":1666375394.308336,"msg":"enabling","pkg":"capabilities","cap":"cap_sys_admin"}
{"level":"debug","ts":1666375394.3083417,"msg":"enabling","pkg":"capabilities","cap":"cap_kill"}
libbpf: map 'fd_arg_path_map': failed to create: Operation not permitted(-1)
libbpf: failed to load object 'embedded-core'
{"level":"debug","ts":1666375394.3091176,"msg":"capabilities change","pkg":"capabilities"}
{"level":"fatal","ts":1666375394.3094344,"msg":"app","error":"error initializing Tracee: failed to load BPF object: operation not permitted"}

or simply:

$ ./dist/tracee-ebpf
TIME             UID    COMM             PID     TID     RET              EVENT                ARGS
libbpf: map 'fd_arg_path_map': failed to create: Operation not permitted(-1)
libbpf: failed to load object 'embedded-core'

End of events stream
Stats: {EventCount:0 EventsFiltered:0 NetEvCount:0 ErrorCount:0 LostEvCount:0 LostWrCount:0 LostNtCount:0}
{"level":"fatal","ts":1666375513.650924,"msg":"app","error":"error initializing Tracee: failed to load BPF object: operation not permitted"}

Trying to reproduce into multiple envs and not able to....

rafaeldtinoco · 2022-10-21T18:30:19Z

Got it:

@josedonizetti

Reduce tracee capabilities during its execution. Do it through "protection rings": * Privileged (ring 0): all capabilities are Effective (almost never) * Required (ring 1): only required (by events) capabilities are Effective (during config) * Requested (ring 2): single time requested capabilities are Effective (special needs) * Unprivileged (ring 3): no capabilities are Effective at all (most of the time) This way, tracee will be most of its time (99%) without any Effective capability (Unprivileged ring). The Required() ring will set capabilities needed by Events as Effective until Unprivileged ring is set. This change is needed to allow tracee to raise its capabilities for certain operations (like mounting/umounting cgroupv2 filesystems, for example). Documentation provided at docs/deep-drive/capabilities.md. It also Implements @josedonizetti's idea of executing protected code within an execution block instead of manually changing protection ring back to unprivileged at the end of each needed code.

rafaeldtinoco · 2022-10-21T19:22:49Z

@josedonizetti and I paired remotely to close this for the release (and tested together in multiple envs).

This was referenced Oct 18, 2022

[REFACTOR] Cgroup Interface (cgroupv1 and cgroupv2 initialization) #2233

Merged

[FEATURE] New network code with tests #2200

Merged

rafaeldtinoco changed the title ~~[FEATURE] New capabilities with rings (replacing old capabilities feature)~~ [FEATURE][FIX] New capabilities with rings (replacing old capabilities feature) Oct 18, 2022

rafaeldtinoco self-assigned this Oct 18, 2022

rafaeldtinoco added this to the v0.9.0-rc1 milestone Oct 18, 2022

rafaeldtinoco added the area/network label Oct 18, 2022

rafaeldtinoco requested review from NDStrahilevitz, yanivagman and josedonizetti October 18, 2022 03:51

rafaeldtinoco marked this pull request as ready for review October 18, 2022 04:12

This comment was marked as outdated.

Sign in to 8000 view

This comment was marked as outdated.

Sign in to view

josedonizetti reviewed Oct 18, 2022

View reviewed changes

cmd/tracee-ebpf/flags/capabilities.go Outdated Show resolved Hide resolved

rafaeldtinoco requested a review from josedonizetti October 19, 2022 01:55

NDStrahilevitz reviewed Oct 19, 2022

View reviewed changes

rafaeldtinoco mentioned this pull request Oct 20, 2022

Capabilities: create unit tests for the new capabilities code #2258

Open

yanivagman reviewed Oct 20, 2022

View reviewed changes

capabilities: remove old capabilities implementation

1126417

josedonizetti reviewed Oct 21, 2022

View reviewed changes

josedonizetti approved these changes Oct 21, 2022

View reviewed changes

rafaeldtinoco merged commit ec83c00 into aquasecurity:main Oct 21, 2022

rafaeldtinoco deleted the new-capabilities-with-rings branch October 21, 2022 19:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[FEATURE][FIX] New capabilities with rings (replacing old capabilities feature) #2243

[FEATURE][FIX] New capabilities with rings (replacing old capabilities feature) #2243

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

[FEATURE][FIX] New capabilities with rings (replacing old capabilities feature) #2243

[FEATURE][FIX] New capabilities with rings (replacing old capabilities feature) #2243

Uh oh!

Conversation

Uh oh!

Initial Checklist

Description (git log)

Type of change

How Has This Been Tested?

Final Checklist:

Git Log Checklist:

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!