Added flag to enforce syscall dependencies #6131
Conversation
…call dependencies Syzkaller often breaks dependencies across syscalls (e.g., due minimization, stochastic resource generation, and mutation) when generating programs, thus failing to build fuzzing inputs that exercise deep states in the target program. This patch addresses this issue by adding a Boolean (called PromoteDeps), which if set, certain measures are taken to enforce that syscall dependencies are respected in any generated program: Mutation does not break dependencies, Resource generation is no longer stochastic, and if a minimized program has broken dependencies, it gets discarded. To this end, I introduced two optional fields in the config file: promote_syscalls_dependency, and dynamic_promote_syscalls_dependency. The former flag is a boolean and when set to true will enable the PromoteDeps flag. The latter field, contains a time expressed in minutes (e.g, 30). Once the manager starts, it sets a timer with the value contained in this flag. Once the timer reaches the 0, the Boolean PromoteDep is switched (i.e., if it contained false it now contains true, and vice versa), and the timer starts again. This switch allows us to introduce more randomness in the generated programs achieving the best of both worlds.
|
Hi Nilo, How/when these tunables should be set? And when shouldn't they be set? Have you done any benchmarking? What are the results? Can this be enabled by default always? We specifically tried to avoid lots of obscure knobs that nobody knows how to set right. |
|
Hello Dmitry, Nice to meet you! Thanks for your message, let me answer to each one of your questions by topic: BenchmarksWe did experiments during our fuzzing campaigns here at Qualcomm. In particular, we ran syzkaller with and without the proposed patch for a week-time on three different drivers. We observed that by enabling the option "dynamic_promote_syscalls_dependency" and setting it to 10 (minutes), syzkaller covered ~47% functions against ~42% functions covered without the patch (this data was recorded by using GCOV), thus improving function coverage of, on average, ~5%. How/when to set the turnablesYou can enable the flag "dynamic_promote_syscalls_dependency=10" and leave it always enabled, that's what we do in our fuzzing campaigns. What that does is: 1) Run syzkaller with the flag "PromoteDeps" set to true for 10 minutes, and then 2) Run syzkaller with the flag set to False for 10 minutes, and 3) repeat. In phase 1 Syzkaller generates fuzzing programs that do not contain any broken dependencies, thus triggering deeper states in the fuzzed driver. During phase 2, Syzkaller behaves normally and it does not enforce any dependency. Phase 2 allows syzkaller to use the previous generated programs in phase 1 and apply mutation, etc.. By setting the "PromoteDeps" on and off we are able to generate a more complex and thorough fuzzing corpus. The knob could be set by default and never touched again. There are not really any cases in which the knob should not be set. All this said, I'd be happy to change/update the patch, if you think there is a better design to fix the issue of broken dependencies. Please let me know of any other questions/feedback. |
Issue
Syzkaller often breaks dependencies across syscalls (expressed through the use of resources in Syzkaller programs). And because of this, Syzkaller struggles to build fuzzing inputs (i.e., programs) that would exercise deeper paths in the drivers. Syzkaller breaks syscalls dependencies because: 1) fuzzing mutation might remove random calls (and resources), 2) the resource generation itself is stochastic -- with a certain probability P, Syzkaller purposefully disregards syscall dependencies in an att 8000 empt to increase randomness, and 3) program minimization -- Syzkaller minimizes the programs that it generates, as they are initially quite big.
Patch
The patch addresses all these issues by adding a Boolean (called PromoteDep) in the validation module of Syzkaller. If the Boolean is set to true, certain measures are taken to enforce that IOCTL dependencies are respected in any generated program. In particular, Mutation does not break dependencies, Resource generation is no longer stochastic, and dependencies across syscalls are never disregarded, If a minimized program has broken dependencies, it gets discarded. The patch allows for different ways to enable the PromoteDep flag: 1) The configuration flag promote_syscalls_dependency. If enabled Syzkaller is instructed to not break dependencies, and 2) The flag dynamic_promote_syscalls_dependency. This flag contains a time expressed in minutes (e.g, 30). Once the manager starts it sets a timer with the value contained in this flag. Once the timer reaches the 0, the Boolean PromoteDep is switched (i.e., if it contained false it now contains true, and vice versa), and the timer starts again. This switch allows us to introduce more randomness in the generated programs.
All flags are optional, and they don't have to be set.