8000 scripts: add script to sync kernel changes by anakryiko · Pull Request #13 · libbpf/libbpf · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

scripts: add script to sync kernel changes #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 17, 2019

Conversation

anakryiko
Copy link
Member
@anakryiko anakryiko commented Feb 16, 2019

This script automates the process of applying libbpf-relevant changes
from kernel repository on top of current state of libbpf repository.

It uses CHECKPOINT-COMMIT file to keep track of last commit in kernel
repo up to which libbpf is in sync with. If there are any new libbpf
changes in kernel repository, script extracts them, preserving original
commit metadata. It also creates a "sync commit" using cover letter as
a template, which nicely summarizes changes since last sync with kernel.

Usage: ./scripts/sync-kernel.sh <linux-repo> <libbpf-repo>

If it succeeds, script will create a bunch of local commits in
in separate branch, which can be easily pushed into github
to create a pull request.

Script tries to clean up after itself, except in case of failure. But it
doesn't clean up timestamped branches it creates in both kernel and
libbpf repositories for now. We can add that later.

Signed-off-by: Andrii Nakryiko andriin@fb.com

echo "PATCHSET: /tmp/libbpf-${SUFFIX}.patchset"

cd ${LIBBPF_REPO}
git co -b libbpf-sync-${SUFFIX}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this is git checkout?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yes, it's my local alias. I'm so used to it that didn't even realize it's not standard feature, will fix.

@yonghong-song
Copy link
Contributor

Thanks for implementation. This is in the right direction for automation!

I tried to add a few artificial patches in bpf-next and got the following failures:

Ref 'refs/heads/libbpf-2019-02-17T01-17-25.489Z' was rewritten
Rewrite a63440df54bf4159b9389a70450579f990095c29 (3/3) (0 seconds passed, remaining 0 predicted)    
Ref 'refs/heads/libbpf-2019-02-17T01-17-25.489Z' was rewritten
/tmp/libbpf-2019-02-17T01-17-25.489Z.patches/0000-cover-letter.patch
/tmp/libbpf-2019-02-17T01-17-25.489Z.patches/0001-fix1.patch
/tmp/libbpf-2019-02-17T01-17-25.489Z.patches/0002-fix-3.patch
[libbpf-2019-02-17T01-17-25.489Z b5ea6613c83e] sync: latest libbpf changes from kernel
 1 file changed, 1 insertion(+)
 create mode 100644 CHECKPOINT-COMMIT
PATCHSET: /tmp/libbpf-2019-02-17T01-17-25.489Z.patchset
Switched to a new branch 'libbpf-sync-2019-02-17T01-17-25.489Z'
Applying: fix1
error: patch failed: src/btf.h:4
error: src/btf.h: patch does not apply
Patch failed at 0001 fix1
Use 'git am --show-current-patch' to see the failed patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
-bash-4.4$

The command line I tried:

-bash-4.4$ ./scripts/sync-kernel.sh /home/yhs/work/net-next .

in the libbpf repo directory.

@anakryiko
Copy link
Member Author

Can you publish full output and patches you applied on top of bpf-next, I'll try to repro locally. Also what was HEAD of libbpf and 8000 kernel?
This happened to me only when master on libbpf and linux repos were out of sync (thus patches didn't apply clean). When I was testing I was even able to re-apply quite a long history of commits on top of some older commits in libbpf, but the trickiest part was correlating commits in linux and libbpf.

@yonghong-song
Copy link
Contributor

The kernel part:

commit 6bcf3db00a5931ccbd78730d85ab6124f826d9ba (HEAD -> copy)
Author: Yonghong Song <yhs@fb.com>
Date:   Sat Feb 16 17:09:37 2019 -0800

    patch3
    
    Signed-off-by: Yonghong Song <yhs@fb.com>

commit add7beebf02a190ccd74ac4a221105090408e9e9
Author: Yonghong Song <yhs@fb.com>
Date:   Sat Feb 16 17:09:03 2019 -0800

    fix 3
    
    Signed-off-by: Yonghong Song <yhs@fb.com>

commit 0ccb09e4c26f3d0189f689b3596f6c4b6655a35a
Author: Yonghong Song <yhs@fb.com>
Date:   Sat Feb 16 17:08:33 2019 -0800

    fix1
    
    Signed-off-by: Yonghong Song <yhs@fb.com>

commit b251f9f63a3bea7864bf627960926d978e97814d (bpf-next/master, bpf-next2)
Author: Peter Oskolkov <posk@google.com>
Date:   Fri Feb 15 09:51:35 2019 -0800

    bpf: make LWTUNNEL_BPF dependent on INET

Individual commits:

-bash-4.4$ git show 6bcf3db00a5931ccbd78730d85ab6124f826d9ba
commit 6bcf3db00a5931ccbd78730d85ab6124f826d9ba (HEAD -> copy)
Author: Yonghong Song <yhs@fb.com>
Date:   Sat Feb 16 17:09:37 2019 -0800

    patch3
    
    Signed-off-by: Yonghong Song <yhs@fb.com>

diff --git a/t.c b/t.c
new file mode 100644
index 000000000000..f349bb94c37e
--- /dev/null
+++ b/t.c
@@ -0,0 +1 @@
+iinwe filees
-bash-4.4$ git show add7beebf02a190ccd74ac4a221105090408e9e9
commit add7beebf02a190ccd74ac4a221105090408e9e9
Author: Yonghong Song <yhs@fb.com>
Date:   Sat Feb 16 17:09:03 2019 -0800

    fix 3
    
    Signed-off-by: Yonghong Song <yhs@fb.com>

diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index ade1c32fb083..e96428c9a47b 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -13,6 +13,8 @@
 #include "libbpf.h"
 #include "libbpf_util.h"
 
+#error no
+
 #define max(a, b) ((a) > (b) ? (a) : (b))
 #define min(a, b) ((a) < (b) ? (a) : (b))
 
-bash-4.4$ git show 0ccb09e4c26f3d0189f689b3596f6c4b6655a35a
commit 0ccb09e4c26f3d0189f689b3596f6c4b6655a35a
Author: Yonghong Song <yhs@fb.com>
Date:   Sat Feb 16 17:08:33 2019 -0800

    fix1
    
    Signed-off-by: Yonghong Song <yhs@fb.com>

diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h
index 94bbc249b0f1..c57d42385806 100644
--- a/tools/lib/bpf/btf.h
+++ b/tools/lib/bpf/btf.h
@@ -4,6 +4,8 @@
 #ifndef __LIBBPF_BTF_H
 #define __LIBBPF_BTF_H
 
+#error no
+
 #include <linux/types.h>
 
 #ifdef __cplusplus
-bash-4.4$

@yonghong-song
Copy link
Contributor

Next time I will remember to go to a repo I can push to github.

@anakryiko
Copy link
Member Author

What was sha on libbpf side?

@yonghong-song
Copy link
Contributor

libbpf side

commit 8521d84455f7bfd17b088ea642767e370939185a (HEAD -> script)
Author: Andrii Nakryiko <andriin@fb.com>
Date:   Sat Feb 16 14:57:53 2019 -0800

    scripts: add script to sync kernel changes
    
    This script automates the process of applying libbpf-relevant changes
    from kernel repository on top of current state of libbpf repository.
    
    It uses CHECKPOINT-COMMIT file to keep track of last commit in kernel
    repo up to which libbpf is in sync with. If there are any new libbpf
    changes in kernel repository, script extracts them, preserving original
    commit metadata. It also creates a "sync commit" using cover letter as
    a template, which nicely summarizes changes since last sync with kernel.
    
    Usage: ./scripts/sync-kernel.sh <linux-repo> <libbpf-repo>
    
    If it succeeds, script will create a bunch of local commits in
    <libbpf-repo> in separate branch, which can be easily pushed into github
    to create a pull request.
    
    Script tries to clean up after itself, except in case of failure. But it
    doesn't clean up timestamped branches it creates in both kernel and
    libbpf repositories for now. We can add that later.
    
    Signed-off-by: Andrii Nakryiko <andriin@fb.com>

commit f0bcba631dec4540fc6ab2cd0a0923a111cf4cf2
Author: yonghong-song <ys114321@gmail.com>
Date:   Thu Feb 7 21:58:07 2019 -0800

    sync with latest bpf-next (#10)
    
    sync with latest bpf-next. tested with fb internal testcase and bcc.
    
    Signed-off-by: Yonghong Song <yhs@fb.com>

commit b19c6dcf623a7adc9e538ddbe2964c2f58dd2417
Author: Yonghong Song <yhs@fb.com>
Date:   Mon Feb 4 19:42:31 2019 -0800

    sync with bpf-next
    
    Sync with the following bug fix:
      commit a8a1f7d09cfc7e18874786c7634c9e71384fcd4e (HEAD -> bpf-next2, bpf-next/master)
      Author: Stanislav Fomichev <sdf@google.com>
      Date:   Mon Feb 4 16:20:55 2019 -0800
    
        libbpf: fix libbpf_print
    
        With the recent print rework we now have the following problem:
        pr_{warning,info,debug} expand to __pr which calls libbpf_print.
        libbpf_print does va_start and calls __libbpf_pr with va_list argument.
        In __base_pr we again do va_start. Because the next argument is a
        va_list, we don't get correct pointer to the argument (and print noting
        in my case, I don't know why it doesn't crash tbh).
    
        ......
    
        Signed-off-by: Stanislav Fomichev <sdf@google.com>
    
    Signed-off-by: Yonghong Song <yhs@fb.com>

@anakryiko
Copy link
Member Author

ah, I get it, it's the logic of updating CHECKPOINT-COMMIT file, it never exists in linux repo, I have a simpler idea on how to handle that. Thanks for catching this!

@anakryiko
Copy link
Member Author

ahah, so there are two problems:

  1. CHECKPOINT-COMMIT should be updated/commited in libbpf repo, not in linux repo (adding it in linux makes it harder to generate correct patch).
  2. You used "." as libbpf-repo, script doesn't handle that properly. Try with absolute path for now, I'll think how to properly fix relative paths meanwhile.

I fixed first in updated pull request. Will fix second soon.

BTW, if you are going to try again, keep in mind that you might need to do "git am --abort" in linux repo, and checkout your old HEAD (copy branch).

This script automates the process of applying libbpf-relevant changes
from kernel repository on top of current state of libbpf repository.

It uses CHECKPOINT-COMMIT file to keep track of last commit in kernel
repo up to which libbpf is in sync with. If there are any new libbpf
changes in kernel repository, script extracts them, preserving original
commit metadata. It also creates a "sync commit" using cover letter as
a template, which nicely summarizes changes since last sync with kernel.

Usage: ./scripts/sync-kernel.sh <linux-repo> <libbpf-repo>

If it succeeds, script will create a bunch of local commits in
<libbpf-repo> in separate branch, which can be easily pushed into github
to create a pull request.

Script tries to clean up after itself, except in case of failure. But it
doesn't clean up timestamped branches it creates in both kernel and
libbpf repositories for now. We can add that later.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
@anakryiko
Copy link
Member Author

Ok, fixed second problem as well by switching back to original working dir and then doing cd LIBBPF_REPO. Now works as expected:

Ref 'refs/heads/libbpf-2019-02-17T02-25-37.047Z' was rewritten
Rewrite 146c3786b73a6b20c524feaebd7196203dea5533 (3/3) (0 seconds passed, remaining 0 predicted)
Ref 'refs/heads/libbpf-2019-02-17T02-25-37.047Z' was rewritten
/tmp/libbpf-2019-02-17T02-25-37.047Z.patches/0000-cover-letter.patch
/tmp/libbpf-2019-02-17T02-25-37.047Z.patches/0001-fix-1.patch
/tmp/libbpf-2019-02-17T02-25-37.047Z.patches/0002-fix-3.patch
PATCHSET: /tmp/libbpf-2019-02-17T02-25-37.047Z.patchset
Switched to a new branch 'libbpf-sync-2019-02-17T02-25-37.047Z'
Applying: fix 1
Applying: fix 3
[libbpf-sync-2019-02-17T02-25-37.047Z 022af37] sync: latest libbpf changes from kernel
 1 file changed, 1 insertion(+), 1 deletion(-)
Success! 2 commits applied.

State after running script:

commit 022af3793929ce6ab4abd1f178b4b67e03039fba (HEAD -> libbpf-sync-2019-02-17T02-25-37.047Z)
Author: Andrii Nakryiko <andriin@fb.com>
Date:   Sat Feb 16 18:26:12 2019 -0800

    sync: latest libbpf changes from kernel

    Syncing latest libbpf commits from kernel repository.
    Baseline commit:   789f6bab849e04ea029c09b81dc8401dc0268cf9
    Checkpoint commit: 6911e801fa1da0e54b200af8df6908b9fc1c391d

    Andrii Nakryiko (2):
      fix 1
      fix 3

     src/btf.c | 2 ++
     src/btf.h | 2 ++
     2 files changed, 4 insertions(+)

    --
    2.17.1

commit 7825e77a9f04bd272b6aa6e5bf6ffdd239361da5
Author: Andrii Nakryiko <andriin@fb.com>
Date:   Sat Feb 16 17:44:23 2019 -0800

    fix 3

commit 480c282161177b66931fe5175e88115f7394f791
Author: Andrii Nakryiko <andriin@fb.com>
Date:   Sat Feb 16 17:41:06 2019 -0800

    fix 1

commit 88d35a3d80b52017bba32ecb3a4a9d412ac45a9d (kernel-sync-script)
Author: Andrii Nakryiko <andriin@fb.com>
Date:   Sat Feb 16 14:57:53 2019 -0800

    scripts: add script to sync kernel changes

<SNIP>

git rebase --onto ${SQUASHED_BASELINE_COMMIT} ${BASELINE_COMMIT} libbpf-${SUFFIX}

# Move all libbpf files into libbpf-projection directory
git filter-branch --prune-empty -f --tree-filter ' \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

didn't know such git command exist :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's a "with great power comes great responsibility" type of command :)

@anakryiko
Copy link
Member Author

@yonghong-song, you can check how sync commit looks like here: https://github.com/libbpf/libbpf/compare/master...anakryiko:libbpf-sync-2019-02-17T02-58-38.109Z?expand=1

I'm not yet creating that pull request, because we should probably land this one first and then I'll rebase (otherwise we'll have conflicts on CHECKPOINT-COMMIT file).

@yonghong-song
Copy link
Contributor

I got a failure like below:

-bash-4.4$ ./scripts/sync-kernel.sh /home/yhs/work/net-next /home/yhs/work/libbpf
WORKDIR:     /home/yhs/work/libbpf
LINUX REPO:  /home/yhs/work/net-next
LIBBPF REPO: /home/yhs/work/libbpf
SUFFIX: 2019-02-17T06-04-31.687Z
BASELINE COMMIT: 789f6bab849e04ea029c09b81dc8401dc0268cf9
CHECKPOINT COMMIT: 6bcf3db00a5931ccbd78730d85ab6124f826d9ba
SQUASHED BASELINE COMMIT: 81818e2fe001f6e7b23d01ef4e3d9c38655ecfa8
It looks like 'git am' is in progress. Cannot rebase.
-bash-4.4$

Any idea what's the issue? I have the same bpf-next hack earlier.

@anakryiko
Copy link
Member Author

Yeah, due to that initial bug, your linux repository now has git am in progress. Just execute git am --abort && git checkout master in your linux repo.

@yonghong-song
Copy link
Contributor

thanks. this time succeeded! Look likes need to learn a few git advanced commands :-)

@yonghong-song yonghong-song merged commit 8e42f42 into libbpf:master Feb 17, 2019
@anakryiko anakryiko deleted the kernel-sync-script branch February 17, 2019 06:33
evverx added a commit to evverx/libbpf that referenced this pull request Apr 11, 2022
Fixes
```
./out/bpf-object-fuzzer: Running 1 inputs 1 time(s) each.
Running: CORPUS/036ff286c13e4590646c7ef59435ec642432da8e
elf_begin.c:232:20: runtime error: member access within misaligned address 0x000001655e71 for type 'Elf64_Shdr', which requires 8 byte alignment
0x000001655e71: note: pointer points here
 00 00 00  7f 45 4c 46 02 02 01 00  00 00 07 fb 00 1d 00 00  6c 69 63 65 42 fb 00 41  00 57 03 00 20
              ^
    #0 0x574d51 in get_shnum /home/libbpf/elfutils/libelf/elf_begin.c:232:20
    #1 0x574d51 in file_read_elf /home/libbpf/elfutils/libelf/elf_begin.c:296:19
    #2 0x569c2c in __libelf_read_mmaped_file /home/libbpf/elfutils/libelf/elf_begin.c:559:14
    #3 0x58e812 in elf_memory /home/libbpf/elfutils/libelf/elf_memory.c:49:10
    #4 0x4905b4 in bpf_object__elf_init /home/libbpf/src/libbpf.c:1255:9
    #5 0x4905b4 in bpf_object_open /home/libbpf/src/libbpf.c:7104:8
    libbpf#6 0x49144e in bpf_object__open_mem /home/libbpf/src/libbpf.c:7171:20
    libbpf#7 0x483018 in LLVMFuzzerTestOneInput /home/libbpf/fuzz/bpf-object-fuzzer.c:16:8
    libbpf#8 0x439389 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/libbpf/out/bpf-object-fuzzer+0x439389)
    libbpf#9 0x419e2f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/libbpf/out/bpf-object-fuzzer+0x419e2f)
    libbpf#10 0x421aee in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/libbpf/out/bpf-object-fuzzer+0x421aee)
    libbpf#11 0x410f96 in main (/home/libbpf/out/bpf-object-fuzzer+0x410f96)
    libbpf#12 0x7f153e21255f in __libc_start_call_main (/lib64/libc.so.6+0x2d55f)
    libbpf#13 0x7f153e21260b in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2d60b)
    libbpf#14 0x410fe4 in _start (/home/libbpf/out/bpf-object-fuzzer+0x410fe4)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior elf_begin.c:232:20 in
```
and
```
./out/bpf-object-fuzzer: Running 1 inputs 1 time(s) each.
Running: CORPUS/446b578d82c47fe177de6fd675f4cb6bae8d1ea9
elf_begin.c:485:40: runtime error: addition of unsigned offset to 0x000002277e70 overflowed to 0x0000021d7e6f
    #0 0x5748f1 in file_read_elf /home/libbpf/elfutils/libelf/elf_begin.c:485:40
    #1 0x569c2c in __libelf_read_mmaped_file /home/libbpf/elfutils/libelf/elf_begin.c:559:14
    #2 0x58e812 in elf_memory /home/libbpf/elfutils/libelf/elf_memory.c:49:10
    #3 0x4905b4 in bpf_object__elf_init /home/libbpf/src/libbpf.c:1255:9
    #4 0x4905b4 in bpf_object_open /home/libbpf/src/libbpf.c:7104:8
    #5 0x49144e in bpf_object__open_mem /home/libbpf/src/libbpf.c:7171:20
    libbpf#6 0x483018 in LLVMFuzzerTestOneInput /home/libbpf/fuzz/bpf-object-fuzzer.c:16:8
    libbpf#7 0x439389 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/libbpf/out/bpf-object-fuzzer+0x439389)
    libbpf#8 0x419e2f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/libbpf/out/bpf-object-fuzzer+0x419e2f)
    libbpf#9 0x421aee in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/libbpf/out/bpf-object-fuzzer+0x421aee)
    libbpf#10 0x410f96 in main (/home/libbpf/out/bpf-object-fuzzer+0x410f96)
    libbpf#11 0x7f753e38255f in __libc_start_call_main (/lib64/libc.so.6+0x2d55f)
    libbpf#12 0x7f753e38260b in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2d60b)
    libbpf#13 0x410fe4 in _start (/home/libbpf/out/bpf-object-fuzzer+0x410fe4)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior elf_begin.c:485:40 in
```
anakryiko pushed a commit that referenced this pull request Apr 11, 2022
Fixes
```
./out/bpf-object-fuzzer: Running 1 inputs 1 time(s) each.
Running: CORPUS/036ff286c13e4590646c7ef59435ec642432da8e
elf_begin.c:232:20: runtime error: member access within misaligned address 0x000001655e71 for type 'Elf64_Shdr', which requires 8 byte alignment
0x000001655e71: note: pointer points here
 00 00 00  7f 45 4c 46 02 02 01 00  00 00 07 fb 00 1d 00 00  6c 69 63 65 42 fb 00 41  00 57 03 00 20
              ^
    #0 0x574d51 in get_shnum /home/libbpf/elfutils/libelf/elf_begin.c:232:20
    #1 0x574d51 in file_read_elf /home/libbpf/elfutils/libelf/elf_begin.c:296:19
    #2 0x569c2c in __libelf_read_mmaped_file /home/libbpf/elfutils/libelf/elf_begin.c:559:14
    #3 0x58e812 in elf_memory /home/libbpf/elfutils/libelf/elf_memory.c:49:10
    #4 0x4905b4 in bpf_object__elf_init /home/libbpf/src/libbpf.c:1255:9
    #5 0x4905b4 in bpf_object_open /home/libbpf/src/libbpf.c:7104:8
    #6 0x49144e in bpf_object__open_mem /home/libbpf/src/libbpf.c:7171:20
    #7 0x483018 in LLVMFuzzerTestOneInput /home/libbpf/fuzz/bpf-object-fuzzer.c:16:8
    #8 0x439389 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/libbpf/out/bpf-object-fuzzer+0x439389)
    #9 0x419e2f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/libbpf/out/bpf-object-fuzzer+0x419e2f)
    #10 0x421aee in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/libbpf/out/bpf-object-fuzzer+0x421aee)
    #11 0x410f96 in main (/home/libbpf/out/bpf-object-fuzzer+0x410f96)
    #12 0x7f153e21255f in __libc_start_call_main (/lib64/libc.so.6+0x2d55f)
    #13 0x7f153e21260b in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2d60b)
    #14 0x410fe4 in _start (/home/libbpf/out/bpf-object-fuzzer+0x410fe4)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior elf_begin.c:232:20 in
```
and
```
./out/bpf-object-fuzzer: Running 1 inputs 1 time(s) each.
Running: CORPUS/446b578d82c47fe177de6fd675f4cb6bae8d1ea9
elf_begin.c:485:40: runtime error: addition of unsigned offset to 0x000002277e70 overflowed to 0x0000021d7e6f
    #0 0x5748f1 in file_read_elf /home/libbpf/elfutils/libelf/elf_begin.c:485:40
    #1 0x569c2c in __libelf_read_mmaped_file /home/libbpf/elfutils/libelf/elf_begin.c:559:14
    #2 0x58e812 in elf_memory /home/libbpf/elfutils/libelf/elf_memory.c:49:10
    #3 0x4905b4 in bpf_object__elf_init /home/libbpf/src/libbpf.c:1255:9
    #4 0x4905b4 in bpf_object_open /home/libbpf/src/libbpf.c:7104:8
    #5 0x49144e in bpf_object__open_mem /home/libbpf/src/libbpf.c:7171:20
    #6 0x483018 in LLVMFuzzerTestOneInput /home/libbpf/fuzz/bpf-object-fuzzer.c:16:8
    #7 0x439389 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/libbpf/out/bpf-object-fuzzer+0x439389)
    #8 0x419e2f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/libbpf/out/bpf-object-fuzzer+0x419e2f)
    #9 0x421aee in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/libbpf/out/bpf-object-fuzzer+0x421aee)
    #10 0x410f96 in main (/home/libbpf/out/bpf-object-fuzzer+0x410f96)
    #11 0x7f753e38255f in __libc_start_call_main (/lib64/libc.so.6+0x2d55f)
    #12 0x7f753e38260b in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2d60b)
    #13 0x410fe4 in _start (/home/libbpf/out/bpf-object-fuzzer+0x410fe4)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior elf_begin.c:485:40 in
```
anakryiko pushed a commit to anakryiko/libbpf that referenced this pull request Oct 17, 2022
ASAN reports an use-after-free in btf_dump_name_dups:

ERROR: AddressSanitizer: heap-use-after-free on address 0xffff927006db at pc 0xaaaab5dfb618 bp 0xffffdd89b890 sp 0xffffdd89b928
READ of size 2 at 0xffff927006db thread T0
    #0 0xaaaab5dfb614 in __interceptor_strcmp.part.0 (test_progs+0x21b614)
    libbpf#1 0xaaaab635f144 in str_equal_fn tools/lib/bpf/btf_dump.c:127
    libbpf#2 0xaaaab635e3e0 in hashmap_find_entry tools/lib/bpf/hashmap.c:143
    libbpf#3 0xaaaab635e72c in hashmap__find tools/lib/bpf/hashmap.c:212
    libbpf#4 0xaaaab6362258 in btf_dump_name_dups tools/lib/bpf/btf_dump.c:1525
    libbpf#5 0xaaaab636240c in btf_dump_resolve_name tools/lib/bpf/btf_dump.c:1552
    libbpf#6 0xaaaab6362598 in btf_dump_type_name tools/lib/bpf/btf_dump.c:1567
    libbpf#7 0xaaaab6360b48 in btf_dump_emit_struct_def tools/lib/bpf/btf_dump.c:912
    libbpf#8 0xaaaab6360630 in btf_dump_emit_type tools/lib/bpf/btf_dump.c:798
    libbpf#9 0xaaaab635f720 in btf_dump__dump_type tools/lib/bpf/btf_dump.c:282
    libbpf#10 0xaaaab608523c in test_btf_dump_incremental tools/testing/selftests/bpf/prog_tests/btf_dump.c:236
    libbpf#11 0xaaaab6097530 in test_btf_dump tools/testing/selftests/bpf/prog_tests/btf_dump.c:875
    libbpf#12 0xaaaab6314ed0 in run_one_test tools/testing/selftests/bpf/test_progs.c:1062
    libbpf#13 0xaaaab631a0a8 in main tools/testing/selftests/bpf/test_progs.c:1697
    libbpf#14 0xffff9676d214 in __libc_start_main ../csu/libc-start.c:308
    libbpf#15 0xaaaab5d65990  (test_progs+0x185990)

0xffff927006db is located 11 bytes inside of 16-byte region [0xffff927006d0,0xffff927006e0)
freed by thread T0 here:
    #0 0xaaaab5e2c7c4 in realloc (test_progs+0x24c7c4)
    libbpf#1 0xaaaab634f4a0 in libbpf_reallocarray tools/lib/bpf/libbpf_internal.h:191
    libbpf#2 0xaaaab634f840 in libbpf_add_mem tools/lib/bpf/btf.c:163
    libbpf#3 0xaaaab636643c in strset_add_str_mem tools/lib/bpf/strset.c:106
    libbpf#4 0xaaaab6366560 in strset__add_str tools/lib/bpf/strset.c:157
    libbpf#5 0xaaaab6352d70 in btf__add_str tools/lib/bpf/btf.c:1519
    libbpf#6 0xaaaab6353e10 in btf__add_field tools/lib/bpf/btf.c:2032
    libbpf#7 0xaaaab6084fcc in test_btf_dump_incremental tools/testing/selftests/bpf/prog_tests/btf_dump.c:232
    libbpf#8 0xaaaab6097530 in test_btf_dump tools/testing/selftests/bpf/prog_tests/btf_dump.c:875
    libbpf#9 0xaaaab6314ed0 in run_one_test tools/testing/selftests/bpf/test_progs.c:1062
    libbpf#10 0xaaaab631a0a8 in main tools/testing/selftests/bpf/test_progs.c:1697
    libbpf#11 0xffff9676d214 in __libc_start_main ../csu/libc-start.c:308
    libbpf#12 0xaaaab5d65990  (test_progs+0x185990)

previously allocated by thread T0 here:
    #0 0xaaaab5e2c7c4 in realloc (test_progs+0x24c7c4)
    libbpf#1 0xaaaab634f4a0 in libbpf_reallocarray tools/lib/bpf/libbpf_internal.h:191
    libbpf#2 0xaaaab634f840 in libbpf_add_mem tools/lib/bpf/btf.c:163
    libbpf#3 0xaaaab636643c in strset_add_str_mem tools/lib/bpf/strset.c:106
    libbpf#4 0xaaaab6366560 in strset__add_str tools/lib/bpf/strset.c:157
    libbpf#5 0xaaaab6352d70 in btf__add_str tools/lib/bpf/btf.c:1519
    libbpf#6 0xaaaab6353ff0 in btf_add_enum_common tools/lib/bpf/btf.c:2070
    libbpf#7 0xaaaab6354080 in btf__add_enum tools/lib/bpf/btf.c:2102
    libbpf#8 0xaaaab6082f50 in test_btf_dump_incremental tools/testing/selftests/bpf/prog_tests/btf_dump.c:162
    libbpf#9 0xaaaab6097530 in test_btf_dump tools/testing/selftests/bpf/prog_tests/btf_dump.c:875
    libbpf#10 0xaaaab6314ed0 in run_one_test tools/testing/selftests/bpf/test_progs.c:1062
    libbpf#11 0xaaaab631a0a8 in main tools/testing/selftests/bpf/test_progs.c:1697
    libbpf#12 0xffff9676d214 in __libc_start_main ../csu/libc-start.c:308
    libbpf#13 0xaaaab5d65990  (test_progs+0x185990)

The reason is that the key stored in hash table name_map is a string
address, and the string memory is allocated by realloc() function, when
the memory is resized by realloc() later, the old memory may be freed,
so the address stored in name_map references to a freed memory, causing
use-after-free.

Fix it by storing duplicated string address in name_map.

Fixes: 919d2b1dbb07 ("libbpf: Allow modification of BTF and add btf__add_str API")
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Martin KaFai Lau <martin.lau@kernel.org>
Link: https://lore.kernel.org/bpf/20221011120108.782373-2-xukuohai@huaweicloud.com
anakryiko pushed a commit that referenced this pull request Oct 17, 2022
ASAN reports an use-after-free in btf_dump_name_dups:

ERROR: AddressSanitizer: heap-use-after-free on address 0xffff927006db at pc 0xaaaab5dfb618 bp 0xffffdd89b890 sp 0xffffdd89b928
READ of size 2 at 0xffff927006db thread T0
    #0 0xaaaab5dfb614 in __interceptor_strcmp.part.0 (test_progs+0x21b614)
    #1 0xaaaab635f144 in str_equal_fn tools/lib/bpf/btf_dump.c:127
    #2 0xaaaab635e3e0 in hashmap_find_entry tools/lib/bpf/hashmap.c:143
    #3 0xaaaab635e72c in hashmap__find tools/lib/bpf/hashmap.c:212
    #4 0xaaaab6362258 in btf_dump_name_dups tools/lib/bpf/btf_dump.c:1525
    #5 0xaaaab636240c in btf_dump_resolve_name tools/lib/bpf/btf_dump.c:1552
    #6 0xaaaab6362598 in btf_dump_type_name tools/lib/bpf/btf_dump.c:1567
    #7 0xaaaab6360b48 in btf_dump_emit_struct_def tools/lib/bpf/btf_dump.c:912
    #8 0xaaaab6360630 in btf_dump_emit_type tools/lib/bpf/btf_dump.c:798
    #9 0xaaaab635f720 in btf_dump__dump_type tools/lib/bpf/btf_dump.c:282
    #10 0xaaaab608523c in test_btf_dump_incremental tools/testing/selftests/bpf/prog_tests/btf_dump.c:236
    #11 0xaaaab6097530 in test_btf_dump tools/testing/selftests/bpf/prog_tests/btf_dump.c:875
    #12 0xaaaab6314ed0 in run_one_test tools/testing/selftests/bpf/test_progs.c:1062
    #13 0xaaaab631a0a8 in main tools/testing/selftests/bpf/test_progs.c:1697
    #14 0xffff9676d214 in __libc_start_main ../csu/libc-start.c:308
    #15 0xaaaab5d65990  (test_progs+0x185990)

0xffff927006db is located 11 bytes inside of 16-byte region [0xffff927006d0,0xffff927006e0)
freed by thread T0 here:
    #0 0xaaaab5e2c7c4 in realloc (test_progs+0x24c7c4)
    #1 0xaaaab634f4a0 in libbpf_reallocarray tools/lib/bpf/libbpf_internal.h:191
    #2 0xaaaab634f840 in libbpf_add_mem tools/lib/bpf/btf.c:163
    #3 0xaaaab636643c in strset_add_str_mem tools/lib/bpf/strset.c:106
    #4 0xaaaab6366560 in strset__add_str tools/lib/bpf/strset.c:157
    #5 0xaaaab6352d70 in btf__add_str tools/lib/bpf/btf.c:1519
    #6 0xaaaab6353e10 in btf__add_field tools/lib/bpf/btf.c:2032
    #7 0xaaaab6084fcc in test_btf_dump_incremental tools/testing/selftests/bpf/prog_tests/btf_dump.c:232
    #8 0xaaaab6097530 in test_btf_dump tools/testing/selftests/bpf/prog_tests/btf_dump.c:875
    #9 0xaaaab6314ed0 in run_one_test tools/testing/selftests/bpf/test_progs.c:1062
    #10 0xaaaab631a0a8 in main tools/testing/selftests/bpf/test_progs.c:1697
    #11 0xffff9676d214 in __libc_start_main ../csu/libc-start.c:308
    #12 0xaaaab5d65990  (test_progs+0x185990)

previously allocated by thread T0 here:
    #0 0xaaaab5e2c7c4 in realloc (test_progs+0x24c7c4)
    #1 0xaaaab634f4a0 in libbpf_reallocarray tools/lib/bpf/libbpf_internal.h:191
    #2 0xaaaab634f840 in libbpf_add_mem tools/lib/bpf/btf.c:163
    #3 0xaaaab636643c in strset_add_str_mem tools/lib/bpf/strset.c:106
    #4 0xaaaab6366560 in strset__add_str tools/lib/bpf/strset.c:157
    #5 0xaaaab6352d70 in btf__add_str tools/lib/bpf/btf.c:1519
    #6 0xaaaab6353ff0 in btf_add_enum_common tools/lib/bpf/btf.c:2070
    #7 0xaaaab6354080 in btf__add_enum tools/lib/bpf/btf.c:2102
    #8 0xaaaab6082f50 in test_btf_dump_incremental tools/testing/selftests/bpf/prog_tests/btf_dump.c:162
    #9 0xaaaab6097530 in test_btf_dump tools/testing/selftests/bpf/prog_tests/btf_dump.c:875
    #10 0xaaaab6314ed0 in run_one_test tools/testing/selftests/bpf/test_progs.c:1062
    #11 0xaaaab631a0a8 in main tools/testing/selftests/bpf/test_progs.c:1697
    #12 0xffff9676d214 in __libc_start_main ../csu/libc-start.c:308
    #13 0xaaaab5d65990  (test_progs+0x185990)

The reason is that the key stored in hash table name_map is a string
address, and the string memory is allocated by realloc() function, when
the memory is resized by realloc() later, the old memory may be freed,
so the address stored in name_map references to a freed memory, causing
use-after-free.

Fix it by storing duplicated string address in name_map.

Fixes: 919d2b1dbb07 ("libbpf: Allow modification of BTF and add btf__add_str API")
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Martin KaFai Lau <martin.lau@kernel.org>
Link: https://lore.kernel.org/bpf/20221011120108.782373-2-xukuohai@huaweicloud.com
danielocfb pushed a commit to danielocfb/libbpf that referenced this pull request Feb 15, 2023
Original change: undetermined

Change-Id: I32955427576e59c77aeb84dcaab6aa72f1862490
danielocfb pushed a commit to danielocfb/libbpf that referenced this pull request Feb 15, 2023
…cf7c79f

Original change: undetermined

Change-Id: If01a0204b1e18e197116dcf03de0f3ce7e07f69e
danielocfb pushed a commit to danielocfb/libbpf that referenced this pull request Feb 15, 2023
…cf7c79f am: 656f471

Original change: undetermined

Change-Id: I201cf3dc6afcaba2094d05fa8df88b1709a9ea29
anakryiko pushed a commit to anakryiko/libbpf that referenced this pull request Apr 29, 2025
When the binary path is excessively long, the generated probe_name in libbpf
exceeds the kernel's MAX_EVENT_NAME_LEN limit (64 bytes).
This causes legacy uprobe event attachment to fail with error code -22.

The fix reorders the fields to place the unique ID before the name.
This ensures that even if truncation occurs via snprintf, the unique ID
remains intact, preserving event name uniqueness. Additionally, explicit
checks with MAX_EVENT_NAME_LEN are added to enforce length constraints.

Before Fix:
	./test_progs -t attach_probe/kprobe-long_name
	......
	libbpf: failed to add legacy kprobe event for 'bpf_testmod_looooooooooooooooooooooooooooooong_name+0x0': -EINVAL
	libbpf: prog 'handle_kprobe': failed to create kprobe 'bpf_testmod_looooooooooooooooooooooooooooooong_name+0x0' perf event: -EINVAL
	test_attach_kprobe_long_event_name:FAIL:attach_kprobe_long_event_name unexpected error: -22
	test_attach_probe:PASS:uprobe_ref_ctr_cleanup 0 nsec
	libbpf#13/11   attach_probe/kprobe-long_name:FAIL
	libbpf#13      attach_probe:FAIL

	./test_progs -t attach_probe/uprobe-long_name
	......
	libbpf: failed to add legacy uprobe event for /root/linux-bpf/bpf-next/tools/testing/selftests/bpf/test_progs:0x13efd9: -EINVAL
	libbpf: prog 'handle_uprobe': failed to create uprobe '/root/linux-bpf/bpf-next/tools/testing/selftests/bpf/test_progs:0x13efd9' perf event: -EINVAL
	test_attach_uprobe_long_event_name:FAIL:attach_uprobe_long_event_name unexpected error: -22
	libbpf#13/10   attach_probe/uprobe-long_name:FAIL
	libbpf#13      attach_probe:FAIL
After Fix:
	./test_progs -t attach_probe/uprobe-long_name
	libbpf#13/10   attach_probe/uprobe-long_name:OK
	libbpf#13      attach_probe:OK
	Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED

	./test_progs -t attach_probe/kprobe-long_name
	libbpf#13/11   attach_probe/kprobe-long_name:OK
	libbpf#13      attach_probe:OK
	Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED

Fixes: 46ed5fc33db9 ("libbpf: Refactor and simplify legacy kprobe code")
Fixes: cc10623c6810 ("libbpf: Add legacy uprobe attaching support")
Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
Signed-off-by: Feng Yang <yangfeng@kylinos.cn>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20250417014848.59321-2-yangfeng59949@163.com
anakryiko pushed a commit that referenced this pull request Apr 29, 2025
When the binary path is excessively long, the generated probe_name in libbpf
exceeds the kernel's MAX_EVENT_NAME_LEN limit (64 bytes).
This causes legacy uprobe event attachment to fail with error code -22.

The fix reorders the fields to place the unique ID before the name.
This ensures that even if truncation occurs via snprintf, the unique ID
remains intact, preserving event name uniqueness. Additionally, explicit
checks with MAX_EVENT_NAME_LEN are added to enforce length constraints.

Before Fix:
	./test_progs -t attach_probe/kprobe-long_name
	......
	libbpf: failed to add legacy kprobe event for 'bpf_testmod_looooooooooooooooooooooooooooooong_name+0x0': -EINVAL
	libbpf: prog 'handle_kprobe': failed to create kprobe 'bpf_testmod_looooooooooooooooooooooooooooooong_name+0x0' perf event: -EINVAL
	test_attach_kprobe_long_event_name:FAIL:attach_kprobe_long_event_name unexpected error: -22
	test_attach_probe:PASS:uprobe_ref_ctr_cleanup 0 nsec
	#13/11   attach_probe/kprobe-long_name:FAIL
	#13      attach_probe:FAIL

	./test_progs -t attach_probe/uprobe-long_name
	......
	libbpf: failed to add legacy uprobe event for /root/linux-bpf/bpf-next/tools/testing/selftests/bpf/test_progs:0x13efd9: -EINVAL
	libbpf: prog 'handle_uprobe': failed to create uprobe '/root/linux-bpf/bpf-next/tools/testing/selftests/bpf/test_progs:0x13efd9' perf event: -EINVAL
	test_attach_uprobe_long_event_name:FAIL:attach_uprobe_long_event_name unexpected error: -22
	#13/10   attach_probe/uprobe-long_name:FAIL
	#13      attach_probe:FAIL
After Fix:
	./test_progs -t attach_probe/uprobe-long_name
	#13/10   attach_probe/uprobe-long_name:OK
	#13      attach_probe:OK
	Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED

	./test_progs -t attach_probe/kprobe-long_name
	#13/11   attach_probe/kprobe-long_name:OK
	#13      attach_probe:OK
	Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED

Fixes: 46ed5fc33db9 ("libbpf: Refactor and simplify legacy kprobe code")
Fixes: cc10623c6810 ("libbpf: Add legacy uprobe attaching support")
Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
Signed-off-by: Feng Yang <yangfeng@kylinos.cn>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20250417014848.59321-2-yangfeng59949@163.com
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0