8000 binfmt_misc tuning can break ape · Issue #2 · jart/cosmopolitan · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

binfmt_misc tuning can break ape #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
omimo opened this issue Aug 24, 2020 · 25 comments
Closed

binfmt_misc tuning can break ape #2

omimo opened this issue Aug 24, 2020 · 25 comments

Comments

@omimo
Copy link
omimo commented Aug 24, 2020

Hello,

I'm just trying this on Ubuntu 20.04 and get the following error:

$ make -j8 -O o//examples/hello.com

MKDEPS         o//depend
ZIPOBJ         o//libc/stubs/errno.S.zip.o
run-detectors: unable to find an interpreter for o/build/bootstrap/zipobj.com
make: *** [build/rules.mk:76: o//libc/stubs/errno.S.zip.o] Error 2
ZIPOBJ         o//libc/stubs/triplf.S.zip.o
run-detectors: unable to find an interpreter for o/build/bootstrap/zipobj.com
ZIPOBJ         o//libc/stubs/asanjmp.greg.S.zip.o
run-detectors: unable to find an interpreter for o/build/bootstrap/zipobj.com
ZIPOBJ         o//libc/stubs/subvsi3.S.zip.o
run-detectors: unable to find an interpreter for o/build/bootstrap/zipobj.com
ZIPOBJ         o//libc/stubs/mulvdi3.S.zip.o
run-detectors: unable to find an interpreter for o/build/bootstrap/zipobj.com
ZIPOBJ         o//libc/stubs/errnolocation.S.zip.o
run-detectors: unable to find an interpreter for o/build/bootstrap/zipobj.com
ZIPOBJ         o//libc/stubs/mcount.S.zip.o
run-detectors: unable to find an interpreter for o/build/bootstrap/zipobj.com
ZIPOBJ         o//libc/stubs/panic.S.zip.o
run-detectors: unable to find an interpreter for o/build/bootstrap/zipobj.com
make: *** [build/rules.mk:76: o//libc/stubs/panic.S.zip.o] Error 2

If I try to run zipobj.com directly, it returns the following:

$build/bootstrap/zipobj.com

error:tool/build/zipobj.c:93:zipobj.com: check failed
        CHECK_NE(NULL, outpath_);
                 → 0 (outpath_)
                != 0 (NULL)
        E?/err=0/errno:0/GetLastError:0
        build/bootstrap/zipobj.com

Thanks!
@0atman
Copy link
0atman commented Aug 24, 2020 •
edited
Loading

Same here, same OS: Ubuntu 20.04 on x86_64. Are we missing some objects?

λ make -j8 o//tool/viz/printvideo.com
MKDEPS         o//depend
ZIPOBJ         o//libc/stubs/mcount.S.zip.o
run-detectors: unable to find an interpreter for o/build/bootstrap/zipobj.com
run-detectors: unable to find an interpreter for o/build/bootstrap/zipobj.com
run-detectors: make: *** [build/rules.mk:76: o//libc/stubs/errno.S.zip.o] Error 2
unable to find an interpreter for o/build/bootstrap/zipobj.com
make: *** Waiting for unfinished jobs....
run-detectors: unable to find an interpreter for o/build/bootstrap/zipobj.com
make: *** [build/rules.mk:76: o//libc/stubs/subvsi3.S.zip.o] Error 2
make: *** [build/rules.mk:76: o//libc/stubs/gcov.S.zip.o] Error 2
make: *** [build/rules.mk:76: o//libc/stubs/triplf.S.zip.o] Error 2
run-detectors: unable to find an interpreter for o/build/bootstrap/zipobj.com
make: *** [build/rules.mk:76: o//libc/stubs/asanjmp.greg.S.zip.o] Error 2
run-detectors: unable to find an interpreter for o/build/bootstrap/zipobj.com
run-detectors: unable to find an interpreter for o/build/bootstrap/zipobj.com
make: *** [build/rules.mk:76: o//libc/stubs/errnolocation.S.zip.o] Error 2
make: *** [build/rules.mk:76: o//libc/stubs/addvsi3.S.zip.o] Error 2
run-detectors: unable to find an interpreter for o/build/bootstrap/zipobj.com
make: *** [build/rules.mk:76: o//libc/stubs/mcount.S.zip.o] Error 2

I'm running on Linux oatman-desktop 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

@jart
Copy link
Owner
jart commented Aug 25, 2020

Have you configured binfmt_misc to automatically launch Windows binaries under WINE? You may need to disable WINE while using this project. Cosmopolitan can only supports the stock behavior. I recommend avoiding binfmt_misc in general if possible.

@jart jart closed this as completed Aug 25, 2020
@0atman
Copy link
0atman commented Aug 25, 2020

Wine's not installed

@0atman
Copy link
0atman commented Aug 25, 2020 •
edited
Loading

We'll get to the bottom of it, here's some lightly-edited output from vanilla ubuntu to the example not working - what am I missing?

λ docker run -it ubuntu
root@66c04a69b391:/# cd /root

root@66c04a69b391:~# apt update && apt install -y git
Get:1 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
[...]
done.

root@66c04a69b391:~# git clone https://github.com/jart/cosmopolitan.git
Cloning into 'cosmopolitan'...
[...]
Resolving deltas: 100% (4569/4569), done.

root@66c04a69b391:~# cd cosmopolitan/

root@66c04a69b391:~/cosmopolitan # ls
Makefile  README.md  build  examples  net   third_party  usr
NOTICE    ape        dsp    libc      test  tool

root@66c04a69b391:~/cosmopolitan # apt install -y make
[...]
Setting up make (4.2.1-1.2) ...

root@66c04a69b391:~/cosmopolitan # make -j8 o//tool/viz/printvideo.com
MKDEPS         o//depend
ZIPOBJ         o//libc/stubs/subvsi3.S.zip.o
ZIPOBJ         o//libc/stubs/errno.S.zip.o
make: *** [build/rules.mk:77: o//libc/stubs/subvsi3.S.zip.o] Error 127
make: *** Waiting for unfinished jobs....
build/zipobj: 54: exec: build/zipobj: 54: exec: o/build/bootstrap/zipobj.com: not found
o/build/bootstrap/zipobj.com: not found
make: *** [build/rules.mk:77: o//libc/stubs/errno.S.zip.o] Error 127
ZIPOBJ         o//libc/stubs/addvsi3.S.zip.o
build/zipobj: 54: exec: o/build/bootstrap/zipobj.com: not found
build/zipobj: 54: exec: o/build/bootstrap/zipobj.com: not found
ZIPOBJ         o//libc/stubs/errnolocation.S.zip.o
make: *** [build/rules.mk:77: o//libc/stubs/addvsi3.S.zip.o] Error 127
ZIPOBJ         o//libc/stubs/gcov.S.zip.o
build/zipobj: 54: exec: o/build/bootstrap/zipobj.com: not found
make: *** [build/rules.mk:77: o//libc/stubs/errnolocation.S.zip.o] Error 127
make: *** [build/rules.mk:77: o//libc/stubs/gcov.S.zip.o] Error 127

@jart jart reopened this Aug 25, 2020
@jart
Copy link
Owner
jart commented Aug 25, 2020

What shell and what version is Docker Ubuntu using? Maybe they mapped zsh to /bin/sh or possibly have a regression with Thompson Shell backwards compatibility, since build/zipobj doesn't have a shebang line. What happens if you do this?

echo echo hello >hello
chmod +x hello
/bin/sh -c './hello'

If that doesn't print hello then I'd be very concerned. Another troubleshooting step is:

make SILENT=0 o//tool/viz/printvideo.com

Which should hopefully make it clearer which commands are being invoked.

@0atman
Copy link
0atman commented Aug 25, 2020 •
edited
Loading 
docker run -it ubuntu
root@bcd27fd06d69:/# echo echo hello >hello
root@bcd27fd06d69:/# chmod +x hello
root@bcd27fd06d69:/# /bin/sh -c './hello'
hello

@0atman
Copy link
0atman commented Aug 25, 2020 •
edited
Loading 
docker run -it ubuntu
root@bcd27fd06d69:/# echo $SHELL
/bin/bash
root@bcd27fd06d69:/# bash --version
GNU bash, version 5.0.16(1)-release (x86_64-pc-linux-gnu)

@0atman
Copy link
0atman commented Aug 25, 2020

how odd!

@jart
Copy link
Owner
jart commented Aug 25, 2020

I just installed the latest Ubuntu in a VM since you've got me curious. However I failed to reproduce the issue.

image

Something peculiar must be happening with the Docker variant of Ubuntu. Would you be kind enough to help troubleshoot this down a bit further? Possibly by using strace? I'm very reluctant to install Docker. I am however super happy to implement whatever fix or workaround you need, once we identify the issue.

@omimo
Copy link
Author
omimo commented Aug 25, 2020

I'm not running Ubuntu in docker!

@omimo
Copy link
Author
omimo commented Aug 25, 2020

On my system, the following works and prints hello:

echo echo hello >hello
chmod +x hello
/bin/sh -c './hello'

And running make SILENT=0 o//tool/viz/printvideo.com returns the following:

$ make SILENT=0 o//tool/viz/printvideo.com
♥cosmo
o/build/bootstrap/mkdeps.com -o o//depend -r o// o//srcs.txt o//hdrs.txt
run-detectors: unable to find an interpreter for o/build/bootstrap/mkdeps.com
o/build/bootstrap/zipobj.com -o o//libc/stubs/errno.S.zip.o  \
  libc/stubs/errno.S
run-detectors: unable to find an interpreter for o/build/bootstrap/zipobj.com
make: *** [build/rules.mk:76: o//libc/stubs/errno.S.zip.o] Error 2

@omimo
Copy link
Author
omimo commented Aug 25, 2020

Another thing that might help, in Ubuntu /bin/sh points to /bin/dash.

@jart
Copy link
Owner
jart commented Aug 26, 2020

Almquist shell works fine (outside FreeBSD which added restrictions to it) and is highly recommended. Try exploring the contents of ls /proc/sys/fs/binfmt_misc/ and report back. I installed Ubuntu 20.01 server and didn't happen. It's possible that some other package that got installed on your machines, possibly in the desktop base, that's globally hooking shell scripts and then throws an error if the script starts with the letters "MZ".

@omimo
Copy link
Author
omimo commented Aug 26, 2020 
❯ ls /proc/sys/fs/binfmt_misc/
cli  jar  llvm-10-runtime.binfmt  llvm-6.0-runtime.binfmt  llvm-8-runtime.binfmt  python2.7  python3.8  register  sbcl  status

@jart
Copy link
Owner
jart commented Aug 28, 2020

Could you try running the following command and let me know if it fixes the issue?

sudo sh -c "echo ':APE:M::MZqFpD::/bin/sh:' >/proc/sys/fs/binfmt_misc/register"

@omimo
Copy link
Author
omimo commented Aug 28, 2020

Running the above command fixed it!

So what was the problem?

Thanks!

@omimo omimo closed this as completed Aug 31, 2020
@jart
Copy link
Owner
jart commented Sep 1, 2020

You're welcome! Thanks for the report. I'll update the Makefile shortly to auto-detect and report this workaround.

This was referenced Sep 1, 2020
Closed
Closed
@chr15m
Copy link
chr15m commented Sep 1, 2020

@omimo you can read more about binfmt handlers here. I'm guessing here but I think cosmopolitan's binary format requires the host OS to be able to recognize a particular pattern at the start of the file, which this workaround registers. The reason it has to be there for the compilation phase is (again a guess) the compiler has some level of self-hosting going on where it must compile and then use its own toolchain.

@jart if you add an auto-detect I'd humbly suggest some explanation of why the binfmt handler is required and what it does.

@jart
Copy link
Owner
jart commented Sep 1, 2020

Binfmt is absolutely not a requirement. I recommend against using it entirely. Cosmopolitan's executable format is just a shell script without a shebang line. Please read https://justine.storage.googleapis.com/ape.html to learn more. What happens is binfmt_misc gives people like package maintainers the ability to arbitrarily redefine how the kernel processes executables, and in certain cases that break things.

@chr15m
Copy link
chr15m commented Sep 1, 2020 •
edited
Loading

@jart ah so the workaround is undoing something that Ubuntu has overridden in binfmt_misc?

PS I read and very much enjoyed the post. My apologies for misunderstanding.

jart added a commit that referenced this issue Sep 1, 2020
@jart
Add binfmt_misc workaround detector to builds
f213556 
This is a recently introduced Linux Kernel feature that gives people
like Debian package mantainers the power to arbitrarily redefine how
executables are interpreted by the kernel. If your system gets tuned
this way and you're not able to disable it, then you need to restore
default behavior for the APE MZqFpD prefix as follows:

    sudo sh -c "echo ':APE:M::MZqFpD::/bin/sh:' >/proc/sys/fs/binfmt_misc/register"

This prefix will cover all .com executables built with this tooling.
Please don't run the above command unless you're certain you need it.
See #2 for additional context.
@jart
Copy link
Owner
jart commented Sep 1, 2020

The workaround one-liner doesn't override anything. It simply restores the default behavior for the APE executable prefix MZqFpD, as specified here:

cosmopolitan/ape/ape.S

Line 121 in f4f4caa

ape.mz: .ascii "MZ" # Mark 'Zibo' Joseph Zbikowski
As far as I know, if Ubuntu package maintainers have tuned your system to run MZ executables under WINE, then the MZqFpD prefix basically just clarifies how to do the right thing in case of αcτµαlly pδrταblε εxεcµταblε.

@chr15m
Copy link
chr15m commented Sep 1, 2020

Understood, thanks for the explanation.

@jart jart changed the title Build fails binfmt_misc tuning can break ape Feb 27, 2021
@jart jart mentioned this issue Feb 27, 2021
Closed
@jart jart mentioned this issue Feb 28, 2021
Closed
@jart jart mentioned this issue May 20, 2021
Closed
@acailly acailly mentioned this issue Jun 8, 2021
Closed
@chrysn
Copy link
chrysn commented Aug 8, 2022 •
edited
Loading

Users who don't want to modify their /proc/sys/fs/binfmt_misc/ long-term (eg. because they don't need APE) can select a mode that doesn't produce them at build time, eg. MODE=optlinux make -j8 o/optlinux/third_party/make/make.com.

This still requires the binfmt workaround at build time (as the bootstrap executables cocmd, compile, fixupobj, package, unbundle and zipobj as well as the custom ar are necessary to build it and not generally available otherwise), but it moves the problem to one of bootstrapping / building from one of running.

@suchipi
Copy link
suchipi commented Sep 11, 2023 •
edited
Loading

future readers: this issue was fixed in Ubuntu 22.04. APE-format binaries run properly in Ubuntu 22.04 without changes to binfmt_misc.

EDIT: nevermind, I was wrong. the ape loader is still needed in Ubuntu 22.04

@phil294
Copy link
phil294 commented Nov 5, 2023

May I ask what's the path forward here? Doesn't this mean that APE binaries like hello.com can never be fit for non-technical end users on Linux distros given they might have installed something like Wine?

@cjpais cjpais mentioned this issue Apr 9, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@jart @chr15m @0atman @chrysn @omimo @suchipi @phil294
0