8000 Use UDisks to find mounted filesystems on Linux by jwillikers · Pull Request #2683 · thonny/thonny · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Use UDisks to find mounted filesystems on Linux #2683

New iss 8000 ue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Mar 26, 2023

Conversation

jwillikers
Copy link
Contributor
@jwillikers jwillikers commented Mar 4, 2023

This should generally work better on Linux than using the mount command directly. UDisks is the utility that all major Linux desktop distributions use to handle mounting filesystems in userspace. UDisks is accessible through a D-Bus API on the system bus.

D-Bus is complex and requires adding a dependency on a Python library for handling asynchronous communication with the bus. I chose dbus-next since it is actively developed, modern, and doesn't require a glib main loop. Since Thonny is not a Gtk application, it makes more sense to use native Python async functionality instead.

Inside a Flatpak, it's necessary to use UDisks via the D-Bus API to mount and check mounted filesystems. The Flatpak sandbox prevents mounting otherwise and does not supply a mount command with the runtime.

Fixes flathub/org.thonny.Thonny#153, #2650, and #2426.
Probably fixes #2649.

@aivarannamaa
8000 Copy link
Member

Thanks, and sorry for the late response!

It looks like python-dbus-next isn't too heavy and doesn't pull in other deps. In this case I'm happy to try this as the fallback when mount is not available (and possibly as the only option in Thonny 4.1 or 5.0).

@jwillikers
Copy link
Contributor Author

Thanks, and sorry for the late response!

It looks like python-dbus-next isn't too heavy and doesn't pull in other deps. In this case I'm happy to try this as the fallback when mount is not available (and possibly as the only option in Thonny 4.1 or 5.0).

Great! I was thinking I would attempt to use Udisks before falling back to mount on Linux. So far, I've got a standalone udisks.py file in this MR that I can run locally and get the file path to an attached RP2040 board with MicroPython. I just need to find the best way to integrate this with the current filesystem-mounting logic and how to integrate the necessary async event loop. I'll also be adding more robust error handling. I'm hoping to find the time to finish this in the next couple of weeks. Let me know it you have any suggestions or advice. Thank you much!

@jwillikers jwillikers marked this pull request as ready for review March 18, 2023 22:14
@jwillikers
Copy link
Contributor Author
jwillikers commented Mar 18, 2023

@aivarannamaa I've got this working now. I've tested it in the Flatpak and verified that flashing a Raspberry Pi Pico in the UF2 dialog works without issue. It actually works better than Thonny's current behavior on my system since the mount command returns duplicate volumes which are actually symlinks that exist on Fedora Silverblue.

This should generally work better on Linux than using the mount command directly.
UDisks is the utility that all major Linux desktop distributions use to handle mounting filesystems in userspace.
UDisks is accessible through a D-Bus API on the system bus.

D-Bus is complex and requires adding a dependency on a Python library for handling asynchronous communication with the bus.
I chose dbus-next since it is actively developed, modern, and doesn't require a glib main loop.
Since Thonny is not a Gtk application, it makes more sense to use native Python async functionality instead.

Inside a Flatpak, it's necessary to use UDisks via the D-Bus API to mount and check mounted filesystems.
The Flatpak sandbox prevents mounting otherwise and does not supply a mount command with the runtime.
Add XML files for D-Bus introspection.
Provide synchronous wrapper function to list volumes.
Check for volumes with Udisks on Linux before resorting to mount.
Ensure that only devices with "filesystem" usage are listed.
This excludes device nodes like /dev/sda when we only care about filesystems which are on partitions like /dev/sda1.
List volumes according to their detection times.
This is a convenience for the user.
Most recently detected devices are listed first.
This prevents unnecessary errors from the mount command not being found when Udisks is available.
This avoids potential issues with the license header from the XML file taken directly from the UDisks project.
Additionally, place the DBus XML files in a dedicated `dbus` subdirectory instead of using the res directory.
@ghost
Copy link
ghost commented Mar 20, 2023

Should we also consider adding support of xdg desktop portals ??

@jwillikers
Copy link
Contributor Author
jwillikers commented Mar 20, 2023

Should we also consider adding support of xdg desktop portals ??

I believe Tcl/Tk already uses the native file-chooser dialog when selecting a file, which is why it is possible to select files to open and save when the Flatpak sandbox is completely isolated. However, for the Flatpak I've allowed permissions to the user's home directory and the default locations for mounted media. This is so that paths to files are consistent with a person's directory layout on their machine and to allow access to mounted devices. Having Python files individually opened and mounted to arbitrary locations inside the Flatpak makes it untenable to develop anything in Thonny larger than a single file. Most Flatpaks of developer tools grant even greater permissions by default for the same reason. I don't think there is an XDG desktop portal which allows opening directories either, which would be the primary requirement to allow the sandbox to be restricted further.

I'm not sure which XDG desktop portals you have specifically in mind, but I've assumed so far you were referring to file access. Was there something more specific to this PR you had in mind or were you asking more generally?

@ghost
Copy link
ghost commented Mar 20, 2023

Should we also consider adding support of xdg desktop portals ??

I believe Tcl/Tk already uses the native file-chooser dialog when selecting a file, which is why it is possible to select files to open and save when the Flatpak sandbox is completely isolated. However, for the Flatpak I've allowed permissions to the user's home directory and the default locations for mounted media. This is so that paths to files are consistent with a person's directory layout on their machine and to allow access to mounted devices. Having Python files individually opened and mounted to arbitrary locations inside the Flatpak makes it untenable to develop anything in Thonny larger than a single file. Most Flatpaks of developer tools grant even greater permissions by default for the same reason. I don't think there is an XDG desktop portal which allows opening directories either, which would be the primary requirement to allow the sandbox to be restricted further.

I'm not sure which XDG desktop portals you have specifically in mind, but I've assumed so far you were referring to file access. Was there something more specific to this PR you had in mind or were you asking more generally?

What i was saying is that although thonny uses xdg portals for file access in flatpaks due to the default mechanism of flatpak being built around xdg portals ,by default thonny does not uses xdg portals since in snap sandbox it continues to use the tk one, this is even after forcing it to use portals, thonny always uses its internally built less interactive yet working tk dialogs to access files, so i was thinking if we somehow force thonny to use xdg portals as its default globally, this was a suggestion or maybe a general question...

@jwillikers
Copy link
Contributor Author

Should we also consider adding support of xdg desktop portals ??

I believe Tcl/Tk already uses the native file-chooser dialog when selecting a file, which is why it is possible to select files to open and save when the Flatpak sandbox is completely isolated. However, for the Flatpak I've allowed permissions to the user's home directory and the default locations for mounted media. This is so that paths to files are consistent with a person's directory layout on their machine and to allow access to mounted devices. Having Python files individually opened and mounted to arbitrary locations inside the Flatpak makes it untenable to develop anything in Thonny larger than a single file. Most Flatpaks of developer tools grant even greater permissions by default for the same reason. I don't think there is an XDG desktop portal which allows opening directories either, which would be the primary requirement to allow the sandbox to be restricted further.
I'm not sure which XDG desktop portals you have specifically in mind, but I've assumed so far you were referring to file access. Was there something more specific to this PR you had in mind or were you asking more generally?

What i was saying is that although thonny uses xdg portals for file access in flatpaks due to the default mechanism of flatpak being built around xdg portals ,by default thonny does not uses xdg portals since in snap sandbox it continues to use the tk one, this is even after forcing it to use portals, thonny always uses its internally built less interactive yet working tk dialogs to access files, so i was thinking if we somehow force thonny to use xdg portals as its default globally, this was a suggestion or maybe a general question...

Ahh, I think that's an aside from this PR, but do you have zenity installed and on PATH in the snap? That's necessary to get the native GNOME file chooser dialog in Thonny.

@ghost
Copy link
ghost commented Mar 20, 2023

Should we also consider adding support of xdg desktop portals ??

I believe Tcl/Tk already uses the native file-chooser dialog when selecting a file, which is why it is possible to select files to open and save when the Flatpak sandbox is completely isolated. However, for the Flatpak I've allowed permissions to the user's home directory and the default locations for mounted media. This is so that paths to files are consistent with a person's directory layout on their machine and to allow access to mounted devices. Having Python files individually opened and mounted to arbitrary locations inside the Flatpak makes it untenable to develop anything in Thonny larger than a single file. Most Flatpaks of developer tools grant even greater permissions by default for the same reason. I don't think there is an XDG desktop portal which allows opening directories either, which would be the primary requirement to allow the sandbox to be restricted further.
I'm not sure which XDG desktop portals you have specifically in mind, but I've assumed so far you were referring to file access. Was there something more specific to this PR you had in mind or were you asking more generally?

What i was saying is that although thonny uses xdg portals for file access in flatpaks due to the default mechanism of flatpak being built around xdg portals ,by default thonny does not uses xdg portals since in snap sandbox it continues to use the tk one, this is even after forcing it to use portals, thonny always uses its internally built less interactive yet working tk dialogs to access files, so i was thinking if we somehow force thonny to use xdg portals as its default globally, this was a suggestion or maybe a general question...

Ahh, I think that's an aside from this PR, but do you have zenity installed and on PATH in the snap? That's necessary to get the native GNOME file chooser dialog in Thonny.

I will try pushing new build with staging zenity, thanks for the tip as for the pr the snap thonny variant has already been registered for dbus by me with the id :- org.thonny-ide.thonny, but i later commented the dbus config in the snap yaml since i came to knew that thonny doesnt use dbus, so if dbus support lands in thonny in future i will be able to easily push that release through snap.

@aivarannamaa aivarannamaa merged commit ec36c76 into thonny:master Mar 26, 2023
@aivarannamaa
Copy link
Member

Thanks!

At first try I got a segfault crash on Ubuntu 22.04, when opening the back-end selector menu (which looks for volumes looking like MicroPython or CircuitPython UF2 installation volumes). I couldn't reproduce it later, though.

I'll leave it as is for now, but if I experience this crash again, I'll make mount to be preferred approach and use udisks as a fallback.

@aivarannamaa aivarannamaa added this to the 4.1.0 milestone Mar 26, 2023
DanielLowry added a commit to DanielLowry/thonny that referenced this pull request Apr 12, 2023
* Enhance esptool GUI, thonny#2707

* Windows fixes for esptool GUI, thonny#2707

* More forgiving image analysis for esptool GUI, thonny#2707

* black

* Misc fixes for esptool GUI, thonny#2707

* Use UDisks to find mounted filesystems on Linux (thonny#2683)

* Use UDisks to find volumes on Linux

This should generally work better on Linux than using the mount command directly.
UDisks is the utility that all major Linux desktop distributions use to handle mounting filesystems in userspace.
UDisks is accessible through a D-Bus API on the system bus.

D-Bus is complex and requires adding a dependency on a Python library for handling asynchronous communication with the bus.
I chose dbus-next since it is actively developed, modern, and doesn't require a glib main loop.
Since Thonny is not a Gtk application, it makes more sense to use native Python async functionality instead.

Inside a Flatpak, it's necessary to use UDisks via the D-Bus API to mount and check mounted filesystems.
The Flatpak sandbox prevents mounting otherwise and does not supply a mount command with the runtime.

* Smal fix for esptool GUI, thonny#2707

* Update CircuitPython links

* CircuitPython 8.1 stubs

* Small fixes for flasher dialogs, thonny#2707

* Update CREDITS.rst

* Restore MicroPython completions in the shell, thonny#2467, thonny#2419

* Rely on Jedi's builtins.pyi

* Fix MicroPython builtins completion

* Consider modules in project directory when completing

* Update MP links

* Refactor MP link collection

* Update MP links

* Require backend ACK before sending initial input

* Prepare for Python 3.12

* Update copyright year

* More Python version range fixes

* Simpler Python version display

* Move PROCESS_ACK

* Simpler Python interpreter selection

* Ensure Thonny's backends don't get confused by other installations, thonny#2616

* Alternative approach for thonny#2616

* Restore user site-packages for CPython backend, thonny#2616

* remove temp logging

* Experiment with module conflict detection, thonny#384

* Fixes for regressions caused by 4104729

---------

Co-authored-by: Aivar Annamaa <aivar.annamaa@gmail.com>
Co-authored-by: Jordan Williams <jordan@jwillikers.com>
@jwillikers jwillikers deleted the mount-with-udisks branch May 5, 2023 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

mount command is missing Could not find volume CIRCUITPY
2 participants
0