Make a basic, public API #1696

kaste · 2025-05-09T13:30:23Z

For the basic functionality, installing, removing, and disabling and enabling
packages, provide an API. (IMO, we generally should provide an API
to open things up).

E.g.

def install_or_upgrade_packages(packages: list[str], progress: ActivityIndicator = None, unattended: bool = True):
def remove_packages(packages: list[str]):
def disable_packages(packages: list[str]):
def enable_packages(packages: list[str]):

(Maybe, scratch that progress arg as it would be nicer if we could solve this differently.)

For install_or_upgrade consider this implementation or something among this:

def install_or_upgrade(packages: list[str], progress: ActivityIndicator, unattended: bool = True):
    upgrader = PackageTaskRunner()
    tasks = upgrader.create_package_tasks(
        actions=(upgrader.INSTALL, upgrader.UPGRADE, upgrader.DOWNGRADE, upgrader.REINSTALL),
        include_packages=packages,
        ignore_packages=upgrader.ignored_packages()  # don't upgrade disabled packages
    )
    to_install, to_upgrade = [], []
    for task in tasks:
        if task.action == upgrader.INSTALL:
            to_install.append(task)
        elif task.action in (upgrader.UPGRADE, upgrader.DOWNGRADE, upgrader.REINSTALL):
            to_upgrade.append(task)

    if to_install:
        upgrader.run_install_tasks(to_install, progress, unattended)
    if to_upgrade:
        upgrader.run_upgrade_tasks(to_upgrade, progress, unattended)

Note that type signature matches the high-level ones of e.g. InstallPackagesCommand.
The difference is the layering -- InstallPackagesCommand et.al. would use in fact install_or_upgrade
under hood -- and that the functions are the blocking implementations. T.i.: the API
functions don't fall through and do some or all of their work after returning.

That is their main advantage and the driving motivator.

The text was updated successfully, but these errors were encountered:

deathaxe · 2025-05-09T13:51:32Z

TBH, I don't want to open up this door.

deathaxe · 2025-05-09T13:52:31Z

PC 4 already added various commands to support all relevant actions in a plugin_host / python version agnostic way and that's enough.

kaste · 2025-05-09T14:07:03Z

But it is the same feature set as the commands, just that these are functions. Commands are ideally bound to keys, the command palette or menus. These functions would drive the 8000 commands. Basically entangling the two concerns: the feature from the way we register features within ST.

These don't have to be blocking. Maybe PC should control the execution/executors. I.e. practically only one executor thread/queue is allowed here as I can't think of doing these commands or functions parallelized in the current shape. And if parts of it could run in parallel, PC had to orchestrate this probably. So there is the option (a) to provide a callback as an arg or (b) to return a Future.

deathaxe · 2025-05-09T14:36:26Z

It was already challanging to get a working transition for "events" api while transitioning from py33 to 38 as various packages kept running or keep running on 3.3 while PC is on 3.8. All direct imports wouldn't work then.

See how all those packages using direct python functions failed.

I just don't want to maintain those or struggle with them, nor is it the goal of PC to be a library or platform for other packages.

kaste · 2025-05-09T15:01:54Z

I don't see why closed software is better than open software. I also don't see how that makes PC a library, it is esp. about using this software.

However, I don't mind calling "install_packages" as a command. That would be fine, it is just that this falls through, and I don't know when PC is done with it. Can you think of some other ways to support that usage?

Didn't we yesterday talked about that ST should provide an API to disable a plugin? We also didn't want to use that in a library-way, we just want a direct way and esp. want to know when it's done.

deathaxe · 2025-05-09T16:06:07Z

The design goal is to provide a GUI and functions to install/upgrade/remove packages.

It is a nongoal to be used by other packages beyond using package_control.events.

All those APIs start immediatelly causing issues or fail as soon as PC decides to switch to asyncio.

Use packagecontrol functionality if you want, but without it being supported I don't need to care about breaking changes, which I don't want to.

kaste · 2025-05-09T16:07:00Z

Let's think about this differently. For all commands/actions, we typically
create and start a new thread.

Searching 274 files for "threading.Thread" (whole word), 14 matches across 11 files

C:\Users\c-flo\AppData\Roaming\Sublime Text\Packages\Package Control\package_control\package_cleanup.py:
   20: class PackageCleanup(threading.Thread, PackageTaskRunner):
   43:         threading.Thread.__init__(self)

C:\Users\c-flo\AppData\Roaming\Sublime Text\Packages\Package Control\package_control\commands\install_package_command.py:
   48:                 threading.Thread(target=worker, args=[tasks[picked]]).start()
   56:         threading.Thread(target=show_quick_panel).start()

C:\Users\c-flo\AppData\Roaming\Sublime Text\Packages\Package Control\package_control\commands\install_packages_command.py:
   41:             threading.Thread(target=worker).start()

C:\Users\c-flo\AppData\Roaming\Sublime Text\Packages\Package Control\package_control\commands\remove_package_command.py:
   62:         threading.Thread(target=worker).start()

C:\Users\c-flo\AppData\Roaming\Sublime Text\Packages\Package Control\package_control\commands\remove_packages_command.py:
   32:             threading.Thread(target=worker).start()

C:\Users\c-flo\AppData\Roaming\Sublime Text\Packages\Package Control\package_control\commands\revert_package_command.py:
   62:         threading.Thread(target=worker).start()

C:\Users\c-flo\AppData\Roaming\Sublime Text\Packages\Package Control\package_control\commands\satisfy_libraries_command.py:
   53:         threading.Thread(target=satisfy).start()

C:\Users\c-flo\AppData\Roaming\Sublime Text\Packages\Package Control\package_control\commands\satisfy_packages_command.py:
   29:         threading.Thread(target=worker).start()

C:\Users\c-flo\AppData\Roaming\Sublime Text\Packages\Package Control\package_control\commands\upgrade_all_packages_command.py:
   30:         threading.Thread(target=worker).start()

C:\Users\c-flo\AppData\Roaming\Sublime Text\Packages\Package Control\package_control\commands\upgrade_package_command.py:
   57:                 threading.Thread(
   82:         threading.Thread(target=show_quick_panel).start()

C:\Users\c-flo\AppData\Roaming\Sublime Text\Packages\Package Control\package_control\commands\upgrade_packages_command.py:
   41:             threading.Thread(target=worker).start()

14 matches across 11 files

Likely this can lead two unwanted concurrent execution. For example: you probably
can't actually run two "install_packages_command" invocations at the same time.
You can't install and remove it at the same time, and so on.

Let's create a custom executor that, based on a queue, runs such core functionality
in a serial manner. Add a function is_idle/is_busy to tell if the PC is actually
doing something right now.

The spinner would be actually a view and listener of this executor state. We had
a spinner as long as the executor is busy, and stop it if the queue gets empty.

The executor holds another state, namely what it is doing right now. These are
the text messages that the spinner could show. (Currently named label iirc.)

All users of the executor could decide if they even want to schedule something
because of possible backpressure. All users could also see if PC is actually doing
something at the moment. Still, it would be a safe, closed system. In theory,
the executor could re-arrange all tasks and optimize them. But that is really next-level...

kaste · 2025-05-09T16:08:06Z

Dang, again you posted while I was writing. 🍭

deathaxe · 2025-05-09T16:13:50Z

Without running those commands in threads UI would block.

The custom executor will most likely be an asyncio loop.

And again we are about fundamental design changes.

Primary use case (besides automatic updater and cleanup tasks at startup) is to open a command palette select a package and do something with that action. It's unlikely someone opens dozens of panels at the same time starting dozens of single install packages at the same time.

All of which changes when opening up things to plugins.

kaste · 2025-05-09T16:26:25Z

All of which changes when opening up things to plugins.

That is true. In the OP I already fell over the progress arg. So I completely changed the approach in the last p 5AFF ost. I could offer to implement the executor. It doesn't matter if all of this runs later on async/await. The usage here is actually that these tasks should run ordered. Other bits can run in parallel but here at the high level, the order matters. E.g. you can't shuffle remove and install.

The interesting bit is that e.g. ActivityIndicator then becomes a function of the executor. So you don't have the with progress_bar wrapping everywhere.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make a basic, public API #1696

Make a basic, public API #1696

Make a basic, public API #1696

Make a basic, public API #1696

Comments