Proposal: Add support for using CLI plugins #13140
Conversation
|
@duglin So, this is like adding new commands to docker on-the-fly? |
|
@LK4D4 kind of. It allows for new docker cli subcommands to be defined w/o needing to changing the docker exe itself. As an example, we have a need to add new commands to manage networks, volumes and even logins. We'd prefer if it were all under one exe for a nice user experience. In some cases the plugin will generate REST calls to the Docker daemon (and it can do so by leveraging the Docker cli call() func) and in some cases it may end up talking to some other endpoint. There are other interesting usecases where people may want to add their own commands that are groupings of other docker commands. Not sure if its the best example, but I think someone once said that "squash" can be done today with existing commands (I think via export/import or perhaps it was save/load, can't recall), anyway, the point is someone might be able to fill a gap in their usage of docker by defining a new subcmd that could do this w/o forcing their users to manually script the solution themselves. In the end, its all about exposing an extension point and giving the community an opportunity to explore new ways that docker can help them. I can envision us creating a page that references popular (community-generated) plugins so that people can easily share and discover what other people have done. |
|
Idea is ok for me. Don't understand implementation though. Plugins talking on named pipes by some protocol, written by you? |
|
Yes sorry for not going into more detail on that - I'm hoping to write-up more on that later today/tomorrow. But yes, it uses a very simple protocol I wrote. I've seen other plugin solutions use http and I wouldn't necessarily mind using that other than it seemed too limiting to me. In particular, I needed either side to initiate the conversation. On the positive side though, as you'll be able to see when I write it down, its very similar to http in that it has a command (think http url), some metadata (think http headers) and then binary data (think http body). The core plugin code itself (like http) doesn't try to understand what is being sent, it just manages the transfer of the data between two endpoints. The code that uses the net.go code is the bit that tries to make sense of what is actually being transferred. |
|
@LK4D4 I updated the docs in the plugin repo (https://github.com/duglin/plugin) . That might help explain the details of how it works. But at this point I'm more interested in whether we can get the concept of CLI plugins considered because I think this is a pretty important extensibility point for:
IMO, this is a perfect usecase to test out the new "experimental channel" #13023. /ping @shykes |
|
@vbatts may be interested in this as well, it reminds me of #5001 (comment) and #5001 (comment) |
|
I am game for the nature of this. Whether it be your plugin library, libchan, or something else. When @shykes mentions splitting up aspects of the daemon, it seems like the perfect fit. Just like |
|
I think I lean towards not using a hand made protocol. Regardless it's simplicity, it's something that I don't want us to be worrying about. |
|
@calavera I'm less worried about any particular protocol right now. I think we need to first get agreement on whether we want to support CLI plugins at all. Having said that, whatever protocol we use we need one 8000 that allows either side to initiate a conversation. |
|
I would love to have the ability to extend the docker client with plugins. It'd be a great way to add image visualization back in for those who find it useful. |
|
I just added 3 new plugins to: https://github.com/duglin/plugin and |
Nice touch! I think this would be a great way to implement features that aren't desired as "core" features, but requested by many as well. ping @aanand @bfirsh I remember the original discussion around "compose" being part of the core or not; I think adding it as a plugin was considered then as well, so you may like this :) |
|
@thaJeztah re: |
|
Don't forget Digging my memory, in a prior proposal for plugins, containers were suggested as a delivery mechanism for plugins; #8968 (comment) would that be an option with the approach taken here? |
|
@thaJeztah daemon-side plugins could be run in container, but not CLI plugins because not everyone will have a daemon running in the CLI env. Plus, at some point the plugins may do more than just add new commands, they may be invoked at certain extensibility points in the CLI's normal flow. And if someone wanted to inject something into all flows to the daemon you get into a bit of a catch-22 (or recursive?) situation. |
Doh! Forgot this is client side for a moment (was looking for opinions/ideas in earlier plugin proposals) |
Signed-off-by: Doug Davis <dug@us.ibm.com>
|
Collective review with @jfrazelle @crosbymichael @calavera Can we please wait for Go 1.5 and native plugin support to look into this? |
|
What's the ETA for Docker's move to go 1.5 ? |
|
I could imagine asap, like it has been with so many golang versions. :-)
|
|
Do you have a link to the specific golang feature you're thinking we could use? |
|
@cpuguy83 thanks for the pointer! |
|
We really don't know what to do on the cli side right now for plugins. There are a couple of competing proposals and with the Go 1.5 changes it really introduces a good way to extend Go programs. Lets wait to see what Go provides to us and if it is insufficient then we can revisit your work done in this PR. |
This PR is the start of a plugin model for the CLI.
It uses the plugin infrastructure from https://github.com/duglin/plugin .
The idea is that all cli plugins (exes found in ~/.docker/plugins/cli) are loaded upload CLI startup. hey are then visible from the
docker helpoutput (e.g.pingsample from https://github.com/duglin/plugin ). Thendocker pingwill then invoke thepingexe to do its job. Thepingexecutable can then call back into the CLI to ask it info or to issue a command to the daemon. The CLI can (and does) ask the plugin for info - like its metadata to display on thedocker helpcommand.It supports multi-threaded, bi-directional, communications between the two exes. See the "sample1" sample in https://github.com/duglin/plugin for an example of 5000 parallel main->plugin requests, and 5000 plugin->main requests, all being executed at the same time.
I would suggest people grab the
pingDocker plugin sample to see it in action and to see how a plugin can call the daemon (GET /infoin this case) via the CLI.I didn't add any docs yet, and I'm sure more clean-up is needed, but I wanted to see if this direction seems reasonable before I invested too much more time in it.
While it just deals with CLI plugins, I don't see any reason why the basic infrastructure couldn't be used for daemon-side plugins too - but I'm not sure I understand all of the requirements over there, so that's still TBD.
Ideally, I'd like to see this in v1.7 as "experimental" but I'm sure lots more discussions are needed and this PR is meant just to get those chats started.
Signed-off-by: Doug Davis dug@us.ibm.com