Closed
Description
We propose to vendor cri-containerd into containerd as a native plugin.
Current Progress
- containerd can add CRI plugin by adding following line in
cmd/containerd/builtin_linux.go:
import _ github.com/containerd/cri-containerd- cri-containerd grpc services are registered on containerd grpc server. Use containerd grpc server cri#552
- cri-containerd uses containerd plugin config Use containerd config cri#557
- cri-containerd switched to use
logrusfor logging. glog to logrus cri#547 - cri-containerd can be gracefully stopped when containerd is gracefully stopped. Update containerd and leverage plugin graceful stop. cri#567
- cri-containerd can be disabled with containerd config. Support disable plugins. #2105
The PR to vendor cri-containerd into containerd is under review #2097.
Overhead
- Memory usage without workload.
containerd:
VmHWM: 25904 kB
VmRSS: 25904 kB
RssAnon: 10420 kB
RssFile: 15484 kB
containerd+cri:
VmHWM: 46104 kB
VmRSS: 46104 kB
RssAnon: 23872 kB
RssFile: 22232 kB
- Containerd binary size:
containerd: 22.77 mB
containerd+cri: 41.00 mB
Next Steps
- Vendor cri-containerd into containerd as a native plugin (Vendor cri plugin into containerd. #2097).
- Get rid of the GRPC between CRI plugin and containerd. (Support in process integration between containerd and CRI plugin cri#645)
- Switch all CI test in cri-containerd repo to test against CRI plugin mode. (Remove standalone mode cri#639)
- Merge
ctrcriintoctr crisubcommand. (Issue: Addctr criand retirectrcricri#590, PR: Plug-in cri subcommand #2175) - Get new benchmark data for
containerd+cri. new benchmark data is from build 547 http://node-perf-dash.k8s.io/#/builds. CPU usage of containerd reduced ~0.01cores, rss memory usage reduced ~5mb. Actually there is no significant performance boost. - Setup CI cluster/node e2e for containerd repo in Kubernetes test-infra. We are recently working on Kubernetes container runtime test policy, containerd should be able to have post submit test in Kubernetes test infra (Test containerd repo cri#673, Add CI node e2e and e2e test for containerd repo. kubernetes/test-infra#7315) (
containerd-build,containerd-node-e2e,containerd-e2e-gciandcontainerd-e2e-ubuntuare setup for containerd repo https://k8s-testgrid.appspot.com/sig-node-containerd) - Rename
containerd/cri-containerdtocontainerd/criand update all the documents. And deprecate standalonecri-containerdsupport. (Renamecri-containerdtocricri#660) - Decide which packages we don't want in containerd and rewrite them. (Moving to use go-cni library from containerd cri#663)
- Versioning CRI. kubernetes/kubernetes@e10042d#diff-a31c74e5ec30d171eb16f2d9496577fc
Open Questions
How to test CRI plugin in cri-containerd repo?- Option 1: Keep standalone
cri-containerdmode, and test against that mode.- Pros: Easy.
- Cons:
- It is an overhead to maintain the standalone mode.
- Standalone
cri-containerd+containerd!=containerd+cri. There might still be surprise when we actually update cri plugin in containerd.
- Option 2: Automatically update containerd code with script. Automatically replace old CRI plugin code with new CRI plugin code.
- Pros: No interface change on neither side.
- Cons: We need complex and ugly script to automatically do this, the script will also be very hard to maintain.
- Option 3: Build cri-containerd in plugin mode, and test the plugin binary with containerd.
- Pros: No interface change on neither side.
- Cons: This doesn't work cmd/containerd: split pkg for ease of 3rd party distribution #1615 (comment).
- Option 4: Provide CLI package in containerd, and cri-containerd vendor that CLI package and build binary for testing.
- Pros: Easy and clear.
- Cons: Extra maintenance overhead for containerd, and potential community split.
- Option 5: Add RPC based containerd plugin interface.
- Cons: Extra complexity.
- Option 1: Keep standalone
Any questions, suggestions and feedback are welcome.
@containerd/containerd-maintainers @containerd/containerd-reviewers
Metadata
Metadata
Assignees
Labels
No labels