chore(deps): address hyper deprecations in policy controller #13493

cratelyn · 2024-12-18T02:12:53Z

NB: this branch is based upon #13492. see #8733 for more information about migrating to hyper 1.0.

this enables the backports and deprecated feature flags in the hyper dependencies in this project, and addresses warnings. see https://hyper.rs/guides/1/upgrading/ for more information about these feature flags.

largely, the control plane is unaffected by this upgrade, besides the following changes:

one usage of a deprecat 8000 ed hyper::body::aggregate function is updated.
a hyper::rt::Executor<E> implementation, which spawns tasks onto the tokio runtime, is provided. once we upgrade to hyper 1.0, we can replace this with the executor provided in hyper-util.
the hyper::service::Service<hyper::Request<tonic::body::BoxBody>> implementation for GrpcHttp now boxes its returned future, on account of SendRequest returning an anonymous impl Future<Output = ...>.
the policy-test additionally depends on the runtime feature of hyper. this is an artifact of an internal config structure shared by the legacy connection builder and the backported connection builder containing two keep-alive fields that were feature gated prior to 1.0.

this contains some small patches that we're interested in, before we address deprecations and upgrade to hyper 1.0. see #8733 for more information. Signed-off-by: katelyn martin <kate@buoyant.io>

NB: this branch is based upon #13492. see #8733 for more information about migrating to hyper 1.0. this enables the `backports` and `deprecated` feature flags in the hyper dependencies in this project, and addresses warnings. see <https://hyper.rs/guides/1/upgrading/> for more information about these feature flags. largely, the control plane is unaffected by this upgrade, besides the following changes: * one usage of a deprecated `hyper::body::aggregate` function is updated. * a `hyper::rt::Executor<E>` implementation, which spawns tasks onto the tokio runtime, is provided. once we upgrade to hyper 1.0, we can replace this with the executor provided in [`hyper-util`](https://docs.rs/hyper-util/latest/hyper_util/rt/tokio/struct.TokioExecutor.html#impl-Executor%3CFut%3E-for-TokioExecutor). * the `hyper::service::Service<hyper::Request<tonic::body::BoxBody>>` implementation for `GrpcHttp` now boxes its returned future, on account of `SendRequest` returning an anonymous `impl Future<Output = ...>`. * the `policy-test` additionally depends on the `runtime` feature of hyper. this is an artifact of an internal config structure shared by the legacy connection builder and the backported connection builder containing two keep-alive fields that were feature gated prior to 1.0. Signed-off-by: katelyn martin <kate@buoyant.io>

cratelyn · 2024-12-18T02:42:19Z

there's some feature flags to fight before i open this up for review. 🎏

cratelyn · 2024-12-18T21:17:48Z

ci was failing for this reason:

error: could not compile `hyper` (lib) due to 2 previous errors
Error: build failed
Error: error[E0609]: no field `keep_alive_interval` on type `h2::server::Config`
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-0.14.32/src/server/conn/http2.rs:199:25
    |
199 |         self.h2_builder.keep_alive_interval = interval.into();
    |                         ^^^^^^^^^^^^^^^^^^^ unknown field
    |
    = note: available fields are: `adaptive_window`, `initial_conn_window_size`, `initial_stream_window_size`, `max_frame_size`, `enable_connect_protocol` ... and 5 others


error: Recipe `_rs-check-dir` failed on line 102 with exit code 1
error: Recipe `rs-check-dirs` failed with exit code 1
Error: error[E0609]: no field `keep_alive_timeout` on type `h2::server::Config`
   --> /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-0.14.32/src/server/conn/http2.rs:213:25
    |
213 |         self.h2_builder.keep_alive_timeout = timeout;
    |                         ^^^^^^^^^^^^^^^^^^ unknown field
    |
    = note: available fields are: `adaptive_window`, `initial_conn_window_size`, `initial_stream_window_size`, `max_frame_size`, `enable_connect_protocol` ... and 5 others


Error: Process completed with exit code 1.

i've landed hyperium/hyper#3816 upstream to conditionally compile these interfaces in the backported http/2 server connection builder.

note: this commit will not compile, code changes are intentionally elided from this commit. this commit upgrades hyper, http, tonic, prost, related dependencies, and their assorted cargo features. see <linkerd/linkerd2#8733>. see also: * #3379 * #3380 * #3382 * #3405 * hyperium/hyper#3796 * #3411 * #3421 * #3427 * #3428 * #3432 * #3433 * #3444 * #3445 * #3454 * #3455 * #3456 * #3457 * #3461 * #3459 * #3465 * #3466 * #3467 * #3468 * linkerd/linkerd2-proxy-api#421 * linkerd/linkerd2#13492 * linkerd/linkerd2#13493 * hyperium/hyper#3816 * #3472 * #3473 * #3479 * tokio-rs/tokio#7059 * #3509 * hyperium/http-body#140 * #3515 * hyperium/http-body#141 * #3530 * #3531 * #3540 * #3556 * #3558 * #3559 * #3564 * #3567 * #3573 * #3583 * hyperium/http-body#144 * #3585 * #3586 * #3597 * #3598 * #3611 * #3614 * #3615 * #3616 * #3647 * #3651 * #3653 * #3654 * #3655 * #3656 * #3657 * #3660 * #3671 * #3672 * #3673 * #3676 * hyperium/http-body#147 * #3692 * #3699 * #3700 * #3701 * #3708 * linkerd/drain-rs#36 * #3715 * #3717 * eminence/procfs#340 --- squash: chore(deps): add hyper-util workspace dependency chore(deps): add http-body-util workspace dependency chore(deps): upgrade linkerd2-proxy-api this commit represents main as of linkerd/linkerd2-proxy-api#421. Signed-off-by: katelyn martin <kate@buoyant.io>

## ✨ chore(deps): upgrade to hyperium 1.x crates this branch performs an exciting upgrade for our proxy. this branch upgrades a number of our dependencies so that we use the 1.0 release family of the `hyper` http framework, and its ecosystem. see the [v1.0 announcement][hyper-v1] for more information. this branch upgrades the following dependencies: * `h2`: 0.3 -> 0.4 * `http`: 0.2 -> 1 * `http-body`: 0.4 -> 1 * `hyper`: 0.14.32 -> 1 * `prost`: 0.12 -> 0.13 * `prost-build`: 0.12 -> 0.13 * `prost-types`: 0.12 -> 0.13 * `tonic`: 0.10 -> 0.12 * `tonic-build`: 0.10 -> 0.12 a `hyper-util` dependency is added, which provides among other things, legacy-compatible interfaces such as `hyper_util::client::legacy::Client`, or glue to use `hyper` with the tokio runtime. see <https://docs.rs/hyper-util/latest/hyper_util/> for more information. a `http-body-util` dependency is added, which provides a `BodyExt` trait and a channel-backed body for use in unit tests. the `deprecated` feature flag that was active on our `0.14` hyper dependency has been removed, along with the `stream` and `runtime` feature flags. the `linkerd2-proxy-api` dependency is updated. see: <linkerd/linkerd2-proxy-api#421> ### 📝 notes for review bear particular attention to changes involving `http_body::Body` middleware. the change from two separate `poll_data()` and `poll_trailers()` functions, to a single `poll_frame()` method, induces some subtle changes to various pieces of middleware. also bear in mind that failing to set a timer, in our case `hyper_util::rt::TokioTimer`, can cause http/2 clients, or http/1 and http/2 servers, to panic. make sure that any uses of `hyper::server::conn::http1::Builder`, `hyper::client::conn::http1::Builder`, or `hyper::client::conn::http2::Builder` install a timer. ### ❗ breaking change: `l5d-proxy-error` values the `l5d-proxy-error` header can be examined to observe the cause of proxy errors encountered when sending meshed traffic. by virtue of this using a newer `hyper` client in the proxy, some error messages may in turn look different. for example, an error like `"connect timed out after 1s"` may now appear as `"client error (Connect)"`. ### 📚 other notes this work, by virtue of touching so many parts of the system, is carried out in distinct commits. an initial commit upgrades the dependencies at th workspace level. subsequent commits will not compile if the `--workspace` flag is provided, but the intent of this branch is to update each crate individually. use commands like, e.g. `cargo check --tests -p linkerd-proxy-http` to build particular crates at intermediate commits within this branch. this commit is also only the final leaf in an _extended_ line of work. this has been done to mitigate the effort of reviewing this change, and the risk of churn in the event of any unanticipated errors. see the top-level comment in linkerd/linkerd2#8733 for an overview of all of the work that brought us to this juncture. [hyper-v1]: https://seanmonstar.com/blog/hyper-v1/ --- * chore(deps): upgrade to hyper 1.x note: this commit will not compile, code changes are intentionally elided from this commit. this commit upgrades hyper, http, tonic, prost, related dependencies, and their assorted cargo features. see <linkerd/linkerd2#8733>. see also: * #3379 * #3380 * #3382 * #3405 * hyperium/hyper#3796 * #3411 * #3421 * #3427 * #3428 * #3432 * #3433 * #3444 * #3445 * #3454 * #3455 * #3456 * #3457 * #3461 * #3459 * #3465 * #3466 * #3467 * #3468 * linkerd/linkerd2-proxy-api#421 * linkerd/linkerd2#13492 * linkerd/linkerd2#13493 * hyperium/hyper#3816 * #3472 * #3473 * #3479 * tokio-rs/tokio#7059 * #3509 * hyperium/http-body#140 * #3515 * hyperium/http-body#141 * #3530 * #3531 * #3540 * #3556 * #3558 * #3559 * #3564 * #3567 * #3573 * #3583 * hyperium/http-body#144 * #3585 * #3586 * #3597 * #3598 * #3611 * #3614 * #3615 * #3616 * #3647 * #3651 * #3653 * #3654 * #3655 * #3656 * #3657 * #3660 * #3671 * #3672 * #3673 * #3676 * hyperium/http-body#147 * #3692 * #3699 * #3700 * #3701 * #3708 * linkerd/drain-rs#36 * #3715 * #3717 * eminence/procfs#340 --- squash: chore(deps): add hyper-util workspace dependency chore(deps): add http-body-util workspace dependency chore(deps): upgrade linkerd2-proxy-api this commit represents main as of linkerd/linkerd2-proxy-api#421. Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/box): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(hyper-balance): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/retain): ugrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/stream-timeouts): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/classify): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/upgrade): upgrade to hyper 1.x NOTE: there is a comment noting that the upgrade middleware does not expect to be cloneable. it is unfortunately, however, at odds with the new bounds expected of extensions. so, `Http11Upgrade` is now Clone'able, but a comment is left in place noting this weakened invariant. it's worth investigating how upgrades have changed since, in more detail, but for the current moment we are interested in being especially conservative about changing behavior, and focusing on api changes like `Body::poll_frame(..)`. Signed-off-by: katelyn martin <kate@buoyant.io> * chore(metrics): upgrade to hyper 1.x a brief note; this commit happened to tickle an unfortunate sharp edge in `BoxBody` and `Full`'s respective constructors. type inference could not figure out how to construct the body, so we refrain from boxing the response body now. Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/metrics): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/prom): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/insert): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/retry): deprecate linkerd-http-body-compat Signed-off-by: katelyn martin <kate@buoyant.io> * chore(mock/http-body): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/retry): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(proxy/tap): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(proxy/http): update to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(app/core): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(app/test): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(app/admin): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(app/outbound): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(app/inbound): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(app/integration): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(app): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(transport-header): update generated code Signed-off-by: katelyn martin <kate@buoyant.io> * chore(spiffe-proto): update generated code Signed-off-by: katelyn martin <kate@buoyant.io> * chore(opencensus-proto): update generated code Signed-off-by: katelyn martin <kate@buoyant.io> * chore(opentelemetry-proto): update generated code Signed-off-by: katelyn martin <kate@buoyant.io> * chore(deny.toml): update cargo-deny directives this commit updates the contents of `deny.toml`. Signed-off-by: katelyn martin <kate@buoyant.io> * chore: `compile` has been renamed to `compile_protos` this addresses deprecation warnings, updating calls to a function that has since been renamed. Signed-off-by: katelyn martin <kate@buoyant.io> * chore(deps): remove `linkerd-http-body-compat` dependencies this commit removes this crate, which we added to future proof code for this upgrade, from its dependents. Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/body-compat): remove `linkerd-http-body-compat` cra 6068 te Signed-off-by: katelyn martin <kate@buoyant.io> * chore(deps): update to drain 0.2.1 see linkerd/drain-rs#41. Signed-off-by: katelyn martin <kate@buoyant.io> --------- Signed-off-by: katelyn martin <kate@buoyant.io>

cratelyn mentioned this pull request Dec 18, 2024

proxy: Figure out the plan for Hyper 1.0 migration #8733

Closed

cratelyn force-pushed the kate/hyper-1.x-upgrade branch from 33432fe to f68339e Compare December 18, 2024 02:24

chore(deps): update from hyper 0.14.28 to 0.14.32

f5b4d45

this contains some small patches that we're interested in, before we address deprecations and upgrade to hyper 1.0. see #8733 for more information. Signed-off-by: katelyn martin <kate@buoyant.io>

cratelyn force-pushed the kate/hyper-1.x-upgrade branch from f68339e to f6468a9 Compare December 18, 2024 02:27

cratelyn force-pushed the kate/hyper-1.x-upgrade branch from f6468a9 to 36c17fb Compare December 18, 2024 02:33

cratelyn changed the title ~~chore(deps): address hyper deprecations in policy controllers~~ chore(deps): address hyper deprecations in policy controller Dec 18, 2024

cratelyn closed this Mar 11, 2025

cratelyn deleted the kate/hyper-1.x-upgrade branch March 20, 2025 15:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

chore(deps): address hyper deprecations in policy controller #13493

chore(deps): address hyper deprecations in policy controller #13493

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chore(deps): address hyper deprecations in policy controller #13493

chore(deps): address hyper deprecations in policy controller #13493

Uh oh!

Conversation

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!