chore(cargo.toml): define http workspace dependencies #3466

cratelyn · 2024-12-16T15:19:06Z

this is a follow-up to #3456, addressing a suggestion that was made
during review:

this branch modifies the workspace manifest, defining http and http-body
as common workspace dependencies.

no changes to the lockfile are made because these commits do not affect
the dependency graph of the project.

this is a follow-up to #3456, addressing a suggestion that was made during review: - <#3456 (review)> - <linkerd/linkerd2#8733> this commit modifies the workspace manifest, defining http and http-body as common workspace dependencies. no changes to the lockfile are made because this commit does not affect the dependency graph of the project. Signed-off-by: katelyn martin <kate@buoyant.io>

this is an incremental step away from the 0.14 request and response body interfaces, and towards the 1.0 body types. see <linkerd/linkerd2#8733> for more information about upgrading to hyper 1.0. hyper 0.14 provides a `hyper::body::Body` that is removed in the 1.0 interface. `hyper-util` now provides a workable default body type. hyper 0.14 reëxports `http_body::Body` as `HttpBody`. hyper 1.0 reëxports this trait as `hyper::body::Body` without any renaming. in order to move away from hyper's legacy `Body` type, this commit moves assorted interfaces towards the boxed `BoxBody` type instead. when possible, code is tweaked such that it refers to the reëxport in `linkerd-proxy-http`, rather than directly through `hyper`. NB: this commit is based upon #3466. Signed-off-by: katelyn martin <kate@buoyant.io>

this is an incremental step away from hyper 0.14's request and response body interfaces, and towards the 1.0 body types. see <linkerd/linkerd2#8733> for more information about upgrading to hyper 1.0. hyper 0.14 provides a `hyper::body::Body` that is removed in the 1.0 interface. `hyper-util` now provides a workable default body type. hyper 0.14 reëxports `http_body::Body` as `HttpBody`. hyper 1.0 reëxports this trait as `hyper::body::Body` without any renaming. this commit moves application code away from hyper's legacy `Body` type and the `HttpBody` trait alias. this commit moves assorted interfaces towards the boxed `BoxBody` type instead. when possible, code is tweaked such that it refers to the reëxport in `linkerd-proxy-http`, rather than directly through `hyper`. NB: this commit is based upon #3466. Signed-off-by: katelyn martin <kate@buoyant.io>

zaharidichev

LGTM

this is an incremental step away from hyper 0.14's request and response body interfaces, and towards the 1.0 body types. see <linkerd/linkerd2#8733> for more information about upgrading to hyper 1.0. hyper 0.14 provides a `hyper::body::Body` that is removed in the 1.0 interface. `hyper-util` now provides a workable default body type. hyper 0.14 reëxports `http_body::Body` as `HttpBody`. hyper 1.0 reëxports this trait as `hyper::body::Body` without any renaming. this commit moves application code away from hyper's legacy `Body` type and the `HttpBody` trait alias. this commit moves assorted interfaces towards the boxed `BoxBody` type instead. when possible, code is tweaked such that it refers to the reëxport in `linkerd-proxy-http`, rather than directly through `hyper`. NB: this commit is based upon #3466. Signed-off-by: katelyn martin <kate@buoyant.io>

@sfleen

* refactor: move away from legacy `hyper::body::HttpBody` this is an incremental step away from hyper 0.14's request and response body interfaces, and towards the 1.0 body types. see <linkerd/linkerd2#8733> for more information about upgrading to hyper 1.0. hyper 0.14 provides a `hyper::body::Body` that is removed in the 1.0 interface. `hyper-util` now provides a workable default body type. hyper 0.14 reëxports `http_body::Body` as `HttpBody`. hyper 1.0 reëxports this trait as `hyper::body::Body` without any renaming. this commit moves application code away from hyper's legacy `Body` type and the `HttpBody` trait alias. this commit moves assorted interfaces towards the boxed `BoxBody` type instead. when possible, code is tweaked such that it refers to the reëxport in `linkerd-proxy-http`, rather than directly through `hyper`. NB: this commit is based upon #3466. Signed-off-by: katelyn martin <kate@buoyant.io> * feat(http/box): `BoxBody::from_static` constructor this commit relates to review feedback offered here: #3467 (comment) it is slightly ungainly to place a static string into a BoxBody, something that is a fairly common pattern throughout e.g. our admin server. to help nudge the compiler to infer a `B: Body` of `String`, various code follows a common convention of calling e.g. `BoxBody::new::<String>("ready\n".into())` or, `BoxBody::new("ready\n".to_string())`. this commit helps reduce that boilerplate by adding a `from_static(..)` constructor that accepts a static string. ```rust /// Returns a [`BoxBody`] with the contents of a static string. pub fn from_static(body: &'static str) -> Self { Self::new(body.to_string()) } ``` Co-authored-by: Scott Fleener <scott@buoyant.io> Signed-off-by: katelyn martin <kate@buoyant.io> * refactor(app/admin): outline json bytes body construction see <#3467 (comment)>. @sfleen points out this unfortunate part of our diff: ```diff - .body(json.into()) + .body(BoxBody::new(http_body::Full::new(bytes::Bytes::from(json)))) ``` this *is* a bit of an unfortunate edge, where boxing up a body feels especially cumbersome. this commit takes an attempt at tweaking the function in question so that this large nested expression reads a bit nicer. first, to justify why this gets a little more involved: hyper will no longer provide the `Body` type after 1.0, so we are tasked with providing our own body. for our purposes, `Full` works because we have a single chunk of bytes in hand. in order to create a `Full`, we must provide a `D: Buf`, which can be satisfied by the following types: <https://docs.rs/bytes/1.6.0/bytes/buf/trait.Buf.html#foreign-impls> most simply, we can cast our vector of bytes into a `Bytes`. with all of this in hand, we can hoist this creation of the body up out of the `match` arm, and use `Result::map` to apply these constructors in sequential order: ```rust // Serialize the value into JSON, and then place the bytes in a boxed response body. let json = serde_json::to_vec(val) .map(Bytes::from) .map(http_body::Full::new) .map(BoxBody::new); ``` Signed-off-by: katelyn martin <kate@buoyant.io> --------- Signed-off-by: katelyn martin <kate@buoyant.io> Co-authored-by: Scott Fleener <scott@buoyant.io>

see linkerd/linkerd2#8733 for more information. this commit moves `prost-build` so that it is now managed as a workspace dependency. while only used in tests, these tests can fail if this is not versioned in lockstep with our other protobuffer dependencies. see #3456 (c740b6d), #3466 (ca50d6b), and especially #3473 (b87455a) for some other previous pr's that moved dependencies to be managed at the workspace level. Signed-off-by: katelyn martin <kate@buoyant.io>

see linkerd/linkerd2#8733 for more information. see #3504 as well. see #3456 (c740b6d), #3466 (ca50d6b), #3473 (b87455a), and #3701 (cf4ef39) for some other previous pr's that moved dependencies to be managed at the workspace level. see also linkerd/drain-rs#36 for another related pull request that relates to our tower dependency. Signed-off-by: katelyn martin <kate@buoyant.io>

* chore(deps): `tower` is a workspace dependency see linkerd/linkerd2#8733 for more information. see #3504 as well. see #3456 (c740b6d), #3466 (ca50d6b), #3473 (b87455a), and #3701 (cf4ef39) for some other previous pr's that moved dependencies to be managed at the workspace level. see also linkerd/drain-rs#36 for another related pull request that relates to our tower dependency. Signed-off-by: katelyn martin <kate@buoyant.io> * chore(deps): `tower-service` is a workspace dependency Signed-off-by: katelyn martin <kate@buoyant.io> * chore(deps): `tower-test` is a workspace dependency Signed-off-by: katelyn martin <kate@buoyant.io> --------- Signed-off-by: katelyn martin <kate@buoyant.io>

see linkerd/linkerd2#8733 and #3504 for more information. see also linkerd/drain-rs#36 for a related pull request that updated `drain`'s `tower` dependency. see #3715 (2f97549) for a related pull request that made `tower` a workspace dependency. see #3456 (c740b6d), #3466 (ca50d6b), #3473 (b87455a), and #3701 (cf4ef39), for some other previous pr's that moved dependencies to be managed at the workspace level. Signed-off-by: katelyn martin <kate@buoyant.io>

note: this commit will not compile, code changes are intentionally elided from this commit. this commit upgrades hyper, http, tonic, prost, related dependencies, and their assorted cargo features. see <linkerd/linkerd2#8733>. see also: * #3379 * #3380 * #3382 * #3405 * hyperium/hyper#3796 * #3411 * #3421 * #3427 * #3428 * #3432 * #3433 * #3444 * #3445 * #3454 * #3455 * #3456 * #3457 * #3461 * #3459 * #3465 * #3466 * #3467 * #3468 * linkerd/linkerd2-proxy-api#421 * linkerd/linkerd2#13492 * linkerd/linkerd2#13493 * hyperium/hyper#3816 * #3472 * #3473 * #3479 * tokio-rs/tokio#7059 * #3509 * hyperium/http-body#140 * #3515 * hyperium/http-body#141 * #3530 * #3531 * #3540 * #3556 * #3558 * #3559 * #3564 * #3567 * #3573 * #3583 * hyperium/http-body#144 * #3585 * #3586 * #3597 * #3598 * #3611 * #3614 * #3615 * #3616 * #3647 * #3651 * #3653 * #3654 * #3655 * #3656 * #3657 * #3660 * #3671 * #3672 * #3673 * #3676 * hyperium/http-body#147 * #3692 * #3699 * #3700 * #3701 * #3708 * linkerd/drain-rs#36 * #3715 * #3717 * eminence/procfs#340 --- squash: chore(deps): add hyper-util workspace dependency chore(deps): add http-body-util workspace dependency chore(deps): upgrade linkerd2-proxy-api this commit represents main as of linkerd/linkerd2-proxy-api#421. Signed-off-by: katelyn martin <kate@buoyant.io>

## ✨ chore(deps): upgrade to hyperium 1.x crates this branch performs an exciting upgrade for our proxy. this branch upgrades a number of our dependencies so that we use the 1.0 release family of the `hyper` http framework, and its ecosystem. see the [v1.0 announcement][hyper-v1] for more information. this branch upgrades the following dependencies: * `h2`: 0.3 -> 0.4 * `http`: 0.2 -> 1 * `http-body`: 0.4 -> 1 * `hyper`: 0.14.32 -> 1 * `prost`: 0.12 -> 0.13 * `prost-build`: 0.12 -> 0.13 * `prost-types`: 0.12 -> 0.13 * `tonic`: 0.10 -> 0.12 * `tonic-build`: 0.10 -> 0.12 a `hyper-util` dependency is added, which provides among other things, legacy-compatible interfaces such as `hyper_util::client::legacy::Client`, or glue to use `hyper` with the tokio runtime. see <https://docs.rs/hyper-util/latest/hyper_util/> for more information. a `http-body-util` dependency is added, which provides a `BodyExt` trait and a channel-backed body for use in unit tests. the `deprecated` feature flag that was active on our `0.14` hyper dependency has been removed, along with the `stream` and `runtime` feature flags. the `linkerd2-proxy-api` dependency is updated. see: <linkerd/linkerd2-proxy-api#421> ### 📝 notes for review bear particular attention to changes involving `http_body::Body` middleware. the change from two separate `poll_data()` and `poll_trailers()` functions, to a single `poll_frame()` method, induces some subtle changes to various pieces of middleware. also bear in mind that failing to set a timer, in our case `hyper_util::rt::TokioTimer`, can cause http/2 clients, or http/1 and http/2 servers, to panic. make sure that any uses of `hyper::server::conn::http1::Builder`, `hyper::client::conn::http1::Builder`, or `hyper::client::conn::http2::Builder` install a timer. ### ❗ breaking change: `l5d-proxy-error` values the `l5d-proxy-error` header can be examined to observe the cause of proxy errors encountered when sending meshed traffic. by virtue of this using a newer `hyper` client in the proxy, some error messages may in turn look different. for example, an error like `"connect timed out after 1s"` may now appear as `"client error (Connect)"`. ### 📚 other notes this work, by virtue of touching so many parts of the system, is carried out in distinct commits. an initial commit upgrades the dependencies at th workspace level. subsequent commits will not compile if the `--workspace` flag is provided, but the intent of this branch is to update each crate individually. use commands like, e.g. `cargo check --tests -p linkerd-proxy-http` to build particular crates at intermediate commits within this branch. this commit is also only the final leaf in an _extended_ line of work. this has been done to mitigate the effort of reviewing this change, and the risk of churn in the event of any unanticipated errors. see the top-level comment in linkerd/linkerd2#8733 for an overview of all of the work that brought us to this juncture. [hyper-v1]: https://seanmonstar.com/blog/hyper-v1/ --- * chore(deps): upgrade to hyper 1.x note: this commit will not compile, code changes are intentionally elided from this commit. this commit upgrades hyper, http, tonic, prost, related dependencies, and their assorted cargo features. see <linkerd/linkerd2#8733>. see also: * #3379 * #3380 * #3382 * #3405 * hyperium/hyper#3796 * #3411 * #3421 * #3427 * #3428 * #3432 * #3433 * #3444 * #3445 * #3454 * #3455 * #3456 * #3457 * #3461 * #3459 * #3465 * #3466 * #3467 * #3468 * linkerd/linkerd2-proxy-api#421 * linkerd/linkerd2#13492 * linkerd/linkerd2#13493 * hyperium/hyper#3816 * #3472 * #3473 * #3479 * tokio-rs/tokio#7059 * #3509 * hyperium/http-body#140 * #3515 * hyperium/http-body#141 * #3530 * #3531 * #3540 * #3556 * #3558 * #3559 * #3564 * #3567 * #3573 * #3583 * hyperium/http-body#144 * #3585 * #3586 * #3597 * #3598 * #3611 * #3614 * #3615 * #3616 * #3647 * #3651 * #3653 * #3654 * #3655 * #3656 * #3657 * #3660 * #3671 * #3672 * #3673 * #3676 * hyperium/http-body#147 * #3692 * #3699 * #3700 * #3701 * #3708 * linkerd/drain-rs#36 * #3715 * #3717 * eminence/procfs#340 --- squash: chore(deps): add hyper-util workspace dependency chore(deps): add http-body-util workspace dependency chore(deps): upgrade linkerd2-proxy-api this commit represents main as of linkerd/linkerd2-proxy-api#421. Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/box): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(hyper-balance): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/retain): ugrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/stream-timeouts): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/classify): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/upgrade): upgrade to hyper 1.x NOTE: there is a comment noting that the upgrade middleware does not expect to be cloneable. it is unfortunately, however, at odds with the new bounds expected of extensions. so, `Http11Upgrade` is now Clone'able, but a comment is left in place noting this weakened invariant. it's worth investigating how upgrades have changed since, in more detail, but for the current moment we are interested in being especially conservative about changing behavior, and focusing on api changes like `Body::poll_frame(..)`. Signed-off-by: katelyn martin <kate@buoyant.io> * chore(metrics): upgrade to hyper 1.x a brief note; this commit happened to tickle an unfortunate sharp edge in `BoxBody` and `Full`'s respective constructors. type inference could not figure out how to construct the body, so we refrain from boxing the response body now. Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/metrics): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/prom): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/insert): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/retry): deprecate linkerd-http-body-compat Signed-off-by: katelyn martin <kate@buoyant.io> * chore(mock/http-body): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/retry): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(proxy/tap): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(proxy/http): update to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(app/core): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(app/test): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(app/admin): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(app/outbound): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(app/inbound): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(app/integration): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(app): upgrade to hyper 1.x Signed-off-by: katelyn martin <kate@buoyant.io> * chore(transport-header): update generated code Signed-off-by: katelyn martin <kate@buoyant.io> * chore(spiffe-proto): update generated code Signed-off-by: katelyn martin <kate@buoyant.io> * chore(opencensus-proto): update generated code Signed-off-by: katelyn martin <kate@buoyant.io> * chore(opentelemetry-proto): update generated code Signed-off-by: katelyn martin <kate@buoyant.io> * chore(deny.toml): update cargo-deny directives this commit updates the contents of `deny.toml`. Signed-off-by: katelyn martin <kate@buoyant.io> * chore: `compile` has been renamed to `compile_protos` this addresses deprecation warnings, updating calls to a function that has since been renamed. Signed-off-by: katelyn martin <kate@buoyant.io> * chore(deps): remove `linkerd-http-body-compat` dependencies this commit removes this crate, which we added to future proof code for this upgrade, from its dependents. Signed-off-by: katelyn martin <kate@buoyant.io> * chore(http/body-compat): remove `linkerd-http-body-compat` crate Signed-off-by: katelyn martin <kate@buoyant.io> * chore(deps): update to drain 0.2.1 see linkerd/drain-rs#41. Signed-off-by: katelyn martin <kate@buoyant.io> --------- Signed-off-by: katelyn martin <kate@buoyant.io>

cratelyn force-pushed the kate/http-workspace-dependency branch from b775d2b to a7538d6 Compare December 16, 2024 15:24

cratelyn marked this pull request as ready for review December 16, 2024 16:29

cratelyn requested a review from a team as a code owner December 16, 2024 16:29

DavidMcLaughlin assigned zaharidichev Dec 16, 2024

cratelyn mentioned this pull request Dec 16, 2024

proxy: Figure out the plan for Hyper 1.0 migration linkerd/linkerd2#8733

Closed

cratelyn marked this pull request as draft December 16, 2024 17:53

cratelyn force-pushed the kate/http-workspace-dependency branch from a7538d6 to 208bb8a Compare December 16, 2024 17:54

cratelyn marked this pull request as ready for review December 16, 2024 17:55

cratelyn force-pushed the kate/http-workspace-dependency branch from 208bb8a to 1a5826f Compare December 16, 2024 17:57

cratelyn mentioned this pull request Dec 16, 2024

refactor: move away from legacy hyper::body::HttpBody #3467

Merged

zaharidichev approved these changes Dec 17, 2024

View reviewed changes

cratelyn merged commit ca50d6b into main Dec 17, 2024
47 checks passed

cratelyn deleted the kate/http-workspace-dependency branch December 17, 2024 12:46

cratelyn mentioned this pull request Dec 19, 2024

chore(cargo.toml): define more workspace dependencies #3473

Merged

cratelyn mentioned this pull request Mar 6, 2025

chore(deps): prost-build is a workspace dependency #3701

Merged

cratelyn mentioned this pull request Mar 7, 2025

chore(deps): tower crates are workspace dependencies #3715

Merged

cratelyn mentioned this pull request Mar 7, 2025

chore(deps): drain is a workspace dependency #3717

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

chore(cargo.toml): define http workspace dependencies #3466

chore(cargo.toml): define http workspace dependencies #3466

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chore(cargo.toml): define http workspace dependencies #3466

chore(cargo.toml): define http workspace dependencies #3466

Uh oh!

Conversation

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!