outbound: Cache balancers within profile stack #641
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This change updates and extends the diagnostic stack checks that we use during development. No functional changes.
The `HasDestination` trait isn't particularly useful, as it's basically just `AsRef<Addr>`. This change updates the `GetRoutes` signature to support this; and it updates the inbound target type to store an `Addr` instead of an `Option<NameAddr>` (so `Target` will be suitable for this in an upcoming change).
We need tower-rs/tower@ad348d8 for an upcoming change. This PR updates the tower dependency in anticipation of this change. The `Balance` constructor has changed.
hawkw
reviewed
Aug 31, 2020
Comment on lines
+18
to
+21
| let rng = SmallRng::from_entropy(); | ||
| layer::mk(move |inner| NewSplit { | ||
| inner, | ||
| rng: rng.clone(), |
There was a problem hiding this comment.
should we be doing something like the second example here? i'll admit i'm not too familiar w/ rand's APIs...
There was a problem hiding this comment.
@hawkw is that the right link? if so, i'm not sure what you're suggesting...
There was a problem hiding this comment.
whoops, it is not --- i meant to link to here: https://docs.rs/rand/0.7.3/rand/rngs/struct.SmallRng.html#examples
Comment on lines
+92
to
+95
| let mut update = None; | ||
| while let Poll::Ready(Some(up)) = self.rx.poll_recv_ref(cx) { | ||
| update = Some(up.clone()); | ||
| } |
There was a problem hiding this comment.
as above, not sure if we need to do this...
There was a problem hiding this comment.
If the ready cache isn't ready, then we want to be notified when the profile changes, so we have to drive it to pending. In general, we should always be driving to not ready to be sure we're at the latest state.
In an upcoming change, we'd like to do service profile discovery within the TCP-accept stack (and not the HTTP routing stack). But, today, service profile discovery is tightly coupled to the HTTP middleware implementations. This change splits the service profile layers (profile discovery, http request profiles, and traffic splitting) into several layers, so that the discovery logic is decoupled from the http-specific request-routing middleware. This change removes the balancer cache and the balancer-specific buffer so that balancers are owned by the split layer. The buffer has been moved outside of the split layer to drive all balancers in a split and to make the split cloneable (for the retry middleware). All of this is cached under the profile cache. Breaking changes: This likely breaks support for "external" service profiles, where the proxy resolves service profilse for domains that it cannot resolve via the resolver. This feature is not enabled by default and is considered experimental. We'll have to take attention to at least document this in upcoming releases. Side-effects: * Latency improvements at concurrency  * Compile time improved by 20-25% (integration tests in CI ~20m => ~15m);
olix0r
added a commit
to linkerd/linkerd2
that referenced
this pull request
Sep 10, 2020
This release includes several major changes to the proxy's behavior: - Service profile lookups are now necessary and fundamental to outbound discovery for HTTP traffic. That is, if a service profile lookup is rejected, endpoint discovery will not be performed; and endpoint discovery must succeed for all destinations that are permitted by service profiles. This simplifies caching and buffering to reduce latency (especially under concurrency). - Service discovery is now performed for all TCP traffic, and connections are balanced over endpoints according to connection latency. - This enables mTLS for **all** meshed connections; not just HTTP. - Outbound TCP metrics are now hydrated with endpoint-specific labels. --- * outbound: Cache balancers within profile stack (linkerd/linkerd2-proxy#641) * outbound: Remove unused error type (linkerd/linkerd2-proxy#648) * Eliminate the ConnectAddr trait (linkerd/linkerd2-proxy#649) * profiles: Do not rely on tuples as stack targets (linkerd/linkerd2-proxy#650) * proxy-http: Remove unneeded boilerplate (linkerd/linkerd2-proxy#651) * outbound: Clarify Http target types (linkerd/linkerd2-proxy#653) * outbound: TCP discovery and load balancing (linkerd/linkerd2-proxy#652) * metrics: Add endpoint labels to outbound TCP metrics (linkerd/linkerd2-proxy#654)
olix0r
added a commit
to linkerd/linkerd2
that referenced
this pull request
Sep 10, 2020
This release includes several major changes to the proxy's behavior: - Service profile lookups are now necessary and fundamental to outbound discovery for HTTP traffic. That is, if a service profile lookup is rejected, endpoint discovery will not be performed; and endpoint discovery must succeed for all destinations that are permitted by service profiles. This simplifies caching and buffering to reduce latency (especially under concurrency). - Service discovery is now performed for all TCP traffic, and connections are balanced over endpoints according to connection latency. - This enables mTLS for **all** meshed connections; not just HTTP. - Outbound TCP metrics are now hydrated with endpoint-specific labels. --- * outbound: Cache balancers within profile stack (linkerd/linkerd2-proxy#641) * outbound: Remove unused error type (linkerd/linkerd2-proxy#648) * Eliminate the ConnectAddr trait (linkerd/linkerd2-proxy#649) * profiles: Do not rely on tuples as stack targets (linkerd/linkerd2-proxy#650) * proxy-http: Remove unneeded boilerplate (linkerd/linkerd2-proxy#651) * outbound: Clarify Http target types (linkerd/linkerd2-proxy#653) * outbound: TCP discovery and load balancing (linkerd/linkerd2-proxy#652) * metrics: Add endpoint labels to outbound TCP metrics (linkerd/linkerd2-proxy#654)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In an upcoming change, we'd like to do service profile discovery within
the TCP-accept stack (and not the HTTP routing stack). But, today,
service profile discovery is tightly coupled to the HTTP middleware
implementations. This change splits the service profile layers (profile
discovery, http request profiles, and traffic splitting) into several
layers, so that the discovery logic is decoupled from the http-specific
request-routing middleware.
This change removes the balancer cache and the balancer-specific buffer
so that balancers are owned by the split layer. The buffer has been
moved outside of the split layer to drive all balancers in a split and
to make the split cloneable (for the retry middleware). All of this is
cached under the profile cache.
Breaking changes:
This likely breaks support for "external" service profiles, where the
proxy resolves service profilse for domains that it cannot resolve via
the resolver. This feature is not enabled by default and is considered
experimental. We'll have to take attention to at least document this in
upcoming releases.
Side-effects: