outbound: TCP discovery and load balancing #652
Conversation
linkerd/stack/src/make_thunk.rs
Outdated
| #[derive(Clone, Debug)] | ||
| pub struct MakeThunk<M> { | ||
| make: M, | ||
| } |
There was a problem hiding this comment.
it might be nice to have a comment explaining what this thing does exactly, for future readers?
i agree that MakeThunk describes what it does in a general sense, but it might be worth naming it something that uses tower/linkerd-specific language? e.g., when i wrote a version of this for the control plane connector (which we no longer use), i initially named it Curried, and then thought that seemed too abstract, and called it WithTarget.
sorry for bikeshedding, it fundamentally doesn't matter.
There was a problem hiding this comment.
I documented it. I don't think WithTarget is all that clear, personally (especially when compared with MapTarget -- I'd expect WithTarget to take a constant target to be reused for all requests, or something...). Anyway, we can easily change this later.
|
Tested via https://github.com/olix0r/init-net-test/blob/b3860861c54344667a797ece5b685a0574798fb3/k8s.yml -- appears to work. We'll want to do some deeper burnin testing, and we'll definitely need to improve metrics, etc; but I don't think any of that needs to block this moving forward. |
This release includes several major changes to the proxy's behavior: - Service profile lookups are now necessary and fundamental to outbound discovery for HTTP traffic. That is, if a service profile lookup is rejected, endpoint discovery will not be performed; and endpoint discovery must succeed for all destinations that are permitted by service profiles. This simplifies caching and buffering to reduce latency (especially under concurrency). - Service discovery is now performed for all TCP traffic, and connections are balanced over endpoints according to connection latency. - This enables mTLS for **all** meshed connections; not just HTTP. - Outbound TCP metrics are now hydrated with endpoint-specific labels. --- * outbound: Cache balancers within profile stack (linkerd/linkerd2-proxy#641) * outbound: Remove unused error type (linkerd/linkerd2-proxy#648) * Eliminate the ConnectAddr trait (linkerd/linkerd2-proxy#649) * profiles: Do not rely on tuples as stack targets (linkerd/linkerd2-proxy#650) * proxy-http: Remove unneeded boilerplate (linkerd/linkerd2-proxy#651) * outbound: Clarify Http target types (linkerd/linkerd2-proxy#653) * outbound: TCP discovery and load balancing (linkerd/linkerd2-proxy#652) * metrics: Add endpoint labels to outbound TCP metrics (linkerd/linkerd2-proxy#654)
This release includes several major changes to the proxy's behavior: - Service profile lookups are now necessary and fundamental to outbound discovery for HTTP traffic. That is, if a service profile lookup is rejected, endpoint discovery will not be performed; and endpoint discovery must succeed for all destinations that are permitted by service profiles. This simplifies caching and buffering to reduce latency (especially under concurrency). - Service discovery is now performed for all TCP traffic, and connections are balanced over endpoints according to connection latency. - This enables mTLS for **all** meshed connections; not just HTTP. - Outbound TCP metrics are now hydrated with endpoint-specific labels. --- * outbound: Cache balancers within profile stack (linkerd/linkerd2-proxy#641) * outbound: Remove unused error type (linkerd/linkerd2-proxy#648) * Eliminate the ConnectAddr trait (linkerd/linkerd2-proxy#649) * profiles: Do not rely on tuples as stack targets (linkerd/linkerd2-proxy#650) * proxy-http: Remove unneeded boilerplate (linkerd/linkerd2-proxy#651) * outbound: Clarify Http target types (linkerd/linkerd2-proxy#653) * outbound: TCP discovery and load balancing (linkerd/linkerd2-proxy#652) * metrics: Add endpoint labels to outbound TCP metrics (linkerd/linkerd2-proxy#654)
The proxy only forward TCP connections to the original destination address, never
using the mesh's mTLS.
This change modifies the outbound TCP forwarding stack to do discovery based on
the original destination address, resolving service IPs to the individual endpoint IPs
(and their associated identities). This enables mTLS for meshed TCP connections.
When the endpoint cannot be discovered (i.e. due to an InvalidArgument response
from the controller), the connection is forwarded as before.
The PeakEWMA balancer is repurposed for this, using connection latency (and
pending connections) as the load metric. In the future, this should be modified
to count active connections towards the load (but this won't Just Work with the
PeakEwma balancer, so this is deferred for now).
This has been tested manually.
In follow-up changes, we'll extend the transparency tests to validate this behavior.