egress-gateway: Add IPv6 support #39068

syedazeez337 · 2025-04-22T02:39:35Z

egress-gateway: Add IPv6 support

Description

This PR adds IPv6 support to the egress gateway feature by introducing a new --enable-egress-gateway flag that enables egress gateway functionality for both IPv4 and IPv6. The existing --enable-ipv4-egress-gateway flag is marked as deprecated but still functional for backward compatibility.

Changes

Add a new --enable-egress-gateway flag that enables egress gateway for both IPv4 and IPv6
Properly mark the old flag as deprecated using flags.MarkDeprecated
Update the validation format for EgressIP from ipv4 to ip to support both IPv4 and IPv6 addresses
Update documentation to remove unnecessary IPv6 examples and comments
Update command reference documentation
Simplify example file

Testing

Ran unit tests for the affected packages
Verified that both flags can be set in the DaemonConfig struct
Ran make -C Documentation update-cmdref to update the command reference

Related Issues

Fixes #38957
Fixes #38962

Add a new `--enable-egress-gateway` flag that enables egress gateway for both IPv4 and IPv6, deprecating the IPv4-only `--enable-ipv4-egress-gateway` flag.

maintainer-s-little-helper · 2025-04-22T02:39:39Z

Commit df54cf7 does not match "(?m)^Signed-off-by:".

Please follow instructions provided in https://docs.cilium.io/en/stable/contributing/development/contributing_guide/#developer-s-certificate-of-origin

zacharysarah

LGTM for docs

joestringer

One minor change from a @cilium/docs-structure reviewer perspective.

Please also merge all the commits together and rebase against the tip of the main branch. The merge commit seems to be causing some confusion in the pull request.

Documentation/community/release-notes/v1.16.rst

joestringer

The changes to docs look good to me 👍

joestringer · 2025-04-29T18:06:53Z

/test

joestringer · 2025-04-29T22:57:12Z

The egress gateway Go IPv6 tests are failing, looks related to the PR.

syedazeez337 · 2025-04-30T03:50:53Z

When I ran the privileged tests for the egressgateway package with sudo -E PRIVILEGED_TESTS=1 go test -v ./pkg/egressgateway/..., three tests failed:

TestEgressGatewayManager - Failed with error: "mismatched gateway IP"
TestNodeSelector - Failed with error: "mismatched gateway IP"
TestEndpointDataStore - Failed with error: "mismatched gateway IP"
All three tests are failing with the same error message: "mismatched gateway IP".
If these tests are failing because of my code changes or a different issue altogether? I need some clarity and suggestions on this one.

cd /home/aze/Documents/cilium && ./test-ipv6-egress-gateway.sh
Testing IPv6 egress gateway support
Created test policy file
Validating policy with kubectl...
kubectl not found, skipping validation
Checking CRD schema for IPv6 support...
CRD schema supports IPv6 addresses
Checking IPv6 support in policy.go...
IPv6 support is implemented in policy.go
Checking mismatched IP families validation...
Mismatched IP families validation is implemented
Checking IPv6 tests...
IPv6 tests are implemented
All tests passed!
IPv6 egress gateway support is correctly implemented

sudo -E go test -v ./pkg/egressgateway/...
=== RUN   TestEgressGatewayCEGPParser
    manager_privileged_test.go:140: Set PRIVILEGED_TESTS to run this test
--- SKIP: TestEgressGatewayCEGPParser (0.00s)
=== RUN   TestEgressGatewayManager
    manager_privileged_test.go:140: Set PRIVILEGED_TESTS to run this test
--- SKIP: TestEgressGatewayManager (0.00s)
=== RUN   TestNodeSelector
    manager_privileged_test.go:140: Set PRIVILEGED_TESTS to run this test
--- SKIP: TestNodeSelector (0.00s)
=== RUN   TestEndpointDataStore
    manager_privileged_test.go:140: Set PRIVILEGED_TESTS to run this test
--- SKIP: TestEndpointDataStore (0.00s)
=== RUN   TestCell
--- PASS: TestCell (0.00s)
=== RUN   TestParseIPv6EgressGatewayPolicy
--- PASS: TestParseIPv6EgressGatewayPolicy (0.00s)
=== RUN   TestDeriveFromPolicyGatewayConfigIPv6
--- PASS: TestDeriveFromPolicyGatewayConfigIPv6 (0.00s)
=== RUN   TestMismatchedIPFamilies
--- PASS: TestMismatchedIPFamilies (0.00s)
=== RUN   TestPolicyConfig_updateMatchedEndpointIDs
=== RUN   TestPolicyConfig_updateMatchedEndpointIDs/Test_updateMatchedEndpointIDs_with_endpoints_and_nodes
=== RUN   TestPolicyConfig_updateMatchedEndpointIDs/Test_updateMatchedEndpointIDs_endpoints_and_nodes_with_no_match
=== RUN   TestPolicyConfig_updateMatchedEndpointIDs/Test_updateMatchedEndpointIDs_endpoints_and_nodes_with_no_match_label
--- PASS: TestPolicyConfig_updateMatchedEndpointIDs (0.00s)
    --- PASS: TestPolicyConfig_updateMatchedEndpointIDs/Test_updateMatchedEndpointIDs_with_endpoints_and_nodes (0.00s)
    --- PASS: TestPolicyConfig_updateMatchedEndpointIDs/Test_updateMatchedEndpointIDs_endpoints_and_nodes_with_no_match (0.00s)
    --- PASS: TestPolicyConfig_updateMatchedEndpointIDs/Test_updateMatchedEndpointIDs_endpoints_and_nodes_with_no_match_label (0.00s)
PASS
ok      github.com/cilium/cilium/pkg/egressgateway(cached)

maintainer-s-little-helper · 2025-04-30T04:02:36Z

Commits df54cf7, 53a156e, 1f16fc0 do not match "(?m)^Signed-off-by:".

Please follow instructions provided in https://docs.cilium.io/en/stable/contributing/development/contributing_guide/#developer-s-certificate-of-origin

This commit adds IPv6 support to the egress gateway feature. It includes: 1. A new flag `--enable-egress-gateway` that enables both IPv4 and IPv6 egress gateway functionality, while keeping the old `--enable-ipv4-egress-gateway` flag for backward compatibility. 2. Updates to the CRD schema to accept both IPv4 and IPv6 addresses. 3. Validation to reject policies with mismatched IP families. 4. Support for IPv6 egress IPs in policy configuration. 5. Tests for IPv6 egress gateway functionality. Signed-off-by: Syed Azeez <syedazeez337@gmail.com>

rgo3

Hey @syedazeez337,

Sorry for the long delay, I'm not requested as a reviewer on this PR so it slipped of my plate. Apologies for that.

Please see my comments below for a first round of review.

rgo3 · 2025-05-26T15:59:38Z

pkg/egressgateway/manager.go

-	// TODO: deprecate --enable-ipv4-egress-gateway, create new --enable-egress-gateway
-	if !dcfg.EnableIPv4EgressGateway {
+	// Check if either IPv4 or IPv6 egress gateway is enabled
+	if !dcfg.EnableIPv4EgressGateway && !dcfg.EnableEgressGateway {


I think here we can still just have one daemon config value. However you should add some code in pkg/option/config.go, to make sure the config value is set correctly depending on how users set the option configs flag. I would fdo this with the following steps:

Rename DaemonConfig.EnableIPv4EgressGateway to DaemonConfig.EnableEgressGateway

Set DaemonConfig.EnableEgressGateway to true if either of the option flags enable-ipv4-egress-gateway or enable-egress-gateway is set to true. You can access them through viper.

This way we can keep most of the code the same, we added a new option flag, deprecated the old one but nothing for existing users will change if they keep running with enable-ipv4-egress-gateway after an upgrade.

rgo3 · 2025-05-26T16:04:59Z

pkg/egressgateway/manager.go

@@ -198,10 +198,19 @@ func NewEgressGatewayManager(p Params) (out struct {
 		return out, errors.New("egress gateway is not supported in combination with the CiliumEndpointSlice feature")
 	}

-	// TODO: refactor config checks for both ipv4 and ipv6, and derive whether the environment supports egress gateway policies for either protocol


I think for now we should keep this code as is, and keep the TODO comment. We can tackle this independently after this PR. That's why I created two separate issues. Making sure we do this correctly is important so that we don't break existing deployments. For example you suggested change could potentially break an existing user-deployment that uses ipv4 egress gateway policies, has ipv6 enabled as well but sets dcfg.EnableIPv6Masquerade to false deliberately. For such users cilium would suddenly fail after an upgrade, without them having changed a thing. This should be avoided.

So let's stick to deprecating the old option flag and enabling users to use the egressIP field in this PR.

rgo3 · 2025-05-26T16:16:01Z

pkg/egressgateway/policy.go

@@ -359,6 +366,28 @@ func ParseCEGP(cegp *v2.CiliumEgressGatewayPolicy) (*PolicyConfig, error) {
 		}
 	}

+	// Check for mismatched IP families


I think this code could be a bit less verbose and can be run earlier in this function. I would propose to add something like the following, right after we have collected all the destination CIDRs:

Suggested change

// Check for mismatched IP families

for _, cidr := range dstCidrList {

if cidr.Addr().Is6() && policyGwc.egressIP.IsValid() && policyGwc.egressIP.Is4() {

return nil, fmt.Errorf("mismatched IP families: IPv4 egress IP with IPv6 destination CIDRs")

}

if cidr.Addr().Is4() && policyGwc.egressIP.IsValid() && policyGwc.egressIP.Is6() {

return nil, fmt.Errorf("mismatched IP families: IPv6 egress IP with IPv4 destination CIDRs")

}

}

rgo3 · 2025-05-27T07:51:36Z

pkg/option/config.go

@@ -1671,6 +1674,7 @@ type DaemonConfig struct {

 	EnableBPFClockProbe     bool
 	EnableIPv4EgressGateway bool


As I suggested above, I think we can remove this daemon config parameter.

rgo3 · 2025-05-27T08:00:19Z

pkg/egressgateway/policy_ipv6_test.go

Instead of adding a new ipv6 only test file, please see the TestEgressGatewayCEGPParser function in manager_privileged_test.go and add the needed test cases there.

rgo3 · 2025-05-27T09:54:57Z

pkg/k8s/apis/cilium.io/client/crds/v2/ciliumegressgatewaypolicies.yaml

This file is code-generated I believe, to make any changes here you'll have to change pkg/k8s/apis/cilium.io/v2/cegp_types.go.

rgo3 · 2025-05-27T09:57:15Z

pkg/k8s/apis/cilium.io/client/crds/v2/ciliumegressgatewaypolicies.yaml

                      default route.
-                    format: ipv4
+                    format: ip


Are you certain that ip is a valid format parameter? I can't find it in the OpenAPI v3 spec?

syedazeez337 requested review from a team as code owners April 22, 2025 02:39

syedazeez337 requested review from thorn3r, nathanjsweet and derailed April 22, 2025 02:39

maintainer-s-little-helper bot added the dont-merge/needs-sign-off The author needs to add signoff to their commits before merge. label Apr 22, 2025

syedazeez337 requested a review from ysksuzuki April 22, 2025 02:39

maintainer-s-little-helper bot added the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Apr 22, 2025

syedazeez337 requested review from pippolo84, squeed and christarazi April 22, 2025 02:39

github-actions bot added cilium-cli This PR contains changes related with cilium-cli kind/community-contribution This was a contribution made by a community member. labels Apr 22, 2025

zacharysarah approved these changes Apr 22, 2025

View reviewed changes

joestringer requested changes Apr 22, 2025

View reviewed changes

Documentation/community/release-notes/v1.16.rst Outdated Show resolved Hide resolved

joestringer requested review from a team April 22, 2025 21:45

joestringer added the release-note/major This PR introduces major new functionality to Cilium. label Apr 22, 2025

maintainer-s-little-helper bot removed the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Apr 22, 2025

joestringer removed request for a team, thorn3r, nathanjsweet, derailed, pippolo84, squeed and christarazi April 29, 2025 18:05

joestringer approved these changes Apr 29, 2025

View reviewed changes

syedazeez337 force-pushed the ipv6-egress-gateway branch from 0cb967c to 7f2c72a Compare April 30, 2025 04:02

maintainer-s-little-helper bot added the dont-merge/needs-sign-off The author needs to add signoff to their commits before merge. label Apr 30, 2025

syedazeez337 force-pushed the ipv6-egress-gateway branch from 7f2c72a to bb32eba Compare April 30, 2025 04:08

maintainer-s-little-helper bot removed the dont-merge/needs-sign-off The author needs to add signoff to their commits before merge. label Apr 30, 2025

syedazeez337 marked this pull request as draft April 30, 2025 04:09

syedazeez337 force-pushed the ipv6-egress-gateway branch from bb32eba to d1a5e73 Compare April 30, 2025 04:30

rgo3 requested changes May 27, 2025

View reviewed changes

rgo3 mentioned this pull request Jun 4, 2025

WIP: support ipv6 egress ips #39595

Draft

8 tasks

joestringer added the dont-merge/wait-until-release Freeze window for current release is blocking non-bugfix PRs label Jun 20, 2025

joestringer removed the dont-merge/wait-until-release Freeze window for current release is blocking non-bugfix PRs label Jul 1, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

egress-gateway: Add IPv6 support #39068

egress-gateway: Add IPv6 support #39068

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

-	// Check for mismatched IP families
+for _, cidr := range dstCidrList {
+	if cidr.Addr().Is6() && policyGwc.egressIP.IsValid() && policyGwc.egressIP.Is4() {
+		return nil, fmt.Errorf("mismatched IP families: IPv4 egress IP with IPv6 destination CIDRs")
+	}
+	if cidr.Addr().Is4() && policyGwc.egressIP.IsValid() && policyGwc.egressIP.Is6() {
+		return nil, fmt.Errorf("mismatched IP families: IPv6 egress IP with IPv4 destination CIDRs")
+	}
+}

		@@ -1671,6 +1674,7 @@ type DaemonConfig struct {

		EnableBPFClockProbe bool
		EnableIPv4EgressGateway bool

egress-gateway: Add IPv6 support #39068

Are you sure you want to change the base?

egress-gateway: Add IPv6 support #39068

Uh oh!

Conversation

egress-gateway: Add IPv6 support

Description

Changes

Testing

Related Issues

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!