8000 Reverted one change which was introduced to reduce flakiness in e2e tests. by ishan16696 · Pull Request #1034 · gardener/etcd-druid · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Reverted one change which was introduced to reduce flakiness in e2e tests. #1034

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Mar 17, 2025
Merged

Reverted one change which was introduced to reduce flakiness in e2e tests. #1034

merged 1 commit into from
Mar 17, 2025

Conversation

ishan16696
Copy link
Member

How to categorize this PR?

/area testing
/kind test

What this PR does / why we need it:
backupDeltaSnapshotPeriod was increased to 3sec from 1sec to reduce the flakiness of druid e2e tests.
With the fix of this PR: gardener/etcd-backup-restore#845, the flakiness in druid e2e tests will no longer be the case, so I'm reverting this temporary change which was introduced to fix flakiness in druid e2e tests.

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:
cc @unmarshall

Release note:

None

@ishan16696 ishan16696 requested a review from a team as a code owner March 17, 2025 04:12
@gardener-robot gardener-robot added area/testing Testing related kind/test Test needs/review Needs review size/xs Size of pull request is tiny (see gardener-robot robot/bots/size.py) labels Mar 17, 2025
@gardener-robot-ci-3 gardener-robot-ci-3 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Mar 17, 2025
@gardener-robot-ci-1 gardener-robot-ci-1 added needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Mar 17, 2025
@ishan16696
Copy link
Member Author

/test pull-etcd-druid-e2e-kind

@ishan16696
Copy link
Member Author

druid e2e tests passed locally twice.

Logs 
Generating kind cluster config...
Creating cluster "etcd-druid-e2e" ...
 ✓ Ensuring node image (kindest/node:v1.32.0) 🖼
 ✓ Preparing nodes 📦
 ✓ Writing configuration 📜
 ✓ Starting control-plane 🕹️
 ✓ Installing CNI 🔌
 ✓ Installing StorageClass 💾
Set kubectl context to "kind-etcd-druid-e2e"
You can now use your cluster with:

kubectl cluster-info --context kind-etcd-druid-e2e

Thanks for using kind! 😊
Initializing local docker registry...
configmap/local-registry-hosting created

📌 NOTE: To target the newly created KinD cluster, please run the following command:

    export KUBECONFIG=/Users/I539698/Desktop/review/etcd-druid/hack/kind/kubeconfig

node/etcd-druid-e2e-control-plane condition met
deployment.apps/localstack created
service/localstack created
Waiting for deployment "localstack" rollout to finish: 0 of 1 updated replicas are available...
deployment "localstack" successfully rolled out
pod/localstack-7f5b657dd9-2knvw condition met
Attempting to get kubernetes version via kubectl...
Copying CRDs to helm charts...
Creating /Users/I539698/Desktop/review/etcd-druid/charts/crds to copy the CRDs if not present...
Cleaning up existing CRDs if any...
Copying CRD druid.gardener.cloud_etcds.yaml to /Users/I539698/Desktop/review/etcd-druid/charts/crds
Copying CRD druid.gardener.cloud_etcdcopybackupstasks.yaml to /Users/I539698/Desktop/review/etcd-druid/charts/crds
Generating PKI resources if not present or expired...
Generating PKI resources...
Generating CA key...
Creating CA configuration at /Users/I539698/Desktop/review/etcd-druid/charts/pki-resources/requests/ca.cnf...
Generating CA certificate...
Generating server key...
Creating server configuration at /Users/I539698/Desktop/review/etcd-druid/charts/pki-resources/requests/server.cnf...
Generating server CSR...
Generating server certificate...
Certificate request self-signature ok
subject=CN=etcd-druid-server, O=Gardener
namespace/e2e-test created
-------------------
Setting up environment
-------------------
namespace/e2e-test unchanged
No tags generated
Starting test...
Starting deploy...
Starting pre-deploy hooks...
Deleting previous job
No resources found
Completed pre-deploy hooks
 - configmap/aws-infra-common created
 - configmap/aws-run-script created
 - secret/aws-infra-secret created
 - job.batch/aws-infra-job created
Starting post-deploy hooks...
Wait until job is ready
job.batch/aws-infra-job condition met
Completed post-deploy hooks
You can also run [skaffold run --tail] to get the logs

-------------------
Deploying Druid
-------------------
Generating tags...
 - local-skaffold/etcd-druid -> localhost:5001/local-skaffold_etcd-druid:v0.4.0-613-g352b2a89
Checking cache...
 - local-skaffold/etcd-druid: Not found. Building
Starting build...
Found [kind-etcd-druid-e2e] context, using local docker daemon.
Building [local-skaffold/etcd-druid]...
Target platforms: [linux/arm64]
Using base cgr.dev/chainguard/static:latest@sha256:9276a4ebe6b98cd1bbd53b8139228434a0e4f00d06d39e33688e9bd759986656 for github.com/gardener/etcd-druid
Using build config local-skaffold/etcd-druid for github.com/gardener/etcd-druid
Building github.com/gardener/etcd-druid for linux/arm64
Publishing localhost:5001/local-skaffold_etcd-druid:v0.4.0-613-g352b2a89
Published localhost:5001/local-skaffold_etcd-druid:v0.4.0-613-g352b2a89@sha256:6e59a77432abb192143b46acf3335d45f17a57ae741880b212c0504002bf5de2
Build [local-skaffold/etcd-druid] succeeded
Starting test...
Tags used in deployment:
 - local-skaffold/etcd-druid -> localhost:5001/local-skaffold_etcd-druid:v0.4.0-613-g352b2a89@sha256:6e59a77432abb192143b46acf3335d45f17a57ae741880b212c0504002bf5de2
Starting deploy...
Helm release etcd-druid not installed. Installing...
NAME: etcd-druid
LAST DEPLOYED: Mon Mar 17 09:39:32 2025
NAMESPACE: e2e-test
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Etcd-Druid operator has been deployed to Namespace e2e-test
Following resources has been deployed with name "etcd-druid":
 * Operator Deployment
 * Service
 * ServiceAccount
 * ClusterRole
 * ClusterRoleBinding
 * ValidatingWebhookConfiguration


Now that you have deployed etcd-druid operator you can create Etcd clusters by following the 'Getting Started Guide'.
See: https://gardener.github.io/etcd-druid/deployment/getting-started-locally/getting-started-locally.html#04-preparing-etcd-cr
Etcd examples are present at https://github.com/gardener/etcd-druid/tree/master/examples
Waiting for deployments to stabilize...
 - e2e-test:deployment/etcd-druid is ready.
Deployments stabilized in 1.134 second
You can also run [skaffold run --tail] to get the logs

-------------------
Running e2e tests
-------------------
Running Suite: e2e Suite - /Users/I539698/Desktop/review/etcd-druid/test/e2e
============================================================================
Random Seed: 1742184580

Will run 8 of 8 specs
Running in parallel across 9 processes
------------------------------
[BeforeSuite] PASSED [0.040 seconds]
[BeforeSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[BeforeSuite] PASSED [0.041 seconds]
[BeforeSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[BeforeSuite] PASSED [0.042 seconds]
[BeforeSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[BeforeSuite] PASSED [0.042 seconds]
[BeforeSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[BeforeSuite] PASSED [0.042 seconds]
[BeforeSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[BeforeSuite] PASSED [0.042 seconds]
[BeforeSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[BeforeSuite] PASSED [0.042 seconds]
[BeforeSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[BeforeSuite] PASSED [0.042 seconds]
[BeforeSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[BeforeSuite] PASSED [0.051 seconds]
[BeforeSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:67

  Timeline >>
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating namespace","namespace":"shoot"}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"ca-etcd-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"etcd-server-cert-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"etcd-client-tls-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"creating secret","secret":{"name":"etcd-backup-aws","namespace":"shoot"}}
  << Timeline
------------------------------
[SynchronizedAfterSuite] PASSED [0.000 seconds]
[SynchronizedAfterSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:117
------------------------------
• [146.666 seconds]
Etcd Backup when single-node etcd is configured with provider aws Should create, test backup and delete etcd with backup
/Users/I539698/Desktop/review/etcd-druid/test/e2e/etcd_backup_test.go:58

  Captured StdOut/StdErr Output >>
  time="2025-03-17T09:39:47+05:30" level=info msg="Bucket versioning is not found to be enabled."
  << Captured StdOut/StdErr Output

  Timeline >>
  STEP: Purge snapstore @ 03/17/25 09:39:47.596
  STEP: Create etcd @ 03/17/25 09:39:47.624
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:03+05:30","msg":"etcd is ready","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:03+05:30","msg":"Checking statefulset","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:03+05:30","msg":"Checking configmap","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:03+05:30","msg":"Checking client service","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  STEP: Create debug pod @ 03/17/25 09:41:03.986
  STEP: Check initial snapshot is available @ 03/17/25 09:41:12.021
  STEP: Put keys into etcd @ 03/17/25 09:41:12.094
  {"level":"info","ts":"2025-03-17T09:41:12+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-1","fromValue":"bar-1","toKey":"foo-10","toValue":"bar-10"}
  {"level":"info","ts":"2025-03-17T09:41:15+05:30","msg":"put key-value successful","key":"foo-1","value":"bar-1"}
  {"level":"info","ts":"2025-03-17T09:41:15+05:30","msg":"put key-value successful","key":"foo-2","value":"bar-2"}
  {"level":"info","ts":"2025-03-17T09:41:16+05:30","msg":"put key-value successful","key":"foo-3","value":"bar-3"}
  {"level":"info","ts":"2025-03-17T09:41:16+05:30","msg":"put key-value successful","key":"foo-4","value":"bar-4"}
  {"level":"info","ts":"2025-03-17T09:41:16+05:30","msg":"put key-value successful","key":"foo-5","value":"bar-5"}
  {"level":"info","ts":"2025-03-17T09:41:16+05:30","msg":"put key-value successful","key":"foo-6","value":"bar-6"}
  {"level":"info","ts":"2025-03-17T09:41:17+05:30","msg":"put key-value successful","key":"foo-7","value":"bar-7"}
  {"level":"info","ts":"2025-03-17T09:41:17+05:30","msg":"put key-value successful","key":"foo-8","value":"bar-8"}
  {"level":"info","ts":"2025-03-17T09:41:17+05:30","msg":"put key-value successful","key":"foo-9","value":"bar-9"}
  {"level":"info","ts":"2025-03-17T09:41:17+05:30","msg":"put key-value successful","key":"foo-10","value":"bar-10"}
  STEP: Check snapshot after putting data into etcd @ 03/17/25 09:41:17.773
  STEP: Trigger on-demand full snapshot @ 03/17/25 09:41:17.893
  STEP: Put additional data into etcd @ 03/17/25 09:41:17.977
  {"level":"info","ts":"2025-03-17T09:41:17+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-11","fromValue":"bar-11","toKey":"foo-15","toValue":"bar-15"}
  {"level":"info","ts":"2025-03-17T09:41:21+05:30","msg":"put key-value successful","key":"foo-11","value":"bar-11"}
  {"level":"info","ts":"2025-03-17T09:41:21+05:30","msg":"put key-value successful","key":"foo-12","value":"bar-12"}
  {"level":"info","ts":"2025-03-17T09:41:21+05:30","msg":"put key-value successful","key":"foo-13","value":"bar-13"}
  {"level":"info","ts":"2025-03-17T09:41:22+05:30","msg":"put key-value successful","key":"foo-14","value":"bar-14"}
  {"level":"info","ts":"2025-03-17T09:41:22+05:30","msg":"put key-value successful","key":"foo-15","value":"bar-15"}
  STEP: Trigger on-demand delta snapshot @ 03/17/25 09:41:22.346
  STEP: Test cluster restoration by deleting data directory @ 03/17/25 09:41:22.408
  {"level":"info","ts":"2025-03-17T09:41:22+05:30","msg":"waiting for sts to become unready","statefulSetName":"etcd-single-node-aws"}
  {"level":"info","ts":"2025-03-17T09:41:46+05:30","msg":"sts is unready","statefulSetName":"etcd-single-node-aws"}
  {"level":"info","ts":"2025-03-17T09:41:46+05:30","msg":"waiting for sts to become ready again","statefulSetName":"etcd-single-node-aws"}
  {"level":"info","ts":"2025-03-17T09:42:02+05:30","msg":"sts is ready","statefulSetName":"etcd-single-node-aws"}
  {"level":"info","ts":"2025-03-17T09:42:02+05:30","msg":"fetching etcd key-value pairs"}
  {"level":"info","ts":"2025-03-17T09:42:02+05:30","msg":"fetched key-value pair from etcd","key":"foo-1","value":"bar-1"}
  {"level":"info","ts":"2025-03-17T09:42:03+05:30","msg":"fetched key-value pair from etcd","key":"foo-2","value":"bar-2"}
  {"level":"info","ts":"2025-03-17T09:42:03+05:30","msg":"fetched key-value pair from etcd","key":"foo-3","value":"bar-3"}
  {"level":"info","ts":"2025-03-17T09:42:03+05:30","msg":"fetched key-value pair from etcd","key":"foo-4","value":"bar-4"}
  {"level":"info","ts":"2025-03-17T09:42:03+05:30","msg":"fetched key-value pair from etcd","key":"foo-5","value":"bar-5"}
  {"level":"info","ts":"2025-03-17T09:42:04+05:30","msg":"fetched key-value pair from etcd","key":"foo-6","value":"bar-6"}
  {"level":"info","ts":"2025-03-17T09:42:04+05:30","msg":"fetched key-value pair from etcd","key":"foo-7","value":"bar-7"}
  {"level":"info","ts":"2025-03-17T09:42:04+05:30","msg":"fetched key-value pair from etcd","key":"foo-8","value":"bar-8"}
  {"level":"info","ts":"2025-03-17T09:42:04+05:30","msg":"fetched key-value pair from etcd","key":"foo-9","value":"bar-9"}
  {"level":"info","ts":"2025-03-17T09:42:05+05:30","msg":"fetched key-value pair from etcd","key":"foo-10","value":"bar-10"}
  {"level":"info","ts":"2025-03-17T09:42:05+05:30","msg":"fetched key-value pair from etcd","key":"foo-11","value":"bar-11"}
  {"level":"info","ts":"2025-03-17T09:42:05+05:30","msg":"fetched key-value pair from etcd","key":"foo-12","value":"bar-12"}
  {"level":"info","ts":"2025-03-17T09:42:05+05:30","msg":"fetched key-value pair from etcd","key":"foo-13","value":"bar-13"}
  {"level":"info","ts":"2025-03-17T09:42:05+05:30","msg":"fetched key-value pair from etcd","key":"foo-14","value":"bar-14"}
  {"level":"info","ts":"2025-03-17T09:42:06+05:30","msg":"fetched key-value pair from etcd","key":"foo-15","value":"bar-15"}
  STEP: Deleting debug pod @ 03/17/25 09:42:06.222
  STEP: Deleting etcd @ 03/17/25 09:42:06.226
  {"level":"info","ts":"2025-03-17T09:42:06+05:30","msg":"Checking if etcd is gone","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:42:14+05:30","msg":"Checking if statefulset is gone","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:42:14+05:30","msg":"Checking if configmap is gone","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:42:14+05:30","msg":"Checking client service is gone","etcd":{"name":"etcd-single-node-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:42:14+05:30","msg":"Deleting PVCs"}
  << Timeline
------------------------------
[SynchronizedAfterSuite] PASSED [0.000 seconds]
[SynchronizedAfterSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:117
------------------------------
• [210.977 seconds]
Etcd Single-node etcd configuration Scaling up from single-node to multi-node without TLS should scale a single-node etcd (TLS not enabled for peerUrl) to a multi-node etcd cluster (TLS not enabled for peerUrl)
/Users/I539698/Desktop/review/etcd-druid/test/e2e/etcd_multi_node_test.go:137

  Captured StdOut/StdErr Output >>
  time="2025-03-17T09:39:47+05:30" level=info msg="Bucket versioning is not found to be enabled."
  << Captured StdOut/StdErr Output

  Timeline >>
  STEP: Purge snapstore @ 03/17/25 09:39:47.596
  STEP: Creating a single-node etcd @ 03/17/25 09:39:47.627
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:03+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:03+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:03+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:03+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  STEP: Scaling up a healthy cluster (from 1 to 3 replicas) @ 03/17/25 09:41:04.003
  {"level":"info","ts":"2025-03-17T09:42:40+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:43:08+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:43:08+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:43:08+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:43:08+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  STEP: Deleting etcd @ 03/17/25 09:43:08.503
  {"level":"info","ts":"2025-03-17T09:43:08+05:30","msg":"Checking if etcd is gone","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:43:18+05:30","msg":"Checking if statefulset is gone","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:43:18+05:30","msg":"Checking if configmap is gone","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:43:18+05:30","msg":"Checking client service is gone","etcd-multi-node":{"name":"scale-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:43:18+05:30","msg":"Deleting PVCs"}
  << Timeline
------------------------------
[SynchronizedAfterSuite] PASSED [0.000 seconds]
[SynchronizedAfterSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:117
------------------------------
• [253.152 seconds]
Etcd Single-node etcd configuration Scaling down from single-node to zero and back up without TLS executes scaling down to 0 and back up to 3 replicas without TLS
/Users/I539698/Desktop/review/etcd-druid/test/e2e/etcd_multi_node_test.go:188

  Captured StdOut/StdErr Output >>
  time="2025-03-17T09:39:47+05:30" level=info msg="Bucket versioning is not found to be enabled."
  << Captured StdOut/StdErr Output

  Timeline >>
  STEP: Purge snapstore @ 03/17/25 09:39:47.596
  STEP: Creating a single-node etcd @ 03/17/25 09:39:47.625
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:53+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:53+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:53+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:53+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  STEP: Scaling down a healthy cluster (from 1 to 0 replica) @ 03/17/25 09:40:53.941
  {"level":"info","ts":"2025-03-17T09:40:55+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:59+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:59+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:59+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:59+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  STEP: Scaling up cluster (from 0 to 1 replica) @ 03/17/25 09:40:59.993
  {"level":"info","ts":"2025-03-17T09:41:02+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:24+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:24+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:24+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:24+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  STEP: Scaling up a healthy cluster (from 1 to 3 replica) @ 03/17/25 09:41:24.095
  {"level":"info","ts":"2025-03-17T09:43:32+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:43:50+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:43:50+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:43:50+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:43:50+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  STEP: Deleting etcd @ 03/17/25 09:43:50.689
  {"level":"info","ts":"2025-03-17T09:43:50+05:30","msg":"Checking if etcd is gone","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:00+05:30","msg":"Chec
8000
king if statefulset is gone","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:00+05:30","msg":"Checking if configmap is gone","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:00+05:30","msg":"Checking client service is gone","etcd-multi-node":{"name":"scale-down-and-up-non-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:00+05:30","msg":"Deleting PVCs"}
  << Timeline
------------------------------
[SynchronizedAfterSuite] PASSED [0.000 seconds]
[SynchronizedAfterSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:117
------------------------------
• [357.366 seconds]
Etcd Single-node etcd configuration Scaling up from single-node to multi-node with TLS executes scaling up with TLS
/Users/I539698/Desktop/review/etcd-druid/test/e2e/etcd_multi_node_test.go:163

  Captured StdOut/StdErr Output >>
  time="2025-03-17T09:39:47+05:30" level=info msg="Bucket versioning is not found to be enabled."
  << Captured StdOut/StdErr Output

  Timeline >>
  STEP: Purge snapstore @ 03/17/25 09:39:47.596
  STEP: Creating a single-node etcd @ 03/17/25 09:39:47.626
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:12+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:12+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:12+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:12+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  STEP: Scaling up a healthy cluster (from 1 to 3 replicas) with TLS enabled for peerUrl @ 03/17/25 09:41:12.021
  {"level":"info","ts":"2025-03-17T09:43:56+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:36+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:36+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:36+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:36+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  STEP: Deleting etcd @ 03/17/25 09:45:36.909
  {"level":"info","ts":"2025-03-17T09:45:36+05:30","msg":"Checking if etcd is gone","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:44+05:30","msg":"Checking if statefulset is gone","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:44+05:30","msg":"Checking if configmap is gone","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:44+05:30","msg":"Checking client service is gone","etcd-multi-node":{"name":"scale-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:44+05:30","msg":"Deleting PVCs"}
  << Timeline
------------------------------
• [357.366 seconds]
Etcd Single-node etcd configuration Scaling down from single-node to zero and back up with TLS should scale down a single-node etcd to 0 replica, then scale up from 0->1 replica and then from 1->3 replicas with TLS enabled for cluster peerUrl
/Users/I539698/Desktop/review/etcd-druid/test/e2e/etcd_multi_node_test.go:223

  Captured StdOut/StdErr Output >>
  time="2025-03-17T09:39:47+05:30" level=info msg="Bucket versioning is not found to be enabled."
  << Captured StdOut/StdErr Output

  Timeline >>
  STEP: Purge snapstore @ 03/17/25 09:39:47.596
  STEP: Creating a single-node etcd @ 03/17/25 09:39:47.63
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:51+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:51+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:51+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:51+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  STEP: Scaling down a healthy cluster (from 1 to 0 replica) @ 03/17/25 09:40:51.932
  {"level":"info","ts":"2025-03-17T09:40:53+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:55+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:55+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:55+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:55+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  STEP: Scaling up cluster (from 0 to 1 replica) @ 03/17/25 09:40:55.979
  {"level":"info","ts":"2025-03-17T09:40:58+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:22+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:22+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:22+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:41:22+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  STEP: Scaling up a healthy cluster (from 1 to 3 replica) with TLS enabled for peerUrl @ 03/17/25 09:41:22.079
  {"level":"info","ts":"2025-03-17T09:43:56+05:30","msg":"Waiting for etcd to become ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:36+05:30","msg":"etcd is ready","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:36+05:30","msg":"Checking statefulset","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:36+05:30","msg":"Checking configmap","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:36+05:30","msg":"Checking client service","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  STEP: Deleting etcd @ 03/17/25 09:45:36.911
  {"level":"info","ts":"2025-03-17T09:45:36+05:30","msg":"Checking if etcd is gone","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:44+05:30","msg":"Checking if statefulset is gone","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:44+05:30","msg":"Checking if configmap is gone","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:44+05:30","msg":"Checking client service is gone","etcd-multi-node":{"name":"scale-down-and-up-tls","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:44+05:30","msg":"Deleting PVCs"}
  << Timeline
------------------------------
[SynchronizedAfterSuite] PASSED [0.004 seconds]
[SynchronizedAfterSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:117
------------------------------
[SynchronizedAfterSuite] PASSED [0.001 seconds]
[SynchronizedAfterSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:117
------------------------------
• [357.498 seconds]
Etcd when multi-node is configured should perform etcd operations excluding zero downtime maintenance
/Users/I539698/Desktop/review/etcd-druid/test/e2e/etcd_multi_node_test.go:50

  Captured StdOut/StdErr Output >>
  time="2025-03-17T09:39:47+05:30" level=info msg="Bucket versioning is not found to be enabled."
  << Captured StdOut/StdErr Output

  Timeline >>
  STEP: Purge snapstore @ 03/17/25 09:39:47.592
  STEP: Creating etcd @ 03/17/25 09:39:47.621
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:42:46+05:30","msg":"etcd is ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:42:46+05:30","msg":"Checking statefulset","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:42:46+05:30","msg":"Checking configmap","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:42:46+05:30","msg":"Checking client service","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  STEP: Hibernating etcd (Scaling down from 3 replicas to 0) @ 03/17/25 09:42:46.343
  {"level":"info","ts":"2025-03-17T09:42:46+05:30","msg":"Waiting for statefulset spec to reflect change in replicas to 0","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:42:48+05:30","msg":"Checking etcd","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:42:56+05:30","msg":"Checking statefulset","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:42:56+05:30","msg":"etcd is hibernated","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  STEP: Waking up etcd (Scaling up from 0 to 3 replicas) @ 03/17/25 09:42:56.388
  {"level":"info","ts":"2025-03-17T09:42:58+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:30+05:30","msg":"etcd is ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:30+05:30","msg":"Checking statefulset","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:30+05:30","msg":"Checking configmap","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:30+05:30","msg":"Checking client service","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  STEP: Member restart with data-dir/pvc intact @ 03/17/25 09:44:30.723
  {"level":"info","ts":"2025-03-17T09:44:30+05:30","msg":"Deleting one member pod","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:30+05:30","msg":"waiting for sts to become unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:32+05:30","msg":"sts is unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:32+05:30","msg":"Waiting for at least one etcd member to become unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:34+05:30","msg":"at least one etcd member is unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:34+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:50+05:30","msg":"etcd is ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:50+05:30","msg":"Checking statefulset","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:50+05:30","msg":"Checking configmap","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:50+05:30","msg":"Checking client service","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  STEP: Single member restoration @ 03/17/25 09:44:50.823
  {"level":"info","ts":"2025-03-17T09:44:50+05:30","msg":"Creating debug pod","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:54+05:30","msg":"Deleting member directory of one member pod","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:54+05:30","msg":"waiting for sts to become unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:12+05:30","msg":"sts is unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:12+05:30","msg":"Waiting for at least one etcd member to become unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:14+05:30","msg":"at least one etcd member is unready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:14+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:30+05:30","msg":"etcd is ready","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:30+05:30","msg":"Checking statefulset","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:31+05:30","msg":"Checking configmap","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:31+05:30","msg":"Checking client service","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  STEP: Deleting debug pod @ 03/17/25 09:45:31.005
  STEP: Deleting etcd @ 03/17/25 09:45:31.009
  {"level":"info","ts":"2025-03-17T09:45:31+05:30","msg":"Checking if etcd is gone","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:45+05:30","msg":"Checking if statefulset is gone","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:45+05:30","msg":"Checking if configmap is gone","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:45+05:30","msg":"Checking client service is gone","etcd":{"name":"multi-node-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:45+05:30","msg":"Deleting PVCs"}
  << Timeline
------------------------------
[SynchronizedAfterSuite] PASSED [0.000 seconds]
[SynchronizedAfterSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:117
------------------------------
• [387.973 seconds]
Etcd when multi-node is configured should perform zero downtime maintenance operation: defragmentation
/Users/I539698/Desktop/review/etcd-druid/test/e2e/etcd_multi_node_test.go:93

  Captured StdOut/StdErr Output >>
  time="2025-03-17T09:39:47+05:30" level=info msg="Bucket versioning is not found to be enabled."
  << Captured StdOut/StdErr Output

  Timeline >>
  STEP: Purge snapstore @ 03/17/25 09:39:47.595
  STEP: Creating etcd @ 03/17/25 09:39:47.621
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:42:50+05:30","msg":"etcd is ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:42:50+05:30","msg":"Checking statefulset","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:42:50+05:30","msg":"Checking configmap","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:42:50+05:30","msg":"Checking client service","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  STEP: Deploying etcd zero downtime validator job @ 03/17/25 09:42:50.361
  {"level":"info","ts":"2025-03-17T09:42:50+05:30","msg":"Creating job to ensure etcd zero downtime","job":"etcd-zero-down-time-validator-rolling-update"}
  {"level":"info","ts":"2025-03-17T09:43:02+05:30","msg":"Job is ready","job":"etcd-zero-down-time-validator-rolling-update"}
  STEP: Conducting zero downtime rolling updates @ 03/17/25 09:43:02.396
  {"level":"info","ts":"2025-03-17T09:43:48+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:10+05:30","msg":"etcd is ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:10+05:30","msg":"Checking statefulset","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:10+05:30","msg":"Checking configmap","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:10+05:30","msg":"Checking client service","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:10+05:30","msg":"Etcd Cluster is healthy and there is no downtime","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  STEP: Performing zero downtime maintenance operation: defragmentation @ 03/17/25 09:44:10.686
  {"level":"info","ts":"2025-03-17T09:44:10+05:30","msg":"Configuring defragmentation schedule for every 1 minute","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:44:50+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:10+05:30","msg":"etcd is ready","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:10+05:30","msg":"Checking statefulset","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:10+05:30","msg":"Checking configmap","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:10+05:30","msg":"Checking client service","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:10+05:30","msg":"Etcd Cluster is healthy and there is no downtime","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:45:10+05:30","msg":"Waiting for defragmentation to finish","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:46:01+05:30","msg":"Defragmentation is finished","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:46:01+05:30","msg":"Checking for any Etcd downtime","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:46:01+05:30","msg":"Etcd Cluster is healthy and there is no downtime","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  STEP: Deleting etcd @ 03/17/25 09:46:01.421
  {"level":"info","ts":"2025-03-17T09:46:01+05:30","msg":"Checking if etcd is gone","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:46:15+05:30","msg":"Checking if statefulset is gone","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:46:15+05:30","msg":"Checking if configmap is gone","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:46:15+05:30","msg":"Checking client service is gone","etcd":{"name":"defrag-zero-downtime-etcd","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:46:15+05:30","msg":"Deleting PVCs"}
  << Timeline
------------------------------
• [406.730 seconds]
Etcd Compaction when compaction is enabled for single-node etcd with provider aws should test compaction on backup
/Users/I539698/Desktop/review/etcd-druid/test/e2e/etcd_compaction_test.go:53

  Captured StdOut/StdErr Output >>
  time="2025-03-17T09:39:47+05:30" level=info msg="Bucket versioning is not found to be enabled."
  << Captured StdOut/StdErr Output

  Timeline >>
  STEP: Purge snapstore @ 03/17/25 09:39:47.596
  STEP: Create etcd @ 03/17/25 09:39:47.634
  {"level":"info","ts":"2025-03-17T09:39:47+05:30","msg":"Waiting for etcd to become ready","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:57+05:30","msg":"etcd is ready","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:57+05:30","msg":"Checking statefulset","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:57+05:30","msg":"Checking configmap","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:40:57+05:30","msg":"Checking client service","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  STEP: Create debug pod @ 03/17/25 09:40:57.983
  STEP: Check initial snapshot is available @ 03/17/25 09:41:10.019
  STEP: Put keys into etcd @ 03/17/25 09:41:10.084
  {"level":"info","ts":"2025-03-17T09:41:10+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-1","fromValue":"bar-1","toKey":"foo-10","toValue":"bar-10"}
  {"level":"info","ts":"2025-03-17T09:41:14+05:30","msg":"put key-value successful","key":"foo-1","value":"bar-1"}
  {"level":"info","ts":"2025-03-17T09:41:14+05:30","msg":"put key-value successful","key":"foo-2","value":"bar-2"}
  {"level":"info","ts":"2025-03-17T09:41:14+05:30","msg":"put key-value successful","key":"foo-3","value":"bar-3"}
  {"level":"info","ts":"2025-03-17T09:41:14+05:30","msg":"put key-value successful","key":"foo-4","value":"bar-4"}
  {"level":"info","ts":"2025-03-17T09:41:15+05:30","msg":"put key-value successful","key":"foo-5","value":"bar-5"}
  {"level":"info","ts":"2025-03-17T09:41:15+05:30","msg":"put key-value successful","key":"foo-6","value":"bar-6"}
  {"level":"info","ts":"2025-03-17T09:41:15+05:30","msg":"put key-value successful","key":"foo-7","value":"bar-7"}
  {"level":"info","ts":"2025-03-17T09:41:15+05:30","msg":"put key-value successful","key":"foo-8","value":"bar-8"}
  {"level":"info","ts":"2025-03-17T09:41:15+05:30","msg":"put key-value successful","key":"foo-9","value":"bar-9"}
  {"level":"info","ts":"2025-03-17T09:41:16+05:30","msg":"put key-value successful","key":"foo-10","value":"bar-10"}
  STEP: Check snapshot after putting data into etcd @ 03/17/25 09:41:16.238
  STEP: Put additional data into etcd @ 03/17/25 09:41:16.364
  {"level":"info","ts":"2025-03-17T09:41:16+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-11","fromValue":"bar-11","toKey":"foo-15","toValue":"bar-15"}
  {"level":"info","ts":"2025-03-17T09:41:19+05:30","msg":"put key-value successful","key":"foo-11","value":"bar-11"}
  {"level":"info","ts":"2025-03-17T09:41:19+05:30","msg":"put key-value successful","key":"foo-12","value":"bar-12"}
  {"level":"info","ts":"2025-03-17T09:41:20+05:30","msg":"put key-value successful","key":"foo-13","value":"bar-13"}
  {"level":"info","ts":"2025-03-17T09:41:20+05:30","msg":"put key-value successful","key":"foo-14","value":"bar-14"}
  {"level":"info","ts":"2025-03-17T09:41:20+05:30","msg":"put key-value successful","key":"foo-15","value":"bar-15"}
  STEP: Trigger on-demand delta snapshot @ 03/17/25 09:41:20.536
  {"level":"info","ts":"2025-03-17T09:41:20+05:30","msg":"waiting for compaction job to become successful"}
  {"level":"info","ts":"2025-03-17T09:46:25+05:30","msg":"compaction job is successful"}
  STEP: Put additional data into etcd @ 03/17/25 09:46:25.321
  {"level":"info","ts":"2025-03-17T09:46:25+05:30","msg":"populating etcd with sequential key-value pairs","fromKey":"foo-16","fromValue":"bar-16","toKey":"foo-20","toValue":"bar-20"}
  {"level":"info","ts":"2025-03-17T09:46:29+05:30","msg":"put key-value successful","key":"foo-16","value":"bar-16"}
  {"level":"info","ts":"2025-03-17T09:46:29+05:30","msg":"put key-value successful","key":"foo-17","value":"bar-17"}
  {"level":"info","ts":"2025-03-17T09:46:29+05:30","msg":"put key-value successful","key":"foo-18","value":"bar-18"}
  {"level":"info","ts":"2025-03-17T09:46:29+05:30","msg":"put key-value successful","key":"foo-19","value":"bar-19"}
  {"level":"info","ts":"2025-03-17T09:46:30+05:30","msg":"put key-value successful","key":"foo-20","value":"bar-20"}
  STEP: Trigger next on-demand delta snapshot @ 03/17/25 09:46:30.125
  STEP: Verify that there are new delta snapshots as compaction is not triggered yet because delta events have not reached next 15 revision @ 03/17/25 09:46:30.187
  STEP: Deleting debug pod @ 03/17/25 09:46:30.259
  STEP: Deleting etcd @ 03/17/25 09:46:30.264
  {"level":"info","ts":"2025-03-17T09:46:30+05:30","msg":"Checking if etcd is gone","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:46:34+05:30","msg":"Checking if statefulset is gone","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:46:34+05:30","msg":"Checking if configmap is gone","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:46:34+05:30","msg":"Checking client service is gone","etcd":{"name":"etcd-compaction-aws","namespace":"shoot"}}
  {"level":"info","ts":"2025-03-17T09:46:34+05:30","msg":"Deleting PVCs"}
  << Timeline
------------------------------
[SynchronizedAfterSuite] PASSED [0.000 seconds]
[SynchronizedAfterSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:117
------------------------------
[SynchronizedAfterSuite] PASSED [24.858 seconds]
[SynchronizedAfterSuite]
/Users/I539698/Desktop/review/etcd-druid/test/e2e/suite_test.go:117

  Timeline >>
  {"level":"info","ts":"2025-03-17T09:46:34+05:30","msg":"Checking for Etcd resources before deleting namespace","namespace":"shoot","namespace":"shoot"}
  {"level":"info","ts":"2025-03-17T09:46:34+05:30","msg":"No Etcd resources found; proceeding with namespace deletion","namespace":"shoot","namespace":"shoot"}
  << Timeline
------------------------------

Ran 8 of 8 Specs in 412.870 seconds
SUCCESS! -- 8 Passed | 0 Failed | 0 Pending | 0 Skipped

@ishan16696 ishan16696 merged commit 63690d3 into gardener:master Mar 17, 2025
15 checks passed
@gardener-robot gardener-robot added the status/closed Issue is closed (either delivered or triaged) label Mar 17, 2025
@ishan16696 ishan16696 deleted the revert/tempFix branch March 17, 2025 04:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@unmarshall unmarshall unmarshall approved these changes

Assignees
No one assigned
Labels
area/testing Testing related kind/test Test needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) needs/review Needs review size/xs Size of pull request is tiny (see gardener-robot robot/bots/size.py) status/closed Issue is closed (either delivered or triaged)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@ishan16696 @unmarshall @gardener-robot-ci-1 @gardener-robot-ci-3 @gardener-robot
0