[4.18 system test] Test bucket replication with object versioning #11111
Conversation
15b08bd to
8a2e43f
Compare
a82967a to
48f3551
Compare
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-vmf03
Cluster Configuration: conf/deployment/vsphere/ipi_1az_rhcos_vsan_3m_3w.yaml conf/ocsci/enable_huge_pages.yaml conf/ocsci/encryption_in_transit.yaml
PR Test Suite:
PR Test Path: tests/cross_functional/system_test/test_mcg_replication_with_disruptions.py::TestMCGReplicationWithVersioningSystemTest::test_bucket_replication_with_versioning_system_test
Additional Test Params:
OCP VERSION: 4.18
OCS VERSION: 4.18
tested against branch: master
Job UNSTABLE (some or all tests failed).
927ca32 to
b4c0c15
Compare
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-vmf03
Cluster Configuration: conf/deployment/vsphere/ipi_1az_rhcos_vsan_3m_3w.yaml conf/ocsci/enable_huge_pages.yaml conf/ocsci/encryption_in_transit.yaml
PR Test Suite:
PR Test Path: tests/cross_functional/system_test/test_mcg_replication_with_disruptions.py::TestMCGReplicationWithVersioningSystemTest::test_bucket_replication_with_versioning_system_test
Additional Test Params:
OCP VERSION: 4.18
OCS VERSION: 4.18
tested against branch: master
Job UNSTABLE (some or all tests failed).
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-vmf03
Cluster Configuration: conf/deployment/vsphere/ipi_1az_rhcos_vsan_3m_3w.yaml conf/ocsci/enable_huge_pages.yaml conf/ocsci/encryption_in_transit.yaml
PR Test Suite:
PR Test Path: tests/cross_functional/system_test/test_mcg_replication_with_disruptions.py::TestMCGReplicationWithVersioningSystemTest::test_bucket_replication_with_versioning_system_test
Additional Test Params:
OCP VERSION: 4.18
OCS VERSION: 4.18
tested against branch: master
Job UNSTABLE (some or all tests failed).
b4c0c15 to
376d3c3
Compare
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-vmf26
Cluster Configuration: conf/deployment/vsphere/ipi_1az_rhcos_vsan_3m_3w.yaml conf/ocsci/enable_huge_pages.yaml conf/ocsci/encryption_in_transit.yaml
PR Test Suite:
PR Test Path: tests/cross_functional/system_test/test_mcg_replication_with_disruptions.py::TestMCGReplicationWithVersioningSystemTest::test_bucket_replication_with_versioning_system_test
Additional Test Params:
OCP VERSION: 4.18
OCS VERSION: 4.18
tested against branch: master
Job UNSTABLE (some or all tests failed).
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-vmf26
Cluster Configuration: conf/deployment/vsphere/ipi_1az_rhcos_vsan_3m_3w.yaml conf/ocsci/enable_huge_pages.yaml conf/ocsci/encryption_in_transit.yaml
PR Test Suite:
PR Test Path: tests/cross_functional/system_test/test_mcg_replication_with_disruptions.py::TestMCGReplicationWithVersioningSystemTest::test_bucket_replication_with_versioning_system_test
Additional Test Params:
OCP VERSION: 4.18
OCS VERSION: 4.18
tested against branch: master
Job UNSTABLE (some or all tests failed).
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-vmf26
Cluster Configuration: conf/deployment/vsphere/ipi_1az_rhcos_vsan_3m_3w.yaml conf/ocsci/enable_huge_pages.yaml conf/ocsci/encryption_in_transit.yaml
PR Test Suite:
PR Test Path: tests/cross_functional/system_test/test_mcg_replication_with_disruptions.py::TestMCGReplicationWithVersioningSystemTest::test_bucket_replication_with_versioning_system_test
Additional Test Params:
OCP VERSION: 4.18
OCS VERSION: 4.18
tested against branch: master
Job UNSTABLE (some or all tests failed).
1b2d675 to
d82a5a8
Compare
0a40c29 to
12f0134
Compare
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-vmf28
Cluster Configuration: conf/deployment/vsphere/ipi_1az_rhcos_vsan_3m_3w.yaml conf/ocsci/enable_huge_pages.yaml conf/ocsci/encryption_in_transit.yaml
PR Test Suite:
PR Test Path: tests/cross_functional/system_test/test_mcg_replication_with_disruptions.py::TestMCGReplicationWithVersioningSystemTest::test_bucket_replication_with_versioning_system_test
Additional Test Params:
OCP VERSION: 4.18
OCS VERSION: 4.18
tested against branch: master
Job UNSTABLE (some or all tests failed).
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-vmf28
Cluster Configuration: conf/deployment/vsphere/ipi_1az_rhcos_vsan_3m_3w.yaml conf/ocsci/enable_huge_pages.yaml conf/ocsci/encryption_in_transit.yaml
PR Test Suite:
PR Test Path: tests/cross_functional/system_test/test_mcg_replication_with_disruptions.py::TestMCGReplicationWithVersioningSystemTest::test_bucket_replication_with_versioning_system_test
Additional Test Params:
OCP VERSION: 4.18
OCS VERSION: 4.18
tested against branch: master
Job UNSTABLE (some or all tests failed).
12f0134 to
e4fe273
Compare
ocs_ci/ocs/bucket_utils.py
Outdated
| def verify_deletion_marker(mcg_obj, awscli_pod, bucket_name, object_key): | ||
| """ | ||
| Verify if deletion marker exists for the given object key |
There was a problem hiding this comment.
It's possible to have a delete marker with IsLatest=false:
$ my_s3 api put-bucket-versioning --bucket first.bucket --versioning-configuration Status=Enabled
$ echo 'a0' | my_s3 cp - s3://first.bucket/a
$ echo 'a1' | my_s3 cp - s3://first.bucket/a
$ my_s3 rm s3://first.bucket/a
delete: s3://first.bucket/a
$ echo 'a2' | my_s3 cp - s3://first.bucket/a
$ my_s3 api list-object-versions --bucket first.bucket | jq .DeleteMarkers
[
{
"Owner": {
"DisplayName": "admin@noobaa.io",
"ID": "67ebd24eaac453001d17838d"
},
"Key": "a",
"VersionId": "nbver-3",
"IsLatest": false,
"LastModified": "2025-04-02T14:00:57+00:00"
}
]
So please refine the method's name and docstring accordingly:
| def verify_deletion_marker(mcg_obj, awscli_pod, bucket_name, object_key): | |
| """ | |
| Verify if deletion marker exists for the given object key | |
| def verify_soft_deletion(mcg_obj, awscli_pod, bucket_name, object_key): | |
| """ | |
| Verify if deletion marker exists and IsLatest for the given object key |
There was a problem hiding this comment.
@sagihirshfeld sure. why do you call it soft deletion?
There was a problem hiding this comment.
Soft deletion means it's an impermanent and reversible operation.
On a versioned bucket if an object has a delete marker with IsLatest=True, it means we can undelete it by deleting the delete marker, thus the operation that would cause it is considered soft-deletion. Hard-deletion would be if we deleted the version itself via aws s3api delete-object --bucket my-bucket --key my-object --version-id abc123.
This is all just terminology though - use whatever name you like as long as it's not misleading like the previous one.
| @mcg | ||
| @magenta_squad | ||
| @system_test | ||
| class TestMCGReplicationWithVersioningSystemTest: |
There was a problem hiding this comment.
Log-based replication assumes connectivity to AWS:
| class TestMCGReplicationWithVersioningSystemTest: | |
| @skipif_aws_creds_are_missing | |
| @skipif_disconnected_cluster | |
| class TestMCGReplicationWithVersioningSystemTest: |
0d1d651 to
ff1b009
Compare
|
New changes are detected. LGTM label has been removed. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: mashetty330 The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
1 similar comment
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: mashetty330 The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
This PR raised against master (release-4.19 as of today) is closed because of the change in noobaa db HA implementation. This would break many of the MCG system test regression runs as the step to perform Noobaa db backup and recovery will be completely revamped. New PR against release-4.18 branch is raised: #11871 |
ff1b009 to
b355232
Compare
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-vm24
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/cross_functional/system_test/test_mcg_replication_with_disruptions.py::TestMCGReplicationWithVersioningSystemTest
Additional Test Params:
OCP VERSION: 4.19
OCS VERSION: 4.19
tested against branch: master
Job UNSTABLE (some or all tests failed).
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-vm27
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/cross_functional/system_test/test_mcg_replication_with_disruptions.py::TestMCGReplicationWithVersioningSystemTest
Additional Test Params:
OCP VERSION: 4.19
OCS VERSION: 4.19
tested against branch: master
Job UNSTABLE (some or all tests failed).
Signed-off-by: Mahesh Shetty <mashetty@Maheshs-MacBook-Pro.local>
Signed-off-by: Mahesh Shetty <mashetty@Maheshs-MacBook-Pro.local>
Signed-off-by: Mahesh Shetty <mashetty@Maheshs-MacBook-Pro.local>
Signed-off-by: Mahesh Shetty <mashetty@Maheshs-MacBook-Pro.local>
Signed-off-by: Mahesh Shetty <mashetty@Maheshs-MacBook-Pro.local>
Signed-off-by: Mahesh Shetty <mashetty@Maheshs-MacBook-Pro.local>
Signed-off-by: Mahesh Shetty <mashetty@Maheshs-MacBook-Pro.local>
538982e to
08c019a
Compare
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-vm27
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/cross_functional/system_test/test_mcg_replication_with_disruptions.py::TestMCGReplicationWithVersioningSystemTest
Additional Test Params:
OCP VERSION: 4.19
OCS VERSION: 4.19
tested against branch: master
Job UNSTABLE (some or all tests failed).
There was a problem hiding this comment.
PR validation on existing cluster
Cluster Name: mashetty-vm27
Cluster Configuration:
PR Test Suite:
PR Test Path: tests/cross_functional/system_test/test_mcg_replication_with_disruptions.py::TestMCGReplicationWithVersioningSystemTest
Additional Test Params:
OCP VERSION: 4.19
OCS VERSION: 4.19
tested against branch: master
This PR covers the system test automation for 4.18 feature "MCG Bucket Replication - Support versioning"