fix(ci): e2e dd logs failure #23038

pront · 2025-05-13T14:08:40Z

Summary

Changes:

Now dd_url is specified in DD agent config
Fakeintake image was updated to a newer version
The E2E datadog logs test is more robust (using retries instead of sleeping once)
The scripts/int-e2e-test.sh can now be used for local iterations
Improves debug messages, if this fails again we will have better hints

Change Type

Bug fix
New feature
Non-functional (chore, refactoring, docs)
Performance

Is this a breaking change?

Yes
No

How did you test this PR?

Run the following on my machine and on an EC2 instance:

TEST_DATADOG_API_KEY=X RUST_VERSION=1.85 /Users/pavlos.rontidis/CLionProjects/vector/scripts/int-e2e-test.sh e2e datadog-logs

Result:

   Compiling vector v0.47.0 (/home/vector)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 27.53s
    Starting 1 test across 1 binary (2 skipped; run ID: 4036e860-f594-4709-a392-67f785ec0b5c, nextest profile: default)
       START             vector::e2e datadog::logs::validate

running 1 test
2025-05-14T18:06:27.093931Z  INFO e2e::datadog::logs: getting log payloads from agent-only pipeline
2025-05-14T18:06:27.154527Z  INFO e2e::datadog::logs: log events received: 1000
2025-05-14T18:06:27.154638Z  INFO e2e::datadog::logs: getting log payloads from agent-vector pipeline
2025-05-14T18:06:27.209538Z  INFO e2e::datadog::logs: log events received: 1000
test datadog::logs::validate ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 2 filtered out; finished in 0.12s

        PASS [   0.137s] vector::e2e datadog::logs::validate
        SKIP [         ] vector::e2e datadog::metrics::test_decompress_payload_zstd
        SKIP [         ] vector::e2e datadog::metrics::validate
------------
     Summary [   0.138s] 1 test run: 1 passed, 2 skipped
+ RET=0
+ cargo vdev -v e2e stop -a datadog-logs
Executing: "docker" "ps" "-a" "--format" "{{.Names}} {{.State}}"
Executing: "docker" "rm" "--force" "--volumes" "vector-test-runner-1.85"
Stopping service environment
Executing: cd "/home/ubuntu/vector/scripts/e2e/datadog-logs" && DOCKER_SOCKET="/var/run/docker.sock" RUST_VERSION="1.85" VECTOR_NETWORK="vector-integration-tests-datadog-logs" agent_version="latest" "docker" "compose" "--file" "/home/ubuntu/vector/scripts/e2e/datadog-logs/compose.yaml" "down" "--timeout" "0" "--volumes"
  in working directory "/home/ubuntu/vector/scripts/e2e/datadog-logs"
  $DOCKER_SOCKET="/var/run/docker.sock"
  $RUST_VERSION="1.85"
  $VECTOR_NETWORK="vector-integration-tests-datadog-logs"
  $agent_version="latest"
WARN[0000] The "CONFIG_AGENT_VERSION" variable is not set. Defaulting to a blank string.
WARN[0000] The "CONFIG_AGENT_VERSION" variable is not set. Defaulting to a blank string.
WARN[0000] The "CONFIG_VECTOR_IMAGE" variable is not set. Defaulting to a blank string.
WARN[0000] /home/ubuntu/vector/scripts/e2e/datadog-logs/compose.yaml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion
[+] Running 7/7
 ✔ Container datadog-logs-log_generator-1         Removed                                                                                                                     0.0s
 ✔ Container datadog-logs-datadog-agent-vector-1  Removed                                                                                                                     0.2s
 ✔ Container datadog-logs-datadog-agent-1         Removed                                                                                                                     0.2s
 ✔ Container datadog-logs-vector-1                Removed                                                                                                                     0.2s
 ✔ Container datadog-logs-fakeintake-agent-1      Removed                                                                                                                     0.2s
 ✔ Container datadog-logs-fakeintake-vector-1     Removed                                                                                                                     0.1s
 ✔ Volume datadog-logs_log_path                   Removed                                                                                                                     0.0s
+ [[ -n '' ]]
+ exit 0

Does this PR include user facing changes?

Yes. Please add a changelog fragment based on our guidelines.
No. A maintainer will apply the "no-changelog" label to this PR.

Notes

Please read our Vector contributor resources.
Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
The CI checks run only after we manually approve them.
- We recommend adding a pre-push hook, please see this template.
- Alternatively, we recommend running the following locally before pushing to the remote branch:
  - cargo fmt --all
  - cargo clippy --workspace --all-targets -- -D warnings
  - cargo nextest run --workspace (alternatively, you can run cargo test --all)
  - ./scripts/check_changelog_fragments.sh
After a review is requested, please avoid force pushes to help us review incrementally.
- Feel free to push as many commits as you want. They will be squashed into one before merging.
- For example, you can run git merge origin master and git push.
If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
run cargo vdev build licenses to regenerate the license inventory and commit the changes (if any). More details here.

References

datadog-vectordotdev · 2025-05-13T14:50:12Z

Datadog Report

Branch report: pront/fix-e2e-dd-logs
Commit report: f055f61
Test service: vector

✅ 0 Failed, 8 Passed, 0 Skipped, 25.45s Total Time

tilt/Dockerfile

pront

/ci-run-e2e-datadog-logs

pront

/ci-run-integration-datadog-logs

pront · 2025-05-14T18:45:34Z

This didn't fix the original CI failure but it has multiple desirable changes that will help debugging in the future. Merged this manually and on the next failure I will investigate again.

github-actions bot added the domain: ci Anything related to Vector's CI environment label May 13, 2025