10000 feat(remap transform): make `--watch-config` watch external VRL files in `remap` transforms by nekorro · Pull Request #23010 · vectordotdev/vector · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat(remap transform): make --watch-config watch external VRL files in remap transforms #23010

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

feat(remap transform): make --watch-config watch external VRL files in remap transforms #23010

wants to merge 3 commits into from
+35 −2

Conversation

nekorro
Copy link
@nekorro nekorro commented May 8, 2025 •
edited
Loading

Summary

This change ensures that updates to VRL source code are respected when calculating configuration diffs. External VRL files referenced in remap transforms (via file or files fields) are now tracked when --watch-config is enabled, allowing Vector to reload the configuration automatically when these files change.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

How did you test this PR?

Test file field

Added a remap transform to config/vector.yaml :

transforms:
  remap_logs:
    type: remap
    file: vrl/vrl_one
    inputs:
      - demo_logs_one

Started Vector with vector -vvv -c config/vector.yaml -w
Then, modified the /vector/vrl/vrl_one file.
Vector detected the change and reloaded the configuration automatically as expected.
test_1

Test files field

Added a remap transform to config/vector.yaml :

transforms:
  remap_logs:
    type: remap
    files:
      - vrl/vrl_one
      - vrl/vrl_two
    inputs:
      - demo_logs_one

Started Vector with vector -vvv -c config/vector.yaml -w
Then, modified the /vector/vrl/vrl_two file.
Vector detected the change and reloaded the configuration automatically as expected.
test_2

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the "no-changelog" label to this PR.

Notes

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • The CI checks run only after we manually approve them.
    • We recommend adding a pre-push hook, please see this template.
    • Alternatively, we recommend running the following locally before pushing to the remote branch:
      • cargo fmt --all
      • cargo clippy --workspace --all-targets -- -D warnings
      • cargo nextest run --workspace (alternatively, you can run cargo test --all)
      • ./scripts/check_changelog_fragments.sh
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run cargo vdev build licenses to regenerate the license inventory and commit the changes (if any). More details here.

References

@nekorro nekorro requested a review from a team as a code owner May 8, 2025 10:09
@bits-bot
Copy link
bits-bot commented May 8, 2025 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@github-actions github-actions bot added the domain: transforms Anything related to Vector's transform components label May 8, 2025
@nekorro nekorro force-pushed the master branch 2 times, most recently from 0ea7f5e to 52f5547 Compare May 8, 2025 11:20
pront
pront reviewed May 8, 2025
View reviewed changes
nekorro added 3 commits May 10, 2025 15:52
@nekorro
feat(transforms): add files_to_watch support for transforms
967f7de 
Invoke the `files_to_watch` function for applicable transforms to
collect external file paths that should be watched for changes.
@nekorro
feat(remap transfrom): add files_to_watch to track VRL files
df999aa 
Add a `files_to_watch` function to the remap transform to collect file
paths from the `file` and `files` fields. This allows to watch these
files for changes and trigger reloads when the VRL code is modified.
@nekorro
fix(config): relative path handling for external files when `--watch-…
1e3d395 
…config` enabled

Relative paths to external files in the configuration are now correctly
resolved when using --watch-config.
Previously, changes to such files not trigger a reload if the path
provided in the config was relative. This fix ensures that relative
paths are canonicalized, allowing file changes to be properly detected.
@nazarovkv
Copy link
Contributor

Any updates on this PR?

@nekorro
Copy link
Author
nekorro commented May 15, 2025

@pront
Ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pront pront pront left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
domain: transforms Anything related to Vector's transform components
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Make --watch-config respect remap source files
4 participants
@nekorro @bits-bot @nazarovkv @pront
0