8000 cli validate - better messages with filenames and line numbers when validating the configuration · Issue #23008 · vectordotdev/vector · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

cli validate - better messages with filenames and line numbers when validating the configuration #23008

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
8000 Open
Daryes opened this issue May 7, 2025 · 1 comment
Open

cli validate - better messages with filenames and line numbers when validating the configuration #23008

Daryes opened this issue May 7, 2025 · 1 comment
Labels
domain: config Anything related to configuring Vector domain: validation type: feature A value-adding code addition that introduce new functionality.

Comments

@Daryes
Copy link
Daryes commented May 7, 2025

A note for the community

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

When using the command vector validate there are 2 types of errors that are too generic and lacking in information : when validating the basic structure of the configuration files (not the parameters, but the format itself), and when parsing a directory of multiple files.
Such situations cause the error message being too generic without indicating neither :

  • the filename containing the error,
  • nor in the file format case, the line number (not even at least the section name).

See after for concrete examples.

Attempted Solutions

1st case : validity of the file structure

$ vector -vvv validate --config-yaml /etc/vector/test.yaml --skip-healthchecks

2025-05-07T10:30:04.688229Z DEBUG vector::app: Internal log rate limit configured. internal_log_rate_secs=10
2025-05-07T10:30:04.688271Z  INFO vector::app: Log level is enabled. level="trace"
2025-05-07T10:30:04.688303Z DEBUG vector::app: messaged="Building runtime." worker_threads=2
2025-05-07T10:30:04.688343Z TRACE mio::poll: registering event source with poller: token=Token(1), interests=READABLE
Failed to load ["/etc/vector/test.yaml"]

x invalid type: unit value, expected any valid TOML value

To reproduce, take the default vector.yaml file provided with the deb or rpm file, get the sinks section and remove the sub-elements to keep only the section header :

sinks:
  # all cleared

The error is not surprising as vector does not accept an empty sinks declaration, but the problem is, there are no information about the line, context or anything helping to understand.
If possible the line number, or at least the section name currently validated would help (here, "sinks")

2nd case : the filename being the source of the error

Same situation, but change the command line to add --config-dir /etc/vector/vector.d and throw in a bunch of yaml files.
You'll get this result :

$ vector -vvv validate --config-yaml /etc/vector/test.yaml --config-dir /etc/vector/vector.d --skip-healthchecks
(...)
Failed to load ["/etc/vector/vector.yaml", "/etc/vector/vector.d"]

x invalid type: unit value, expected any valid TOML value
(or)
x missing field `xxx` in `some id name`
(or)
x error at line number ...

Even if the error provides the line number or the ID name causing the error, there will be no information about the filename itself.
In the case of the validity of the format (the valid TOML error), as no real information is given, it can take a while to find the file, as vector specify that it loaded everything in the message "Failed to load ..."

bonus case : "TOML" message for a YAML file

It was surprising at first, seeing the message expected any valid TOML value while using only YAML files.
No real harm, still.

Proposal

My suggestion is that for each error the validate command can return, it should, in all case, print :

  • the name of the file in error (always)
  • the name of the section in error (always)
    This is partially done with the type.custom_id printed when in error, but it is not constant with all errors
  • the line number (when possible)

References

Didn't see much, but maybe my search-fu is not good enough.

Version

vector v0.46.1
@Daryes Daryes added the type: feature A value-adding code addition that introduce new functionality. label May 7, 2025
@pront
Copy link
Member
pront commented May 8, 2025

Hey @Daryes, I agree the errors do not contain enough information. The bonus case "TOML errors for YAML configs" might be the hardest one to fix so if someone works on this I would advise to start with the first improvements.

@pront pront added domain: config Anything related to configuring Vector domain: validation labels May 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
domain: config Anything related to configuring Vector domain: validation type: feature A value-adding code addition that introduce new functionality.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@pront @Daryes
0