8000 [BUG] sync `LightevalTaskConfig` docstring with types/defaults · Issue #748 · huggingface/lighteval · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
[BUG] sync LightevalTaskConfig docstring with types/defaults #748
Open
@jamesbraza

Description

@jamesbraza

Describe the bug

The docstring in LightevalTaskConfig (link) lags the type hints:

  • evaluation_splits says list[str] but it's actually ListLike so it can be tuple. Also, the default of validation isn't documented
    • Imo a default of validation should not be there, as most people just have train/test
  • few_shots_split says str, but it's actually optional with a default of None

There's more cases (e.g. generation_size) but I didn't state them here.

To Reproduce

N/a

Expected behavior

Up-to-date docstrings. For typed code, it's a best practice to not to restate types in the docstring for this very reason.

Version info

lighteval==0.9.2

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdocumentationImprovements or additions to documentation

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0