8000 [BUG] incorrect type hints such as `callable` · Issue #759 · huggingface/lighteval · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
[BUG] incorrect type hints such as callable #759
New issue
New issue
Open
Bug
Open
[BUG] incorrect type hints such as callable#759
Bug
Labels
bugSomething isn't workingdocumentationImprovements or additions to documentationfeature/enhancementNew feature/request
@jamesbraza

Description

@jamesbraza
jamesbraza
opened on May 21, 2025

Describe the bug

Reading the code, it's clear this code base can really benefit from type checking, running vanilla mypy reveals over 1.2k type errors.

For example, callable is not a valid type hint: https://github.com/huggingface/lighteval/blob/v0.9.2/src/lighteval/metrics/utils/metric_utils.py#L62-L63

To Reproduce

uv sync --extra dev
uv pip install mypy==0.15.0
mypy src

Expected behavior

Type hints to be correct.

Version info

lighteval==0.9.2

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdocumentationImprovements or additions to documentationfeature/enhancementNew feature/request

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions
      0