Description
I try to evaluate my model on math-ai/minervamath. I used LatexExtractionConfig() in the following way:
latex_gold_metric = multilingual_extractive_match_metric( language=Language.ENGLISH, fallback_mode="first_match", precision=5, gold_extraction_target=(LatexExtractionConfig(),), # Match boxed first before trying other regexes pred_extraction_target=(ExprExtractionConfig(), LatexExtractionConfig(boxed_match_priority=0)), aggregation_function=max, )
def minervamath_prompt_fn(line, task_name: str = None): return Doc( task_name=task_name, query=MATH_QUERY_TEMPLATE.format(Question=line["question"]), choices=[line["answer"]], gold_index=0, )
minervamath = LightevalTaskConfig( name="minervamath", suite=["custom"], prompt_function=minervamath_prompt_fn, hf_repo="math-ai/minervamath", hf_subset="default", hf_avail_splits=["test"], evaluation_splits=["test"], few_shots_split=None, few_shots_select=None, generation_size=32768, metric=[latex_gold_metric], version=1, )
But I have following warning:
How to fix it?