8000 Traces appear in evaluations differently from labeling queue · Issue #639 · lmnr-ai/lmnr · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
Traces appear in evaluations differently from labeling queue #639
New issue
New issue
Open
Open
Traces appear in evaluations differently from labeling queue#639
@zsiegel92

Description

@zsiegel92
zsiegel92
opened on Jun 18, 2025

To reproduce:

  1. run an evaluation that involves some LLM output that creates a laminar OTEL trace. If you use the Vercel AI SDK, it'll look like this:

Image

  1. -click the trace, it will have an output that looks like:
{ "score" : 2 }
  1. Click "Add to labeling queue" and select one
  2. Go to the labeling queue - you will see the target as [{"role":"assistant","content":"{\"score\":2}"}]

Image

They should either BOTH show the internal message array format or neither should show it and they should just show content. This is an abstraction over the model SDKs so in my opinion they should both just show the content.

Thank you!