8000 Update usage on failed json.Unmarshal or validate.Struct · Issue #28 · 567-labs/instructor-go · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Update usage on failed json.Unmarshal or validate.Struct #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ericvanlare opened this issue Jul 12, 2024 · 5 comments
Closed

Update usage on failed json.Unmarshal or validate.Struct #28

ericvanlare opened this issue Jul 12, 2024 · 5 comments

Comments

@ericvanlare
Copy link
Contributor

Response usage for input tokens and output tokens is only counted when it comes from a chat with successful json hydration, usage counts for failures at json.Unmarshal() or validate.Struct() in chat.go are ignored as of now.

Here's a reference I found in instructor-py

I propose instructor-go should count usage from all LLM calls in the retry, successes and failures, and update the successful call's usage counts before returning it.
@ericvanlare
Copy link
Contributor Author

Also in i.chat() there's an err nil check that returns a nil response, could it be changed to a response that's empty except for usage from the call for cases such as chatToolCall() where it will return err for no tool calls, but there was still usage?

@ericvanlare
Copy link
Contributor Author

Created a PR here, @h0rv if you have some time I would greatly appreciate a review on this 🙏 I'm keen to add instructor-go to my projects but accurate usage tracking is a hard requirement

@h0rv h0rv closed this as completed Jul 15, 2024
@h0rv h0rv reopened this Jul 15, 2024
@h0rv
Copy link
Collaborator
h0rv commented Jul 15, 2024

@ericvanlare Now the usage PR is merged, would you be able to add a separate PR to add a section in the README on how to view the usage, noting how the it sums the retries and you can still view the usage even if instructor fails in the end?

@ericvanlare
Copy link
Contributor Author

@h0rv sure, readme PR up here

@h0rv
Copy link
Collaborator
h0rv commented Jul 15, 2024

@ericvanlare Merged - thanks!

@h0rv h0rv closed this as completed Jul 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ericvanlare @h0rv
0