8000 Fix model access errors unhelpful by endoze · Pull Request #43 · Shopify/roast · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Fix model access errors unhelpful #43

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

endoze
Copy link
Contributor
@endoze endoze commented May 16, 2025

In order to make errors more descriptive when things go wrong, this commit adds a few error types to handle api interactions more gracefully when things go wrong. Specifically, it currently handles when a model can't be found or is innaccesible with the api token provided. Also when an api token isn't provided, instead of an unhelpful NoMethodError, we let end users know the token wasn't provided or isn't valid.

This resolves #13

8000

@endoze endoze force-pushed the fix-model-access-errors-unhelpful branch 2 times, most recently from a7874d4 to 377d6a6 Compare May 16, 2025 16:50
@dblock
Copy link
Contributor
dblock commented May 19, 2025

Looks pretty good, add a test please that hits those errors?

@endoze endoze force-pushed the fix-model-access-errors-unhelpful branch from 377d6a6 to d043935 Compare May 19, 2025 15:02
@endoze
Copy link
Contributor Author
endoze commented May 19, 2025

@dblock Sorry about that. I was unsure of how best to stub/mock in order to trigger these specific errors but I think I've got it figured out now. Have a look and let me know what you think.

@endoze endoze force-pushed the fix-model-access-errors-unhelpful branch from d043935 to 445319e Compare May 19, 2025 15:08
Copy link
Contributor
@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good.

I really dislike catching NoMethodError though - we should prevent this from happening in code before the error happens. Why are we ever calling .chat on a client that could not be initialized? (you can totally split this PR if you want and add the NoResourceFound handling first).

rescue NoMethodError => e
execution_time = Time.now - start_time

if e.message.include?("undefined method 'chat' for nil")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we catch this one earlier? IMO it should never get here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've moved the logic to much earlier in the code for catching this sort of misconfiguration.

@endoze endoze force-pushed the fix-model-access-errors-unhelpful branch 2 times, most recently from 259038c to 25943d5 Compare May 20, 2025 16:32
@dblock
Copy link
Contributor
dblock commented May 20, 2025

Looks a lot better! I think some test paths are missing around raised OpenRouter::ConfigurationError for example? Otherwise LGTM, @obie @parruda should check this out too.

@endoze endoze force-pushed the fix-model-access-errors-unhelpful branch from 25943d5 to 9f3372c Compare May 20, 2025 17:13
@endoze
Copy link
Contributor Author
endoze commented May 20, 2025

@dblock I've added a test that should cover the OpenRouter api token missing. I think this should be enough to ensure we've tested OpenRouter::ConfigurationError is rescued and re-raised as an Roast::AuthenticationError. Thank you for all of your feedback, it's very much appreciated!

Copy link
Contributor
@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM, but I'd want @obie or @parruda to review/merge cause I don't know what I am doing.

In order to make errors more descriptive when things go wrong, this
commit adds a few error types to handle api interactions more gracefully
when things go wrong. Specifically, it currently handles when a model
can't be found or is innaccesible with the api token provided. Also when
an api token isn't provided, instead of an unhelpful NoMethodError, we
let end users know the token wasn't provided or isn't valid.
@endoze endoze force-pushed the fix-model-access-errors-unhelpful branch from 9f3372c to ca8e563 Compare May 23, 2025 03:54
@endoze
84BD Copy link
Contributor Author
endoze commented May 23, 2025

I've rebased against the latest main branch so there shouldn't be any more conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Model access errors are not helpful
2 participants
0