8000 Claude 3.7 Thinking support and budget · Issue #154 · crmne/ruby_llm · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
Claude 3.7 Thinking support and budget #154
@hiemanshu

Description

@hiemanshu

Claude 3.7 Sonnet is a hybrid model, and has a thinking mode, that can be enabled using an API parameter (https://docs.anthropic.com/en/api/messages#body-thinking), and looks like RubyLLM doesn't support that just yet. The API also allows to be able to set a max output tokens and thinking budgets (where thinking budget is lower than max output tokens).
I can't find a way to enable these, making sure I'm not missing anything.

I'm also happy to spend time implementing this feature. I wanted to ask before I started any work on this to avoid duplicate work. I was thinking about starting with a slightly more open .with_arbitrary_options, that would take the options, and add it to the messages API.

Or if anyone else has ideas, happy to discuss/collab to get this going.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0