8000 Fix memory step model output in ToolCallingAgent by keetrap · Pull Request #1156 · huggingface/smolagents · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Fix memory step model output in ToolCallingAgent #1156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 9, 2025

Conversation

keetrap
Copy link
Contributor
@keetrap keetrap commented Apr 7, 2025

Fixes #1143

@aymeric-roucher
Copy link
Collaborator
aymeric-roucher commented Apr 8, 2025

Nice, thank you! Could you also test the ToolCallingAgent's replay function to make sure that the issue #1143 does not reappear?

@keetrap
Copy link
Contributor Author
keetrap commented Apr 8, 2025

I tried to add the test like shown below

  agent = ToolCallingAgent(tools=[PythonInterpreterTool()], model=FakeToolCallModel(), verbosity_level=0)
  agent.run("What is 2 multiplied by 3.6452?")
  with agent.logger.console.capture() as capture:
      agent.replay()
  
  output = capture.get().replace("\n", "")
  assert "Called Tool" in output

but captured output includes formatting codes.

assert 'Called Tool' in "\x1b[2;36m[16:57:38]\x1b[0m\x1b[2;36m \x1b[0mReplaying the agent's steps:                                                                                   
                                                      \x1b]8;id=669843;file://C:\\Users\\parte\\Desktop\\Open_Source\\smolagents\\src\\smolagents\\memory.py\x1b\\\x1b[2mmemory.py\x1b[0m\x1b]8;;\x1b\\\x1b[2m:\x1b[0m\x1b]8;id=627247;file://C:\\Users\\parte\\Desktop\\Open_Source\\smolagents\\src\\smolagents\\memory.py#214\x1b\\\x1b[2m214\x1b[0m\x1b]8;;\x1b\\\x1b[38;2;212;183;2m╭─\x1b[0m\x1b[38;2;212;183;2m────────────────────────────────────────────────────────────────────────────────────────\x1b[0m\x1b[38;2;212;183;2m \x1b[0m\x1b[1;38;2;212;183;2mNew run\x1b[0m\x1b[38;2;212;183;2m \x1b[0m\x1b[38;2;212;183;2m────────────────────────────────────────────────────────────────────────────────────────\x1b[0m\x1b[38;2;212;183;2m─╮\x1b[0m\x1b[38;2;212;183;2m│\x1b[0m                                                                                                                                                                                    
       \x1b[38;2;212;183;2m│\x1b[0m\x1b[38;2;212;183;2m│\x1b[0m \x1b[1mWhat is 2 multiplied by 3...;2;230;237;243;48;2;13;17;23m'2*3.6452'}\x1b[0m\x1b[48;2;13;17;23m                        
                                                                                               \x1b[0m\x1b[1;3mAgent output:\x1b[0m \x1b[38;2;212;183;2m───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────\x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23mCalled\x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m \x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23mTool:\x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m \x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m'final_answer'\x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m \x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23mwith\x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m \x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23marguments:\x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m \x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m{'answer':\x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m \x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m'7.2904'}\x1b[0m\x1b[48;2;13;17;23m                                                                                                                             \x1b[0m"

@aymeric-roucher
Copy link
Collaborator

@keetrap I see string Called\x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23m \x1b[0m\x1b[38;2;230;237;243;48;2;13;17;23mTool: in output: maybe assert 'Called' in output and 'Tool:' in output?

@@ -160,7 +160,7 @@ def test_streaming_agent_image_output(self):
)
)

self.assertEqual(len(outputs), 5)
self.assertEqual(len(outputs), 6)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have one extra ChatMessage in outputs here.
In this case it is ChatMessage(content="Called Tool: 'final_answer' with arguments: {'answer': 'image'}", role='assistant', metadata={}, options=[])
``

@sysradium
Copy link
Contributor
sysradium commented Apr 8, 2025

You can either remove ansi sequences before asserting, something like:

def remove_ansi(text):
    ansi_escape = re.compile(r'\x1B\[[0-?]*[ -/]*[@-~]')
    return ansi_escape.sub('', text)

or just disable them altogether by changing AgentLogger:

logger = AgentLogger(2)
logger.console = Console(force_terminal=False)
# inject or monkeypatch the logger

Actually I have no idea why Console isn't DI-able: https://github.com/huggingface/smolagents/blob/main/src/smolagents/monitoring.py#L89
Maybe the time has come to change that.

@aymeric-roucher
Copy link
Collaborator

@sysradium agree that being able to pass a console arg upon logger initialization would be better! If you have some time, it'd be great to make a PR for this!

Copy link
Collaborator
@aymeric-roucher aymeric-roucher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @keetrap !

@aymeric-roucher aymeric-roucher merged commit 3a25900 into huggingface:main Apr 9, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] agent.replay errors
3 participants
0