Implement SimpleWebSearchTool #1300
Conversation
|
Thank you @albertvillanova ! So you said here that bot detector could be triggered in this duckduckgo search, wouldn't it be better to use Bing by default then? |
|
@aymeric-roucher both DuckDuckGo and Bing can trigger the anti-bot protection. I finally could test both in Colab without triggering it. So, it seems randomly triggered in both cases. Indeed, I was thinking that we could use both (DuckDuckGo, Bing, and possibly others in the future) with an additional param (engine), and round-robin in case the anti-bot is triggered.
I was thinking of implementing this in a future PR, and I implemented first DuckDuckGo, so we can make an equivalent replacement of DuckDuckGoSearchTool in all doc examples. |
|
Ok so let's go with DDG for now, thank you for implementing it, and nice job on finding a way to remove a core dependency! |
Implement SimpleWebSearchTool without 3rd-party dependencies:
HTMLParser, that is a little verboseClose #1295
Currently, it uses DuckDuckGo search provider, but it could be extended to use Bing (and others).
The name could also be:
CC: @aymeric-roucher
Note that the anti-bot protection was not triggered during my tests using Colab (while it was triggered when testing locally).