OpenAI's Deep Research is a really cool product. This project is inspired by ByteDance's DeerFlow project, an open-source Deep Research project. DeerFlow is using LangGraph to build the agentic workflow which abstracts away a lot of details, e.g. how the ReAct agent is working.
To make sure I understand how the DeerFlow's Deep Research agent works under the hood, I decided to build it from scratch without relying on any existing agentic framework, e.g. LangGraph.
Note that this project is currently pure backend without any frontend interface.
This project is NOT for production use, since it lacks many production features, e.g. web UI, token streaming, async execution, etc.
But since it is build from scratch, it is easy to see the output of each step in the workflow, e.g.
- for web search, what LLM suggests to search
- for coding, what code the LLM generates
- for planning, what research plan the LLM outputs
- whatever step you are curious about
We need to prepare the API keys for the following services and store them as environment variables:
- OpenAI:
OPENAI_API_KEY - Claude:
ANTHROPIC_API_KEY - Tavily:
TAVILY_API_KEY - Jina:
JINA_API_KEY
Go to the parent directory of this repo and run the following command:
python3 -m nanoDeepResearch.main \
--query "what is the area(land+water) ratio between the largest and smallest states in the US"The example report can be found in example_reports/area_ratio_largest_smallest_state_in_us.md
The ReAct agent is a simple agent that uses a ReAct loop to reason and act. It is inspired by the ReAct paper: https://arxiv.org/abs/2210.03629
Given a task query, the ReAct agent will:
- Reason about the task using the available tools
- Act on the task using the available tools
- Get the observation from the action results
- Repeat the process until the task is completed
Note that all those steps are decided by the LLM without human intervention, really cool!
state_machine.py is the class for the whole Deep Research workflow.
- Planner: the LLM of the planner agent will understand the user query and break it down into a list of task steps
- Research Team: it will take the list of task steps and assign each step to either a researcher agent or a coder agent
- Researcher: it is a ReAct agent and can use
web searchandcrawlertools to solve the task - Coder: it is a ReAct agent and can use
pythonto solve the task - Reporter: it will use the observations from the researcher and coder to generate the final report
make lintThanks to DeerFlow for open-sourcing their project and providing a lot of inspiration.