NVIDIA Agent Intelligence (AIQ) toolkit is a flexible, lightweight, and unifying library that allows you to easily connect existing enterprise agents to data sources and tools across any framework.
Note: Agent Intelligence toolkit was previously known as AgentIQ, however the API has not changed and is fully compatible with previous releases. Users should update their dependencies to depend on
aiqtoolkitinstead ofagentiq. The transitional package namedagentiqis available for backwards compatibility, but will be removed in the future.
-
Framework Agnostic: AIQ toolkit works side-by-side and around existing agentic frameworks, such as LangChain, LlamaIndex, CrewAI, and Microsoft Semantic Kernel, as well as customer enterprise frameworks and simple Python agents. This allows you to use your current technology stack without replatforming. AIQ toolkit complements any existing agentic framework or memory tool you're using and isn't tied to any specific agentic framework, long-term memory, or data source.
-
Reusability: Every agent, tool, and agentic workflow in this library exists as a function call that works together in complex software applications. The composability between these agents, tools, and workflows allows you to build once and reuse in different scenarios.
-
Rapid Development: Start with a pre-built agent, tool, or workflow, and customize it to your needs. This allows you and your development teams to move quickly if you're already developing with agents.
-
Profiling: Use the profiler to profile entire workflows down to the tool and agent level, track input/output tokens and timings, and identify bottlenecks. While we encourage you to wrap (decorate) every tool and agent to get the most out of the profiler, you have the freedom to integrate your tools, agents, and workflows to whatever level you want. You start small and go to where you believe you'll see the most value and expand from there.
-
Observability: Monitor and debug your workflows with any OpenTelemetry-compatible observability tool, with examples using Phoenix and W&B Weave.
-
Evaluation System: Validate and maintain accuracy of agentic workflows with built-in evaluation tools.
-
User Interface: Use the AIQ toolkit UI chat interface to interact with your agents, visualize output, and debug workflows.
-
Full MCP Support: Compatible with Model Context Protocol (MCP). You can use AIQ toolkit as an MCP client to connect to and use tools served by remote MCP servers. You can also use AIQ toolkit as an MCP server to publish tools via MCP.
With AIQ toolkit, you can move quickly, experiment freely, and ensure reliability across all your agent-driven projects.
The following diagram illustrates the key components of AIQ toolkit and how they interact. It provides a high-level view of the architecture, including agents, plugins, workflows, and user interfaces. Use this as a reference to understand how to integrate and extend AIQ toolkit in your projects.
- Documentation: Explore the full documentation for AIQ toolkit.
- Get Started Guide: Set up your environment and start building with AIQ toolkit.
- Examples: Explore examples of AIQ toolkit workflows located in the
examplesdirectory of the source repository. - Create and Customize AIQ toolkit Workflows: Learn how to create and customize AIQ toolkit workflows.
- Evaluate with AIQ toolkit: Learn how to evaluate your AIQ toolkit workflows.
- Troubleshooting: Get help with common issues.
Before you begin using AIQ toolkit, ensure that you meet the following software prerequisites.
- Install Git
- Install Git Large File Storage (LFS)
- Install uv
- Install Python (3.11 or 3.12)
-
Clone the AIQ toolkit repository to your local machine.
git clone git@github.com:NVIDIA/AIQToolkit.git aiqtoolkit cd aiqtoolkit -
Initialize, fetch, and update submodules in the Git repository.
git submodule update --init --recursive
-
Fetch the data sets by downloading the LFS files.
git lfs install git lfs fetch git lfs pull
-
Create a Python environment.
uv venv --seed .venv source .venv/bin/activateMake sure the environment is built with Python version
3.11or3.12. If you have multiple Python versions installed, you can specify the desired version using the--pythonflag. For example, to use Python 3.11:uv venv --seed .venv --python 3.11
You can replace
--python 3.11with any other Python version (3.11or3.12) that you have installed. -
Install the AIQ toolkit library. To install the AIQ toolkit library along with all of the optional dependencies. Including developer tools (
--all-groups) and all of the dependencies needed for profiling and plugins (--all-extras) in the source repository, run the following:uv sync --all-groups --all-extras
Alternatively to install just the core AIQ toolkit without any plugins, run the following:
uv sync
At this point individual plugins, which are located under the
packagesdirectory, can be installed with the following commanduv pip install -e '.[<plugin_name>]'. For example, to install thelangchainplugin, run the following:uv pip install -e '.[langchain]'[!NOTE] Many of the example workflows require plugins, and following the documented steps in one of these examples will in turn install the necessary plugins. For example following the steps in the
examples/simple/README.mdguide will install theaiqtoolkit-langchainplugin if you haven't already done so.In addition to plugins, there are optional dependencies needed for profiling. To install these dependencies, run the following:
uv pip install -e '.[profiling]' -
Verify the installation using the AIQ toolkit CLI
aiq --version
This should output the AIQ toolkit version which is currently installed.
-
Ensure you have set the
NVIDIA_API_KEYenvironment variable to allow the example to use NVIDIA NIMs. An API key can be obtained by visitingbuild.nvidia.comand creating an account.export NVIDIA_API_KEY=<your_api_key>
-
Create the AIQ toolkit workflow configuration file. This file will define the agents, tools, and workflows that will be used in the example. Save the following as
workflow.yaml:functions: # Add a tool to search wikipedia wikipedia_search: _type: wiki_search max_results: 2 llms: # Tell AIQ toolkit which LLM to use for the agent nim_llm: _type: nim model_name: meta/llama-3.1-70b-instruct temperature: 0.0 workflow: # Use an agent that 'reasons' and 'acts' _type: react_agent # Give it access to our wikipedia search tool tool_names: [wikipedia_search] # Tell it which LLM to use llm_name: nim_llm # Make it verbose verbose: true # Retry parsing errors because LLMs are non-deterministic retry_parsing_errors: true # Retry up to 3 times max_retries: 3
-
Run the Hello World example using the
aiqCLI and theworkflow.yamlfile.aiq run --config_file workflow.yaml --input "List five subspecies of Aardvarks"This will run the workflow and output the results to the console.
Workflow Result: ['Here are five subspecies of Aardvarks:\n\n1. Orycteropus afer afer (Southern aardvark)\n2. O. a. adametzi Grote, 1921 (Western aardvark)\n3. O. a. aethiopicus Sundevall, 1843\n4. O. a. angolensis Zukowsky & Haltenorth, 1957\n5. O. a. erikssoni Lönnberg, 1906']
We would love to hear from you! Please file an issue on GitHub if you have any feedback or feature requests.
We would like to thank the following open source projects that made AIQ toolkit possible: