A sophisticated multi-agent LLM orchestration system that enables intelligent task distribution and collaborative problem-solving through specialized AI agents with advanced tool capabilities.
Project Raid implements a hierarchical multi-agent architecture where a Control Agent orchestrates multiple Sub-Agents to solve complex tasks through intelligent decomposition and delegation. The system supports both individual and collaborative agent execution with role-based specialization, secure inter-agent communication, and a comprehensive suite of tools for research, development, and analysis.
- Control Agent: Master orchestrator using ReAct (Reasoning-Action-Observation) cycles
- Sub-Agents: Specialized workers with domain-specific capabilities and advanced tools
- Docker Orchestrator: Container lifecycle management for Sub-Agents
- Message Queue: Redis-based communication system with pub/sub for collaboration
- LLM Backend: Abstracted interface supporting OpenAI and Ollama
- Dynamic Agent Manager: Runtime creation of specialized Sub-Agents
- Collaboration System: Secure direct communication between Sub-Agent groups
- Advanced Tool System: Comprehensive toolkit for web search, code execution, file operations, and more
- 🧠 Intelligent Task Orchestration: Control Agent uses ReAct cycles to plan and execute complex workflows
- 🔧 Dynamic Specialization: Create Sub-Agents with specific roles (financial analyst, data analyst, etc.)
- 🤝 Collaborative Agent Groups: Sub-Agents can work together with predefined, restricted communication
- 🌐 Web Search Integration: Real-time internet research capabilities with multiple search providers
- 🐍 Code Execution: Secure Python code execution with data analysis libraries
- 📁 File Operations: Complete file management within isolated workspaces
- 💻 System Commands: Safe bash command execution in Docker containers
- 🐳 Containerized Execution: Isolated Sub-Agent environments using Docker
- 🔄 Async Message Queues: Reliable Redis-based communication with pub/sub support
- 🎯 Configurable Limits: Control resource usage with agent and collaboration limits
- 🛠️ Extensible Tools: Plugin-based tool system for Sub-Agents
- ✨ Enhanced Stability & Reliability: Increased timeouts for long tasks and robust Docker orchestration ensure agents run dependably on the latest code.
- 🔒 Security-First Design: Strict collaboration restrictions and message validation
websearch: Internet search using DuckDuckGo and SerpAPI- Real-time information gathering
- Multiple search provider fallback
- Credible source prioritization
run_python_code: Secure Python code execution- Data analysis with pandas, numpy, matplotlib
- Statistical computing and visualization
- Sandboxed environment with security restrictions
run_bash_command: Enhanced secure shell command execution (Docker only)- NEW:
output_log_fileParameter: Redirects long outputs (build logs, etc.) to files, preventing context overflows - Intelligence-First Design: Agents automatically use log files + grep/tail for analyzing large command outputs
- Enhanced Security: Whitelist of 50+ allowed commands, forbidden pattern detection, command chaining prevention
- Development Tools: git, gcc, python, npm, maven, gradle, and package managers
- Working Directory Management: Fixed path resolution for reliable command execution
- Timeout Management: Configurable timeouts (30s-300s) with async execution
- NEW:
create_file: Create files with contentread_file: Read file contents with size limitslist_files: List workspace filesdelete_file: Remove files from workspace- Workspace isolation: All file operations in
/tmp/raid_workspace
notification_user: Cross-platform system notifications- Windows: PowerShell toast notifications
- macOS: osascript display notifications
- Linux: notify-send, zenity, or kdialog fallbacks
- Urgency levels: low, normal, critical
- Non-blocking: Async execution without system delays
network_request: Limited HTTP GET requestscalculator: Mathematical calculations and computations
- ✅ Restricted imports (no os, sys, subprocess)
- ✅ Forbidden operations (no exec, eval, file system access)
- ✅ Execution timeout limits (30-120 seconds)
- ✅ Output size restrictions
- ✅ Sandboxed environment isolation
- ✅ Workspace isolation (
/tmp/raid_workspace) - ✅ Allowed file extensions only
- ✅ Path traversal prevention
- ✅ File size limits (10MB max)
- ✅ No system directory access
- ✅ Docker environment requirement
- ✅ Command whitelist approach
- ✅ Dangerous pattern blacklist
- ✅ No system modification commands
- ✅ Safe development tools only
- Python 3.9+
- Docker & Docker Compose
- Redis server
- OpenAI API key OR Ollama installation
- (Optional) SerpAPI key for enhanced web search
-
Clone the repository
git clone <repository-url> cd Raid
-
Install dependencies with CLI support
pip install -e . -
Configure environment
cp .env.example .env # Edit .env with your configuration -
Start Redis
# Using Docker docker run -d -p 6379:6379 redis:alpine # Or using local Redis redis-server
Raid includes a comprehensive CLI for intelligent task processing and system management:
# Check CLI help
raid --help
# RECOMMENDED: Control Agent processing with intelligent orchestration
raid control process "Calculate ROI for $50k investment at 8% over 5 years"
raid control process "Setup React development environment" --show-thinking
raid control interactive # Interactive wizard
# Agent management
raid agents list # List active agents
raid agents stats # Detailed statistics
raid agents health # Health status
# System monitoring
raid status # Quick system overview
raid system health # Full health check
# Profile management
raid profiles list # Available profiles
raid profiles show setup_agent # Show profile details-
Test Sub-Agent functionality
python scripts/test_sub_agent.py
-
Test Control Agent orchestration
python scripts/test_control_agent.py
-
Test dynamic Sub-Agent creation
python scripts/test_dynamic_subagent_creation.py
-
Test collaborative Sub-Agent groups
python scripts/test_collaborative_subagents.py
-
Test advanced tools functionality
python scripts/test_advanced_tools.py
-
Test Setup Agent capabilities
python scripts/test_setup_agent_full.py
| Variable | Description | Default |
|---|---|---|
RAID_LLM_PROVIDER |
LLM provider (openai/ollama) | openai |
OPENAI_API_KEY |
OpenAI API key | - |
RAID_OPENAI_MODEL |
OpenAI model | gpt-4o |
RAID_OLLAMA_URL |
Ollama server URL | http://localhost:11434 |
RAID_OLLAMA_MODEL |
Ollama model | qwen3:30b |
RAID_REDIS_HOST |
Redis host | localhost |
RAID_REDIS_PORT |
Redis port | 6379 |
RAID_MAX_DYNAMIC_SUB_AGENTS |
Max dynamic agents | 5 |
SERP_API_KEY |
SerpAPI key for enhanced web search (optional) | - |
Sub-Agents are configured using YAML profiles in the profiles/ directory:
name: advanced_agent
description: Advanced Sub-Agent with comprehensive tool capabilities
version: "1.0"
llm_config:
provider: openai
model: gpt-4o
max_tokens: 2000
temperature: 0.3
tools:
- calculator
- websearch
- run_python_code
- create_file
- read_file
- list_files
- run_bash_command
system_prompt: |
You are an advanced Sub-Agent with comprehensive capabilities...
docker_config:
base_image: python:3.9-slim
working_dir: /appRaid includes a comprehensive CLI for system management and intelligent task processing:
# Install with CLI support
pip install -e .
# Use the CLI
raid --helpraid control process "<goal>"- Process natural language goals with ReAct reasoningraid control process "<goal>" --show-thinking- Show Control Agent thinking processraid control interactive- Interactive goal creation wizardraid control health- Check Control Agent health and capabilitiesraid control capabilities- Show available meta-tools and features
raid agents list- List all active agents with statusraid agents stats- Detailed statistics and relationshipsraid agents create <profile>- Create new agent from profileraid agents stop <agent-name>- Stop specific agentraid agents cleanup- Remove idle/stale agentsraid agents logs <agent-name>- View agent container logsraid agents health- Check health status of all agents
raid task run <profile> "<prompt>"- Dispatch task to specific agent profileraid task interactive- Interactive task creation wizardraid task collaborative <profile> "<task>" --count N- Multi-agent collaborative tasksraid task templates- Show available task templates
raid status- Quick system status overviewraid system stats- Comprehensive system statisticsraid system health- Perform system health checkraid system config- Show current configurationraid system metrics- Detailed system metrics
raid profiles list- List available agent profilesraid profiles show <profile>- Show profile detailsraid profiles validate <profile>- Validate profile configurationraid profiles create <template> <name>- Create new profile from template
raid collab groups- List collaboration groupsraid collab create <group-name>- Create collaboration groupraid collab status <group-id>- Show group status
--format table(default) - Formatted tables with colors--format json- JSON output for programmatic use--format yaml- YAML output for configuration
- Control Agent Processing - Natural language goals with ReAct reasoning and intelligent orchestration
- Real-time monitoring with agent status and relationships
- Dual task dispatch modes - Control Agent orchestration or direct sub-agent dispatch
- Comprehensive statistics including resource utilization
- Health monitoring with system-wide health checks
- Profile management with validation and templates
- Collaboration support for multi-agent coordination
# RECOMMENDED: Control Agent with intelligent orchestration
raid control process "Calculate ROI for $50k investment at 8% over 5 years"
raid control process "Setup React development environment" --show-thinking
# Alternative: Direct sub-agent dispatch
raid task run calculator_agent "Calculate 15% tip on $85"
raid task run setup_agent "Clone and setup GitHub project"The Setup Agent is a persistent, specialized agent designed for project environment setup and development infrastructure configuration.
- Persistent Lifecycle: Not subject to automatic cleanup, designed for long-running setup tasks
- Advanced LLM Model: Uses o4-mini with 20,000 token capacity for complex setup decisions
- Comprehensive Toolset: Includes repository cloning, environment analysis, dependency installation, and build verification
- Cross-Platform Notifications: Alerts users on Windows, macOS, and Linux when setup is complete
- Enhanced Output Handling: Automatically redirects large build outputs to log files for reliable analysis
- Development Tools: Pre-installed with git, build-essential, Python, Docker, and common development utilities
- Repository Analysis: Clone and analyze project structure and technology stack
- Dependency Management: Install system dependencies and configure package managers
- Build Environment: Set up Python, Node.js, Java, Docker environments as needed
- Configuration: Generate and validate configuration files
- Testing: Run build verification and test suites
- Documentation: Generate setup documentation and troubleshooting guides
- Notification: Alert users when environment is ready
- Languages: Python, Node.js, Java, Go, Rust, and more
- Package Managers: pip, npm, maven, gradle, cargo
- Containerization: Docker and Docker Compose setup
- Databases: Configuration for PostgreSQL, MySQL, Redis
- CI/CD: Pipeline configuration and validation
# Setup a GitHub project
raid task run setup_agent "Setup development environment for: https://github.com/user/project-name"
# Using Control Agent (recommended)
raid control process "Clone and setup the React project at github.com/facebook/react for development"await dispatch_to_sub_agent(
sub_agent_profile="setup_agent",
task_prompt=\"\"\"
Setup development environment for: https://github.com/user/project-name
Requirements:
- Clone repository to /workspace
- Install all dependencies
- Run build verification
- Execute test suite
- Notify when complete
\"\"\",
timeout=1800 # 30 minutes for complex setups
)- Output Management: Large build logs automatically saved to files and analyzed with grep/tail
- Error Recovery: Intelligent diagnosis and fixing of common setup issues
- Resource Monitoring: Tracks resource usage during complex builds
- Cross-Platform: Works consistently across Ubuntu, macOS, and Windows containers
- Persistent Storage: Maintains setup state across container restarts
The Control Agent can automatically create specialized Sub-Agents based on task requirements:
- Financial Analyst: Financial calculations, ROI analysis, market research
- Data Analyst: Statistical analysis, data visualization, trend identification
- Research Analyst: Web research, information synthesis, evidence-based analysis
- Problem Solver: Systematic problem decomposition, multi-tool integration
- Quality Analyst: Validation, quality assurance, accuracy verification
Enhanced Capabilities: All dynamic roles now include advanced tools for comprehensive analysis.
Example:
# Control Agent automatically creates a financial analyst with web search and Python
task = "Research current mortgage rates and calculate payments for $400,000 loan"
result = await control_agent.process_task(task)Revolutionary Feature: The Control Agent can create groups of Sub-Agents that collaborate directly with each other through secure, restricted communication channels.
- Control Agent Assessment: Determines if a task requires multiple agents
- Group Creation: Creates a collaboration group with specific roles
- Secure Communication: Agents communicate through validated message schemas
- Tool Coordination: Agents share results from web searches, calculations, and analyses
- Result Aggregation: Control Agent synthesizes collaborative results
Data Sharing
- Agents share calculation results, research findings, and intermediate data
- Use case: Complex financial analysis requiring web research and calculations
- Restrictions: 20 messages/minute, 45-minute timeout
Validation Chain
- Sequential validation of results by different specialist agents
- Use case: Multi-step verification of research and calculations
- Restrictions: 15 messages/minute, 30-minute timeout
Parallel Analysis
- Multiple agents analyze different aspects simultaneously
- Use case: Comprehensive research from multiple perspectives with tool integration
- Restrictions: 25 messages/minute, 60-minute timeout
Sequential Workflow
- Step-by-step processing where each agent builds on previous work
- Use case: Complex multi-stage problem solving with different tools
- Restrictions: 10 messages/minute, 90-minute timeout
🔒 Security-First Design
- Predefined Groups Only: Collaboration only within Control Agent-created groups
- Message Schema Validation: Strict formats for all inter-agent communication
- Rate Limiting: Configurable maximum messages per minute per agent
- Data Key Whitelisting: Optional restriction of shareable data types
- Message Expiration: Automatic cleanup of expired messages
- Isolated Channels: Each group has its own Redis communication channel
- No Cross-Group Communication: Agents can't communicate outside their group
Message Types (Restricted Format):
DATA_SHARE: Share computed results, research findings, file contentsREQUEST_DATA: Request specific data from another agentSTATUS_UPDATE: Progress updates and tool execution statusCOORDINATION: Coordinate next steps and tool usageVALIDATION: Request result validation and verificationERROR_REPORT: Report issues or tool execution errors
Example 1: Comprehensive Business Analysis
task = """
I need a comprehensive market analysis for launching a SaaS product:
- Research current SaaS market trends and pricing
- Calculate financial projections for different pricing models
- Analyze competitor pricing and features
- Create financial models with Python
- Validate calculations and provide recommendations
Use multiple agents to collaborate on research, analysis, and validation.
"""
result = await control_agent.process_task(task)Example 2: Data Science Project
task = """
Analyze customer churn data and create predictive model:
- Search for latest churn analysis techniques
- Process and analyze the dataset with Python
- Create visualizations and statistical models
- Validate model accuracy and interpret results
- Generate comprehensive report with findings
Collaborate between research, data analysis, and quality assurance agents.
"""
result = await control_agent.process_task(task)Example 3: Research Report Generation
task = """
Create comprehensive report on renewable energy investment opportunities:
- Research current renewable energy market trends
- Analyze investment data and calculate ROI scenarios
- Find latest policy changes and incentives
- Create financial projections and risk assessments
- Compile findings into structured report file
Use collaborative agents for research, analysis, and validation.
"""
result = await control_agent.process_task(task)import asyncio
from raid.config.settings import RaidConfig
from raid.control_agent.agent import ControlAgent
async def main():
config = RaidConfig.from_env()
control_agent = ControlAgent(config)
result = await control_agent.process_task(
"Research current AI development trends and create a summary with key statistics"
)
print(result)
asyncio.run(main())async def advanced_analysis():
config = RaidConfig.from_env()
control_agent = ControlAgent(config)
task = """
I need comprehensive analysis for a tech startup investment decision:
1. Research current AI/ML startup funding trends
2. Calculate investment scenarios for $2M funding
3. Analyze market size and growth projections
4. Create financial models with Python
5. Generate risk assessment report
6. Save all analysis to structured files
Use web search for current data, Python for calculations,
and file operations for deliverables.
"""
result = await control_agent.process_task(task)
print(result)
asyncio.run(advanced_analysis())async def collaborative_research():
config = RaidConfig.from_env()
control_agent = ControlAgent(config)
# Control Agent will automatically create collaborative group if needed
task = """
Comprehensive climate change investment analysis:
Research Requirements:
- Current climate tech investment trends
- Carbon credit market analysis
- Renewable energy ROI calculations
- Policy impact assessments
Analysis Requirements:
- Financial modeling with Python
- Risk-return calculations
- Market size estimations
- Create investment recommendation report
Please create a team of specialized agents to collaborate on
research, financial analysis, and validation of findings.
"""
result = await control_agent.process_task(task)
print(result)
asyncio.run(collaborative_research())discover_sub_agent_profiles- List available Sub-Agent profilesdispatch_to_sub_agent- Send tasks to specific Sub-Agentscreate_specialized_sub_agent- Create dynamic specialized Sub-Agentscreate_collaborative_sub_agent_group- Create groups of Sub-Agents that can collaborate directlyconclude_task_success- Mark tasks as completed successfullyconclude_task_failure- Mark tasks as failed
Raid/
├── src/raid/
│ ├── config/ # Configuration management
│ │ ├── settings.py # Global settings
│ │ ├── sub_agent_config.py # Sub-Agent profiles
│ │ ├── dynamic_subagent.py # Dynamic agent creation
│ │ └── collaboration.py # Collaboration framework
│ ├── control_agent/ # Control Agent implementation
│ │ ├── agent.py # Main Control Agent class
│ │ ├── react_engine.py # ReAct reasoning engine
│ │ └── meta_tools.py # Meta-tools for orchestration
│ ├── sub_agent/ # Sub-Agent implementation
│ │ ├── agent.py # Sub-Agent class with collaboration
│ │ └── main.py # Sub-Agent entry point
│ ├── tools/ # Advanced tool implementations
│ │ ├── base.py # Tool base classes
│ │ ├── calculator.py # Mathematical calculations
│ │ ├── websearch.py # Web search capabilities
│ │ ├── python_executor.py # Python code execution
│ │ ├── file_operations.py # File management
│ │ └── bash_executor.py # Bash command execution
│ ├── docker_orchestrator/ # Container management
│ ├── message_queue/ # Redis-based messaging + pub/sub
│ └── llm_backend/ # LLM abstraction layer
├── profiles/ # Sub-Agent YAML profiles
│ ├── calculator_agent.yaml # Basic calculator agent
│ ├── advanced_agent.yaml # Full-featured agent
│ ├── research_agent.yaml # Research-focused agent
│ └── developer_agent.yaml # Development-focused agent
├── scripts/ # Test and utility scripts
│ ├── test_sub_agent.py
│ ├── test_control_agent.py
│ ├── test_dynamic_subagent_creation.py
│ ├── test_collaborative_subagents.py
│ └── test_advanced_tools.py
└── CLAUDE.md # Claude Code instructions
- Create tool class inheriting from
BaseTool - Implement required methods (
name,description,parameters,execute) - Add security validations and restrictions
- Register in Sub-Agent tool registry
- Add to Sub-Agent profile configurations
- Test with security constraints
- Create YAML file in
profiles/directory - Define capabilities, tools, and system prompt
- Configure Docker environment if needed
- Specify tool combinations for specialization
- Test with Sub-Agent scripts
- Create class inheriting from
MetaTool - Implement required methods and parameters
- Register in
MetaToolRegistry - Add to Control Agent capabilities
- Define new collaboration restrictions in
collaboration.py - Add message type validation
- Update
CreateCollaborativeSubAgentGroupTool - Test with new collaboration scenarios
- Concurrent Processing: Multiple Sub-Agents can process tasks simultaneously
- Collaborative Efficiency: Sub-Agents can share intermediate results to avoid duplicate work
- Advanced Tool Integration: Seamless coordination between web search, code execution, and file operations
- Resource Limits: Configurable limits prevent resource exhaustion
- Container Reuse: Intelligent container lifecycle management
- Async Architecture: Non-blocking operations throughout the system
- Message Optimization: Efficient Redis pub/sub for collaboration
- Automatic Cleanup: Expired messages and inactive groups are automatically removed
- Tool Caching: Intelligent caching of search results and computation outputs
- Sandboxed Execution: All code execution in isolated environments
- Input Validation: Comprehensive validation of all tool inputs
- Output Sanitization: Safe handling of tool outputs and results
- Resource Limits: Memory, CPU, and execution time constraints
- Network Restrictions: Limited and controlled internet access
- No Unauthorized Communication: Sub-Agents can only communicate within predefined groups
- Message Validation: All inter-agent messages must conform to strict schemas
- Rate Limiting: Prevents message flooding and resource abuse
- Data Sanitization: Optional whitelisting of shareable data keys
- Isolation: Each collaboration group operates in complete isolation
- Audit Trail: All collaboration messages are logged and tracked
- Process Isolation: Each Sub-Agent runs in its own Docker container
- Network Isolation: Containers have limited network access
- Resource Limits: CPU and memory constraints prevent resource exhaustion
- File System Restrictions: Limited access to host file system
- Research & Analysis: Web research with data validation
- Financial Modeling: Complex calculations with market research
- Data Science: Statistical analysis with visualization
- Software Development: Code generation with testing and validation
- Content Creation: Research-backed content with fact-checking
- Market Research: Multiple agents handling different aspects (trends, competitors, financial analysis)
- Investment Analysis: Collaborative financial modeling with risk assessment and validation
- Product Development: Research, technical analysis, and market validation by specialized agents
- Academic Research: Literature review, data analysis, and peer validation
- Business Intelligence: Multi-source data gathering, analysis, and strategic recommendations
- Automated Reporting: Web research, data analysis, and document generation
- Due Diligence: Comprehensive company analysis with financial modeling
- Scientific Analysis: Literature research, data processing, and statistical validation
- Competitive Intelligence: Market research, competitor analysis, and strategic planning
- Risk Assessment: Multi-factor analysis with scenario modeling and validation
# Test basic web search and analysis
python -c "
import asyncio
from src.raid.config.settings import RaidConfig
from src.raid.control_agent.agent import ControlAgent
async def main():
config = RaidConfig.from_env()
agent = ControlAgent(config)
result = await agent.process_task('Research the latest developments in quantum computing and summarize key breakthroughs')
print(result)
asyncio.run(main())
"# Test Python code execution with file operations
python -c "
import asyncio
from src.raid.config.settings import RaidConfig
from src.raid.control_agent.agent import ControlAgent
async def main():
config = RaidConfig.from_env()
agent = ControlAgent(config)
result = await agent.process_task('Create a Python script to analyze sales data, generate statistics, and save results to a CSV file')
print(result)
asyncio.run(main())
"# Test multi-agent collaboration
python -c "
import asyncio
from src.raid.config.settings import RaidConfig
from src.raid.control_agent.agent import ControlAgent
async def main():
config = RaidConfig.from_env()
agent = ControlAgent(config)
result = await agent.process_task('Analyze the investment potential of renewable energy stocks: research market trends, calculate financial metrics, and validate findings with multiple specialized agents')
print(result)
asyncio.run(main())
"[License information]
[Contributing guidelines]
Project Raid - Intelligent Multi-Agent Orchestration for Complex Problem Solving
Featuring revolutionary collaborative agent capabilities with enterprise-grade security, advanced tool integration, and real-world problem-solving capabilities.