Problem Statement:
The current method for handling Internal Opinion Requests is fragmented and inefficient. Departments receive documents requiring analysis and provide opinions based on individual expertise, existing strategies, historical context, and relevant documents. This process leads to inconsistent responses, duplicated efforts, and a lack of standardized formatting in summarizing asks, analyses, and recommendations. There's a critical need for a unified system that streamlines this process, ensures consistency across departments, and leverages collective knowledge efficiently.
[info] This sulutions is full offline and does not require any internet connection to run only the first time to download the docker images.
This documentation provides a detailed guide to deploying and running the ADEO solution, designed for the InnovateAI: Streamlining Internal Workflows Hackathon. The solution addresses challenges in automating internal workflows and research tasks.
- Frontend: Built with Nuxt.js (SSR) and styled using TailwindCSS.
- Backend: Powered by FastAPI with real-time WebSocket capabilities via Socket.IO.
- AI Model: Ollama’s
llama3.1:8b, deployed using Docker. - Database: Redis is used for storing chat history efficiently in memory, enabling fast access and updates.
Callout: The current configuration uses Ollama with CPU by default. If you want to enable GPU acceleration, refer to the Ollama GPU Setup Guide for instructions on configuring GPU support in Docker.
Redis is used to store and manage chat history, ensuring efficient handling of conversation data. It acts as an in-memory database, providing low-latency access to historical messages for quick retrieval and updates during interactions.
-
Usage:
- Incoming and outgoing messages are stored as key-value pairs in Redis.
- Redis keys are structured to manage multiple conversations or user-specific histories.
-
Redis Configuration: Redis is configured in the
docker-compose.ymlfile and runs on port6379. Ensure it is running alongside other services.
If you want to use a different AI model with Ollama, you can customize it by editing the following file in the backend service:
File Path: api/app/service/ollama_service.py
Locate the payload dictionary in the file:
payload = {
"model": "llama3.1:8b",
"messages": existing_history,
"stream": False,
}-
Replace
"llama3.1:8b"with the name of the model you want to use.- Example:
"model": "your-preferred-model-name"
- Example:
-
Save the changes and restart the backend service.
To enable GPU usage for Ollama, follow the instructions provided in their official documentation:
Ollama Docker GPU Setup Guide
-
Ollama API Documentation:
For details on using the Ollama API, refer to the Ollama API Documentation. -
Model Library:
Browse and select from a variety of models in the Ollama Model Library.
Ensure the following tools are installed:
- Docker: Install Docker
- Docker Compose: Install Docker Compose
Clone the project repository to your local machine:
git clone <your-project-repository-url>
cd <your-project-directory>-
Start Services: Use Docker Compose to build and start all services:
docker-compose up --build
-
Verify Running Containers: Check that all containers are running and retrieve the CONTAINER ID for the Ollama container:
docker ps
Look for the container with the
IMAGEnameollama/ollamain the output and note its CONTAINER ID. -
Initialize the Ollama Model: After all services are up, ensure the Ollama model (
llama3.1:8b) is operational by running:docker exec -it <CONTAINER_ID> ollama run llama3.1:8b
[info] If you encounter an error, retry the command—it will work after the model is initialized properly.
Once successful, you will see this message:
>>> Send a message (/? for help)
Your Ollama model is now ready to process messages. Proceed to the next step.
Replace
<CONTAINER_ID>with the actual CONTAINER ID for the Ollama container from thedocker psoutput. -
Note on Response Time:
The response time of the model will depend on your system's performance. Keep monitoring the terminal logs for updates and ensure the services are running smoothly.
After starting the services, you can access them via the following endpoints:
- Frontend (Nuxt.js): http://localhost:3000
- Backend API (FastAPI): http://localhost:8000
- Ollama AI Model: Accessible through the backend API on port
11434. - Redis Database: Available on port
6379for internal use.