This 4-in-1 demo showcases the power of Oracle's Cloud Platform and AI Services with real-world utility:
- The LLM Playground shows Oracle Cloud Generative AI in action.
- You can play with the hyperparameters and change models with the sidebar.
- USSC Helpline Agent uses Generative AI Agents for a retrieval-augmented generation (RAG) chat.
- The documents used are all publicly available on ussc.gov.
- This was created originally as a tailored demo for the customer.
- The Speech to Text page uses OCI Speech to transcribe, and Generative AI to provide a summary.
- The Summarize Document page uses Generative AI to provide a summary of any text or PDF file.
- The UI is Streamlit, an open-source Python framework running on OCI Compute.
https://genai.pavlakos.cloud (currently not so live)
- Secure & Private: All data remains confidential within your Oracle Cloud tenancy, benefiting from all of the built-in security features.
- Chat with the GenAI Agent: Have a conversation - ask questions and get insightful answers.
- View Citations: Explore the sources behind the agent's responses to validate the responses are grounded.
- Object Storage stores private data files for the RAG knowledge base and speech to text functionality with AES256 encryption.
- Generative AI Agents provides the RAG pipeline as a PaaS service.
- Open Search is a managed service for the knowledge base, automatically indexed for fast search.
- Generative AI Can be either shared or dedicated hosting, with your choice of Cohere and Meta for Large Language Model (LLM).
- Compute - a E5 Flex (1 OCPU) virtual machine hosts the Streamlit app to provide the UI.
- Live Demo App is down
- need to rotate SSL certs and update dns
- You must have an Oracle Cloud Account subscribed to the Chicago, Frankfurt, or London region
- You must already have an Generative AI Agents endpoint available
- this app only provides a front end
- currently only available with whitelisting in Chicago region
- You must set up oci config in order to authenticate to the agent endpoint.
- You must update
.streamlit/secrets.toml- agent_endpoint_id
- compartment_id
- other items as noted in comments
- Did you know you can sign up for an Oracle Cloud Free Tier Account?
- You get $300 in credits for 30 days to use on all available services
- You get
Always Free Resourcesfor as long as you want them including but not limited to:- Oracle Autonmous Database with APEX
- NoSQL Database
- 2 AMD Compute Instances with 1/8 OCPU and 1 GB RAM each
- Up to 4 ARM Compute Instances with a combined total of 4 OCPU and 24 GB of RAM
- 200 GB Block Storage
- 20 GB Object and Archive Storage
- Security Services including Certificates, Valut and Bastion (managed SSH jumpbox)
- Observability Services including Logging, Monitoring, Notifications
- Networking Services including VCN, Load Balancer, Site-to-Site VPN, and 10 TB of outbound data transfer per month
- If you are one of my customers I can get you up to 60 days and $500 in credits. Reach out to me directly and I will get you set up.
- Set up Generative AI Agents service and note the agent_endpoint_id
- Make sure you have port 8501 open on security list
- Launch a VM with Oracle Linux 8 image and attach setup.sh as cloud-init script
- SSH into your VM (opc@ipaddress) and check the log at /home/opc/genai_agent_setup.log
- Run setup.sh if you did not add it as cloud-init script
- Set up OCI CLI config
- Update
.streamlit/secrets.tomlwith your agent_endpoint_id and compartment_id - Use run.sh to run the demo
- Your application will be running on http://server-ip-address:8501
- Set it up as a service so you can restart it like
sudo systemctl restart streamlit.service