LLM4S provides a simple, robust, and scalable framework for building LLM applications in Scala. While most LLM work is done in Python, we believe that Scala offers a fundamentally better foundation for building reliable, maintainable AI-powered applications.
Note: This is a work in progress project and is likely to change significantly over time.
- Type Safety: Catch errors at compile time rather than runtime
- Functional Programming: Immutable data structures and pure functions for more predictable code
- JVM Ecosystem: Access to a vast array of libraries and tools
- Concurrency: Better handling of concurrent operations with Scala's actor model
- Performance: JVM performance with functional programming elegance
- Containerized Workspace: Secure execution environment for LLM-driven operations
- Workspace Agent Interface: Standardized protocol for file operations and command execution
- Multi-Provider Support: Planned support for multiple LLM providers (OpenAI, Anthropic, etc.)
- llm4s: Main project - contains the core LLM4S framework
- shared: Shared code between main project and workspace runner
- workspaceRunner: Code that performs the requested actions on the workspace within the docker container
- samples: Usage examples demonstrating various features
To get started with the LLM4S project, check out this teaser talk presented by Kannupriya Kalra at the Bay Area Scala Conference. This recording is essential for understanding where we’re headed:
🎥 Teaser Talk: https://www.youtube.com/watch?v=SXybj2P3_DE&ab_channel=SalarRahmanian
- JDK 21+
- SBT
- Docker (for containerized workspace)
sbt compilesbt "samples/runMain org.llm4s.samples.basic.BasicLLMCallingExample"
sbt "samples/runMain org.llm4s.samples.workspace.ContainerisedWorkspaceDemo"Our goal is to implement Scala equivalents of popular Python LLM frameworks:
- Single API access to multiple LLM providers (like LiteLLM)
- A comprehensive toolchain for building LLM apps (like LangChain/LangGraph)
- An agentic framework (like PydanticAI, CrewAI)
- Tokenization utilities (port of tiktoken)
- Full ReAct loop implementation
- Simple tool calling mechanism
Tool calling is a critical integration - we aim to make it as simple as possible:
Using ScalaMeta to automatically generate tool definitions from Scala methods:
/** My tool does some funky things with a & b...
* @param a The first thing
* @param b The second thing
*/
def myTool(a: Int, b: String): ToolResponse = {
// Implementation
}ScalaMeta extracts method parameters, types, and documentation to generate OpenAI-compatible tool definitions.
Mapping LLM tool call requests to actual method invocations through:
- Code generation
- Reflection-based approaches
- ScalaMeta-based parameter mapping
Tools run in a protected Docker container environment to prevent accidental system damage or data leakage.
Interested in contributing? Start here:
LLM4S GitHub Issues: https://lnkd.in/eXrhwgWY
Want to be part of developing this and interact with other developers? Join our Discord community!
LLM4S Discord: https://lnkd.in/eb4ZFdtG
This project is also participating in Google Summer of Code (GSoC) 2025! If you're interested in contributing to the project as a contributor, check out the details here:
👉 Scala Center GSoC Ideas: https://lnkd.in/enXAepQ3
To know everything about GSoC and how it works, check out this talk:
🎥 GSoC Process Explained: https://lnkd.in/e_dM57bZ
Want to connect with maintainers? The LLM4S project is maintained by:
- Rory Graves - https://www.linkedin.com/in/roryjgraves/ | Email: rory.graves@fieldmark.co.uk
- Kannupriya Kalra - https://www.linkedin.com/in/kannupriyakalra/ | Email: kannupriyakalra@gmail.com
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.