Goal: To provide a simple, robust and scalable framework for building LLM applications in Scala.
N.b. This is most definitely a work in progress project and is likely to change significantly over time.
Whilst most LLM work is done in Python we believe that Scala can offer a fundamentally better foundation for building reliable, maintainable AI-powered applications.
The goal of this project is large but unobtainable - take the lessons learned from the Python LLM ecosystem and build even better tooling platform.
This we want to implement (and the equivalent python frameworks):
- Single API access multiple LLM providers (LiteLLM)
- A great tool chain for building LLM apps (LangChain/LangGraph)
- An agentic framework (PydanticAI, CrewAI etc)
- llm4s - main project - contains the core LLM4S framework
- shared - shared code between main project and workspace runner
- workspace-runner - The code that performs the requested actions on the code workspace within the docker container.
- Add a tiktoken port
- Generalize calling API to be able to call anthropic as well.
- Implement full agentic loop code
- Implement a simple tool calling mechanism
Tool calling is a critical integration - make it as simple as possible
e.g. ScalaMeta based tool definition generator
Take a method e.g.
/** My tool does some funky things with a & b...
* @param a: The first thing
* @param b: The second thing
*/
def my_tool(a: Int, b: String): ToolResponse = {
...
}Use ScalaMeta to extract the method params / types, doc comment etc and generate an openai tool definition from it. see: scalameta/scalameta#692 scalameta/scalameta#556
From the tool call request - map the parameters and invoke the method.
This could be code generation - or reflective? Or code generation from the scalameta parse to mean that a lLM request to call a tool can be mapped to the given tool calling arguments.
A lot of tools need to be run in a protected environment (e.g. a docker container to prevent accidental leakage/accidents when for example the LLM tries to do an 'rm -rf /' or similar (even by accident))