This library simplifies creating and running custom Large Language Model (LLM) tools from your command line. It's a Python re-implementation of the excellent rust-devai, providing a user-friendly interface for building your own LLM-powered applications.
To get started, clone the repository and install it as a
uv tool. After cloning the repository, run the
following commands:
cd cli-llm
uv tool install .By default, this tool uses llama3.2:3b and communicates with it via the llm
Python library and its llm-ollama plugin. You can find the llm
documentation
here.
To use the default model, you'll need ollama installed and running.
- Install
ollamausing the instructions here. - Start the Ollama server in a new terminal:
ollama serve. - Pull the required model:
ollama pull llama3.2:3b.
Alternatively, to use an LLM provider's API, you must set an API key. Follow
the llm library's
instructions
to configure your keys.
This library makes it easy to execute your custom LLM tools. By default, it
searches for tools in the $HOME/.local/share/cli-llm folder (or the
equivalent on other platforms).
Here's an example of how to use the run command:
$ clm run python_file --path testsThis command searches for python_file.py in your tools directory. Inside that
file, it looks for a tool attribute, which must be a click.Command
instance. It then executes the command, passing along any additional arguments
like --path tests.
The LLM tool itself must be defined as a click.Command. For example:
# readme.py
"""README generation tool.
Example usage:
`clm run readme src/ --pattern "*.py"`
"""
from pathlib import Path
import click
from cli_llm import ClmConfig, helpers, run
PROMPT = """
- Below are some {{lang}} files from a library.
- Each file will be listed with its name and then its content.
- Write a README in markdown format that explains how to use the {{lang}} library.
{% for f, contents in files %}
Filename: {{f}}
```{{lang}}
{{contents}}
```
{% endfor %}
"""
@click.command()
@click.argument("path", type=Path)
@click.option("-l", "--lang", type=str, default="")
@click.option("-p", "--pattern", type=str, default="*")
@click.option("-o", "--output", type=Path, default=Path("README.md"))
@click.pass_obj
def tool(config: ClmConfig, path: Path, lang: str, pattern: str, output: Path) -> None:
"""Write a README for a given library."""
file_contents = helpers.gather_file_contents(search_path=path, pattern=pattern)
data = {"files": file_contents, "lang": lang}
ai_response = run(config, PROMPT, data)
ai_response.write_to_file(output)Place this readme.py file in ~/.local/share/cli-llm, and you can run it with:
clm run readme src/ --pattern "*.py"This will generate a new README.md in your current directory based on the
contents of your library. Experiment with the prompt to fine-tune the results!
To see a list of all available tools, run:
clm run --helpYou can create a skeleton for a new tool using the new command:
clm new tool-name --dest tools/You can bundle multiple commands into a single Python file using click's
subcommand feature. As long as the primary entry point is named tool (this
would typically be cli in a standard click project), the library will
automatically discover all its subcommands.
To list the sub-tools within a specific file, run the following command, where
filename is the name of your Python file:
clm run filename --helpFor a practical example, see examples/poetry.py in this repository.
If a Python file in your tools_dir does not have a tool attribute, the
library will emit a warning. To suppress this for a specific file, simply add
tool = None to it.
Configuration is flexible: you can use CLI options, environment variables,
a dedicated config file, or your project's pyproject.toml.
The dedicated config file is located at ~/.config/cli-llm/cli_llm.toml on
Unix-like systems and in equivalent locations on other platforms.
Specifies the LLM model to use.
- Default:
"llama3.2:3b" - Type:
str
Examples:
# pyproject.toml
[tool.cli-llm]
ll_model = "other"# cli_llm.toml
ll_model = "other"The directory or directories to search for tools.
- Default:
~/.local/share/cli-llm - Type:
Path | list[Path]
Examples:
# pyproject.toml
[tool.cli-llm]
tools_dir = "~/tools"# cli_llm.toml
tools_dir = ["~/tools", "~/.local/share/cli-llm"]