A Python library providing utility functions for working with Large Language Models (LLMs).
LLM Utils is a collection of utility functions and classes designed to simplify working with language models, particularly for chat applications, message formatting, and model interactions. It provides tools for transforming message formats, managing chat sessions, grouping messages by length, and interacting with various LLM providers.
- Message Formatting: Transform between different chat message formats (ChatML, ShareGPT)
- Chat Sessions: Manage conversation history and interactions with LLMs
- Message Grouping: Efficiently batch messages based on length for optimal processing
- Model Integration: Interface with various LLM backends including OpenAI models
- Visualization: Display chat messages and conversations as HTML in Jupyter notebooks
- Command Line Tools: VLLM serving and load balancing utilities
# Install from PyPI
pip install speedy_llm_utils
# Install from source
pip install -e .
# Or using Poetry
poetry installNote: While the package is named speedy_llm_utils on PyPI, you still import it as llm_utils in your code:
import llm_utils- Python ≥ 3.9
- numpy
- pandas
- google-generativeai
- transformers
- dspy
- langchain
- langchain-core
- langchain-openai
- json-repair
from llm_utils import transform_messages, transform_messages_to_chatml
# Transform messages between formats
chatml_messages = transform_messages_to_chatml(messages)from llm_utils import OAI_LM
# Initialize LLM instance
lm = OAI_LM(model_name="gpt-3.5-turbo")
# Create a chat session
session = lm.create_chat_session(
system_prompt="You are a helpful assistant."
)
# Generate a response
response = session.send_message("Hello, how can you help me today?")
print(response)from llm_utils import split_indices_by_length, group_messages_by_len
# Group messages by length for efficient processing
batches = split_indices_by_length(
lengths=[len(msg) for msg in messages],
batch_size_by_mean_length=1000,
random_seed=42,
verbose=True,
shuffle=True
)from llm_utils import display_chat_messages_as_html, display_conversations
# Display conversations in a Jupyter notebook
display_
6802
conversations(conversations)LLM Utils provides command-line utilities:
svllm: Start a VLLM serversvllm-lb: Run a VLLM load balancer
# Start a VLLM server
svllm --model <model_name> --port <port>
# Run a load balancer
svllm-lb --servers <server1,server2> --port <port>Contributions are welcome! Please feel free to submit a Pull Request.
[Insert your license information here]
- Anh Vo