Stars
What are the principles we can use to build LLM-powered software that is actually good enough to put in the hands of production customers?
✨ Build a machine learning model from a prompt
A Datacenter Scale Distributed Inference Serving Framework
vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization
Cost-efficient and pluggable Infrastructure components for GenAI inference
Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation
Adds playful pets 🦀🐱🐶 in your VS Code window
A course on aligning smol models.
Awesome-LLM-KV-Cache: A curated list of 📙Awesome LLM KV Cache Papers with Codes.
🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN
Autonomous Agents (LLMs) research papers. Updated Daily.
An active fork of curl-impersonate with more versions and build targets. A series of patches that make curl requests look like Chrome and Firefox.
ShellSage saves sysadmins’ sanity by solving shell script snafus super swiftly
Official inference framework for 1-bit LLMs
LLaVA-CoT, a visual language model capable of spontaneous, systematic reasoning
A simple, high-throughput file client for mounting an Amazon S3 bucket as a local file system.
FUSE-based file system backed by Amazon S3
The Open Cookbook for Top-Tier Code Large Language Model
The modern API client that lives in your terminal.
Accessible large language models via k-bit quantization for PyTorch.
dstack is an open-source alternative to Kubernetes and Slurm, designed to simplify GPU allocation and AI workload orchestration for ML teams across top clouds, on-prem clusters, and accelerators.