Change the repository type filter
All
Repositories list
58 repositories
- The simplest way to serve AI/ML models in production
SGLang-Workshop
PublicTensorRT-LLM
PublicTensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.- A unified library of state-of-the-art model optimization techniques such as quantization, pruning, distillation, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM or TensorRT to optimize inference speed on NVIDIA GPUs.
nx-set-shas
Publicchanged-files
Publiclws
Publicaction-slack
Publichoneymarker
Publicsetup-mpi
Publicllm-tools
Publicflashinfer
Public.github
Publicautoscaler
Publicaxolotl
PublicHackMIT-2024
PublicWorkshop-TRT-LLM
Publicgpu-operator
Publictriton-inference-server
Publictensorrtllm_backend
Publicpython_backend
Publiclangchain
Publicdiffusers
Publicchainlit-cookbook
Public