Welcome to this curated repository showcasing the fine-tuning of various open-source large language models (LLMs) such as Gemma, LLaMA, Mistral, and others using Hugging Face Transformers, PEFT (LoRA/QLoRA), and other modern libraries.
This repo is designed for researchers, ML engineers, and enthusiasts looking to explore or build on top of custom fine-tuned LLMs.
- ✅ Fine-tuning with Hugging Face Trainer and PEFT (LoRA / QLoRA)
- ✅ Dataset loading and preprocessing
- ✅ Tokenization and model configuration
- ✅ Evaluation with custom metrics
- ✅ Easy-to-edit configs for reproducibility
- ✅ Support for mixed precision (fp16, bf16)
Got improvements, additional models, or tips? Contributions are welcome! Just open an issue or submit a pull request.