Add support for Quantization-Aware Low-Rank Adaptation (QALoRA) by gapsong · Pull Request #2571 · huggingface/peft

Commits on Jun 17, 2025

Commits on Jun 21, 2025

Update README and config for QALoRA clarity; remove redundant line in tests
gapsong
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add support for Quantization-Aware Low-Rank Adaptation (QALoRA)#2571

Add support for Quantization-Aware Low-Rank Adaptation (QALoRA)#2571
BenjaminBossan merged 17 commits intohuggingface:mainfrom
gapsong:qa_lora

Commits on Jun 4, 2025

Add support for Quantization-Aware Low-Rank Adaptation (QALoRA) in LoraLayer and GPTQLoraLinear

Commits on Jun 9, 2025

Refactor QALoraLinearVariant methods to raise NotImplementedError for unsupported operations.

Fix dtype handling and use foward in GPTQLoraLinear forward method

Commits on Jun 11, 2025

Enhance QALoRA support by refining group size handling and error management in variants

Commits on Jun 14, 2025

Enhance GPTQ and QALoRA implementations with improved error handling and refactor training tests

Commits on Jun 16, 2025

Refactor QALoRA implementation by removing unused memory printing function, enhancing gradient checks in tests, and improving adapter handling in model training.

Commits on Jun 17, 2025

Refactor test for QALoRA with GPTQ quantization by improving indentation and structure for better readability and maintainability.

Changed structure

Commits on Jun 21, 2025

Update README and config for QALoRA clarity; remove redundant line in tests

Commits on Jun 22, 2025

Refactor README and training script for QALoRA: improve readability by breaking long lines, update dataset loading to use provided data path, and remove unused arguments.

Refactor README for QALoRA: remove outdated command examples and unnecessary parameters for clarity and conciseness.

Refactor train_model function in QALoRA script: add data_split parameter, update dataset loading, and remove unused parameters for improved clarity and functionality.

Commits on Jun 23, 2025

Update QALoRA examples and validation: change default group size to 32 in README and add validation for divisibility by group size in QALoraLinearVariant.

Refactor QALoraLinearVariant error messages for clarity and consistency; streamline argument handling in training script.

Commits on Jun 24, 2025

Refactor forward method in QALoraLinearVariant: streamline weight access, improve pooling logic, and enhance LoRA computation for clarity and efficiency.

Remove redundant QALoRA gradient tests from RequiresGradTester for improved clarity and maintainability.

Commits on Jun 26, 2025

Moved QALoraLinearVariant to the bottom of variants.py

Add support for Quantization-Aware Low-Rank Adaptation (QALoRA)#2571BenjaminBossan merged 17 commits intohuggingface:mainhuggingface/peft:mainfrom gapsong:qa_loragapsong/peft:qa_loraCopy head branch name to clipboard

Commits

Commits on Jun 4, 2025

Commits on Jun 9, 2025

Commits on Jun 11, 2025

Commits on Jun 14, 2025

Commits on Jun 16, 2025

Commits on Jun 17, 2025

Commits on Jun 21, 2025

Commits on Jun 22, 2025

Commits on Jun 23, 2025

Commits on Jun 24, 2025

Commits on Jun 26, 2025

Add support for Quantization-Aware Low-Rank Adaptation (QALoRA)#2571
BenjaminBossan merged 17 commits intohuggingface:mainfrom
gapsong:qa_lora