This repository contains the implementation and experimental code for the paper: Prompt-Agnostic Fine-Tuning (PAFT).
Our code is built upon the following frameworks:
- Llama-factory: https://github.com/hiyouga/LLaMA-Factory.git
- OpenCompass: https://github.com/open-compass/opencompass.git
While Large Language Models (LLMs) demonstrate strong adaptability to downstream tasks after fine-tuning, this adaptability often comes at the cost of prompt robustness. Even minor variations in prompts can lead to significant performance degradation. To address this issue, we propose Prompt-Agnostic Fine-Tuning (PAFT), a simple yet effective approach that dynamically adjusts prompts during the fine-tuning process. This encourages the model to learn the underlying task principles rather than overfitting to specific prompt formulations.
PAFT operates in two key stages:
- Prompt Construction: A diverse set of meaningful, synthetic candidate prompts is constructed.
- Dynamic Fine-Tuning: During fine-tuning, prompts are randomly sampled from this set to create dynamic training inputs.
Extensive experiments across diverse datasets and LLMs demonstrate that models trained with PAFT exhibit strong robustness and generalization across a wide range of prompts, including unseen ones. This enhanced robustness not only improves model performance but also enhances inference speed while maintaining training efficiency. Ablation studies further confirm the effectiveness of PAFT.
- data: Contains the training and testing data for PAFT.
- prompts: Includes the training and test prompts generated by PAFT.
- src: Modified data processing and training code to ensure the correct execution of PAFT.
src/llamafactory/data/loader.py: Data processing modifications.src/llamafactory/train/sft/trainer.py: Training code modifications.src/llamafactory/train/sft/workflow.py: Workflow modifications.
- run_prompt_eval_xx.py: Quick test scripts for different datasets, testing all test prompts.
mkdir paft
cd paft
conda create --name paft python=3.10 pytorch torchvision pytorch-cuda -c nvidia -c pytorch -y
conda activate paft
git clone --depth 1 https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory
pip install -e ".[torch,metrics]"
cd ..
git clone https://github.com/open-compass/opencompass opencompass
cd opencompass
pip install -e .
cd ../..
pip install -r requirements.txt# Replace the necessary files and directories
rm- rf paft/LLaMA-Factory/src
mv src paft/LLaMA-Factory/src
mv data paft/LLaMA-Factory/data
mv prompt.yaml paft/LLaMA-Factory/examples/train_lora/prompt.yaml
mv prompt.sh paft/LLaMA-Factory/prompt.sh
mv run_prompt_eval_hel.py paft/opencompass/run_prompt_eval_hel.py
mv run_prompt_eval_piqa.py paft/opencompass/run_prompt_eval_piqa.py
mv run_prompt_eval_race.py paft/opencompass/run_prompt_eval_race.py
mv run_prompt_eval_win.py paft/opencompass/run_prompt_eval_win.py# Navigate to the LLaMA-Factory directory and start training
cd paft/LLaMA-Factory
bash prompt.sh # Navigate to the OpenCompass directory and evaluate the model on different datasets
cd paft/opencompass
python run_prompt_eval_hel.py # evaluate Hellaswag dataset Note: you need to modify "/opencompass/configs/datasets/hellaswag/hellaswag_gen.py" to change evaluate prompt
python run_prompt_eval_piqa.py # evaluate PIQA dataset Note: you need to modify "/opencompass/configs/datasets/piqa/piqa_gen.py" to change evaluate prompt
python run_prompt_eval_race.py # evaluate RACE dataset Note: you need to modify "/opencompass/configs/datasets/race/race_gen.py" to change evaluate prompt
python run_prompt_eval_win.py # evaluate Winogrande dataset Note: you need to modify "/opencompass/configs/datasets/winogrande/winogrande_gen.py" to change evaluate promptIf this work is helpful, please kindly cite as:
@misc{wei2025paftpromptagnosticfinetuning,
title={PAFT: Prompt-Agnostic Fine-Tuning},
author={Chenxing Wei and Yao Shu and Mingwen Ou and Ying Tiffany He and Fei Richard Yu},
year={2025},
eprint={2502.12859},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2502.12859},
}This repo benefits from Llama-factory and OpenCompass. Thanks for their wonderful works.