微调的各种问题 #40

python finetune.py data/AdvertiseGen/ /mnt/workspace/panyun/models/THUDM/glm-4-9b-chat configs/lora.yaml
Traceback (most recent call last):
File "/mnt/workspace/panyun/GLM-4/finetune_demo/finetune.py", line 15, in
from peft import PeftConfig, get_peft_config, get_peft_model
File "/opt/conda/lib/python3.10/site-packages/peft/init.py", line 22, in
from .auto import (
File "/opt/conda/lib/python3.10/site-packages/peft/auto.py", line 32, in
from .mapping import MODEL_TYPE_TO_PEFT_MODEL_MAPPING
File "/opt/conda/lib/python3.10/site-packages/peft/mapping.py", line 22, in
from .mixed_model import PeftMixedModel
File "/opt/conda/lib/python3.10/site-packages/peft/mixed_model.py", line 26, in
from peft.tuners.mixed import COMPATIBLE_TUNER_TYPES
File "/opt/conda/lib/python3.10/site-packages/peft/tuners/init.py", line 21, in
from .lora import LoraConfig, LoraModel, LoftQConfig
File "/opt/conda/lib/python3.10/site-packages/peft/tuners/lora/init.py", line 20, in
from .model import LoraModel
File "/opt/conda/lib/python3.10/site-packages/peft/tuners/lora/model.py", line 50, in
from .awq import dispatch_awq
File "/opt/conda/lib/python3.10/site-packages/peft/tuners/lora/awq.py", line 26, in
from awq.modules.linear import WQLinear_GEMM
File "/opt/conda/lib/python3.10/site-packages/awq/init.py", line 2, in
from awq.models.auto import AutoAWQForCausalLM
File "/opt/conda/lib/python3.10/site-packages/awq/models/init.py", line 1, in
from .mpt import MptAWQForCausalLM
File "/opt/conda/lib/python3.10/site-packages/awq/models/mpt.py", line 1, in
from .base import BaseAWQForCausalLM
File "/opt/conda/lib/python3.10/site-packages/awq/models/base.py", line 46, in
from awq.quantize.quantizer import AwqQuantizer
File "/opt/conda/lib/python3.10/site-packages/awq/quantize/quantizer.py", line 10, in
from awq.quantize.scale import apply_scale, apply_clip
File "/opt/conda/lib/python3.10/site-packages/awq/quantize/scale.py", line 8, in
from transformers.models.llama.modeling_llama import LlamaRMSNorm
File "/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 55, in
from flash_attn import flash_attn_func, flash_attn_varlen_func
File "/opt/conda/lib/python3.10/site-packages/flash_attn/init.py", line 3, in
from flash_attn.flash_attn_interface import (
File "/opt/conda/lib/python3.10/site-packages/flash_attn/flash_attn_interface.py", line 10, in
import flash_attn_2_cuda as flash_attn_cuda
ImportError: /opt/conda/lib/python3.10/site-packages/flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN2at4_ops5zeros4callEN3c108ArrayRefINS2_6SymIntEEENS2_8optionalINS2_10ScalarTypeEEENS6_INS2_6LayoutEEENS6_INS2_6DeviceEEENS6_IbEE

0 replies

pangaass · 2024-07-12T09:12:00Z

pangaass
Jul 12, 2024

8*80GA00,8k文本长度lora能调动吗？ds zero2或者zero3

0 replies

ChIIChI1230 · 2024-07-19T07:03:14Z

ChIIChI1230
Jul 19, 2024

微调后，评估时报错

2 replies

tzm529 Jul 19, 2024

哥们，我也遇到这个问题了，把transformers降到4.40.0，这个问题没有了，但是有了个新问题：

TypeError: Seq2SeqTrainingArguments.init() got an unexpected keyword argument 'eval_strategy'

zxy1119 Jul 31, 2024

我也遇到这个问题了，将 lora.yaml 里 eval_strategy 修改为 evaluation_strategy 就可以了

SongHY-13 · 2024-07-23T09:30:36Z

SongHY-13
Jul 23, 2024

想问一下GLM-4-9B模型是否支持在4张T4卡共64G显存配置的服务器上实现微调，感谢，因为包括我使用多模态模型还有微调时都会出现运行错误
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:2 and cuda:3!

1 reply

SongHY-13 Jul 23, 2024

多模态已经根据issues：https://github.com/THUDM/GLM-4/issues/350改好了，哈哈，主要还是想要用来微调

McRays · 2024-08-02T02:52:33Z

McRays
Aug 2, 2024

我在微调glm4-9b-chat的时候遇见这个报错，还没有找到什么有效的解决方法

2 replies

McRays Aug 2, 2024

我查到tokenizer_config.json中151337映射到的是assistant，我可以理解这个报错，是在训练文本中没有这个assistant角色吗

zhoulq-789 Feb 28, 2025

请问这个问题怎么排查解决的，我也遇到了这个错

Hoonly · 2024-08-09T07:18:18Z

Hoonly
Aug 9, 2024

finetune_vision.py中的process_batch_eval这一段目的是什么？

0 replies

kxy-cheng · 2024-08-12T15:08:34Z

kxy-cheng
Aug 12, 2024

我使用仓库中默认的lora设置，在10w条数据上使用lora微调，loss在下降到1.1附近后，模型不再收敛了，要如何解决呢？

0 replies

KarRoy715 · 2024-08-15T08:14:06Z

KarRoy715
Aug 15, 2024

训练集都是正常的，但是跑到验证集的时候就报错了（换过好几个版本的transformers）都会出错，这个怎么解决呀

1 reply

cssaudrey Nov 4, 2024

1 transformer 4.44.0
2. Python 3.12 需要更新ntlk git的版本，修复方法：https://github.com/THUDM/GLM-4/issues/38，命令如下
pip uninstall nltk
pip install git+https://github.com/nltk/nltk.git@86fa083

Yang-125 · 2024-08-16T09:50:42Z

Yang-125
Aug 16, 2024

请问微调之后模型输出停不下来，在后半段会一直重复某段话反复输出，有什么问题影响到吗？

0 replies

61zzzzz · 2024-08-19T07:17:06Z

61zzzzz
Aug 19, 2024

请问单卡单机运行GLM4v的lora时，如何让gpu为指定的cuda而不是cuda:0呢

0 replies

linwhitehat · 2024-08-19T08:40:57Z

linwhitehat
Aug 19, 2024

想询问下GLM4和GLM2、3版本的性能差异来源。
我在A800-80G单卡下对GLM4进行lora微调，微调的速度比同等资源下GLM2要快15倍左右，这是什么原因？没在官方文档看到提升的内容说明？能否帮忙答疑一下，谢谢

0 replies

delcok · 2024-08-30T07:37:22Z

delcok
Aug 30, 2024

进行微调时使用lora.yaml可以进行从从保存点进行微调，但是使用ptuning_v2.yaml会出现如下图问题。代码使用的demo代码

2 replies

delcok Sep 1, 2024

解决了！更新一下模型配置文件就好了！

cssaudrey Nov 4, 2024

请问具体怎么解决的？

huohuolimo · 2024-09-19T09:40:18Z

huohuolimo
Sep 19, 2024

TypeError: FinetuningConfig.init() got an unexpected keyword argument
'freezeV'

0 replies

WangZhuoxian · 2024-10-29T12:54:58Z

WangZhuoxian
Oct 29, 2024

各种微调的问题，请在此提问
请问微调脚本里的combine参数，作用是什么？看代码是token的处理有所不同，但不理解目的是什么

0 replies

cssaudrey · 2024-11-09T14:57:05Z

cssaudrey
Nov 9, 2024

微调过后，训练样本的问题回答的很好，但是通用能力回答的问题都很简短。这个如何提高？

0 replies

Miklight · 2024-12-03T07:56:52Z

Miklight
Dec 3, 2024

请教下，使用lora微调，第一轮输出的时候会报错，应该如何解决？使用ptuning-v2就能够正常输出。

1 reply

DanLiu0623 Dec 18, 2024

我也是这个问题，验证集上模型输出的pred_txt为空就会报这个错，但即使不是空，pred的结果也是一堆英文，要么乱码

wild0506 · 2024-12-09T08:16:54Z

wild0506
Dec 9, 2024

模型推理正常，LORA微调训练阶段正常，到评估时候报错KeyError: 'loss'，详情如下，问题发生时间2024年12月9日，transformer版本4.47.0
***** Running Evaluation *****
Num examples = 1
Batch size = 2
Trainer.tokenizer is now deprecated. You should use Trainer.processing_class instead.
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /workspace/GLM-4-main/finetune_demo/finetune.py:472 in main │
│ │
│ 469 │ ) │
│ 470 │ │
│ 471 │ if auto_resume_from_checkpoint.upper() == "" or auto_resume_from_checkpoint is None: │
│ ❱ 472 │ │ trainer.train() │
│ 473 │ else: │
│ 474 │ │ output_dir = ft_config.training_args.output_dir │
│ 475 │ │ dirlist = os.listdir(output_dir) │
│ │
│ /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:2164 in train │
│ │
│ 2161 │ │ │ finally: │
│ 2162 │ │ │ │ hf_hub_utils.enable_progress_bars() │
│ 2163 │ │ else: │
│ ❱ 2164 │ │ │ return inner_training_loop( │
│ 2165 │ │ │ │ args=args, │
│ 2166 │ │ │ │ resume_from_checkpoint=resume_from_checkpoint, │
│ 2167 │ │ │ │ trial=trial, │
│ │
│ /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:2589 in _inner_training_loop │
│ │
│ 2586 │ │ │ │ │ │ self.state.global_step += 1 │
│ 2587 │ │ │ │ │ │ self.state.epoch = epoch + (step + 1 + steps_skipped) / steps_in │
│ 2588 │ │ │ │ │ │ self.control = self.callback_handler.on_step_end(args, self.stat │
│ ❱ 2589 │ │ │ │ │ │ self._maybe_log_save_evaluate( │
│ 2590 │ │ │ │ │ │ │ tr_loss, grad_norm, model, trial, epoch, ignore_keys_for_eva │
│ 2591 │ │ │ │ │ │ ) │
│ 2592 │ │ │ │ │ else: B41A │
│ │
│ /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:3047 in _maybe_log_save_evaluate │
│ │
│ 3044 │ │ │
│ 3045 │ │ metrics = None │
│ 3046 │ │ if self.control.should_evaluate: │
│ ❱ 3047 │ │ │ metrics = self._evaluate(trial, ignore_keys_for_eval) │
│ 3048 │ │ │ is_new_best_metric = self._determine_best_metric(metrics=metrics, trial=tria │
│ 3049 │ │ │ │
│ 3050 │ │ │ if self.args.save_strategy == SaveStrategy.BEST: │
│ │
│ /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:3001 in _evaluate │
│ │
│ 2998 │ │ │ ) │
│ 2999 │ │
│ 3000 │ def _evaluate(self, trial, ignore_keys_for_eval, skip_scheduler=False): │
│ ❱ 3001 │ │ metrics = self.evaluate(ignore_keys=ignore_keys_for_eval) │
│ 3002 │ │ self._report_to_hp_search(trial, self.state.global_step, metrics) │
│ 3003 │ │ │
│ 3004 │ │ # Run delayed LR scheduler now that metrics are populated │
│ │
│ /opt/conda/lib/python3.10/site-packages/transformers/trainer_seq2seq.py:195 in evaluate │
│ │
│ 192 │ │ # We don't want to drop samples in general │
│ 193 │ │ self.gather_function = self.accelerator.gather │
│ 194 │ │ self._gen_kwargs = gen_kwargs │
│ ❱ 195 │ │ return super().evaluate(eval_dataset, ignore_keys=ignore_keys, metric_key_prefix │
│ 196 │ │
│ 197 │ def predict( │
│ 198 │ │ self, │
│ │
│ /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:4051 in evaluate │
│ │
│ 4048 │ │ start_time = time.time() │
│ 4049 │ │ │
│ 4050 │ │ eval_loop = self.prediction_loop if self.args.use_legacy_prediction_loop else se │
│ ❱ 4051 │ │ output = eval_loop( │
│ 4052 │ │ │ eval_dataloader, │
│ 4053 │ │ │ description="Evaluation", │
│ 4054 │ │ │ # No point gathering the predictions if there are no metrics, otherwise we d │
│ │
│ /opt/conda/lib/python3.10/site-packages/transformers/trainer.py:4245 in evaluation_loop │
│ │
│ 4242 │ │ │ │ │ batch_size = observed_batch_size │
│ 4243 │ │ │ │
│ 4244 │ │ │ # Prediction step │
│ ❱ 4245 │ │ │ losses, logits, labels = self.prediction_step(model, inputs, prediction_loss │
│ 4246 │ │ │ main_input_name = getattr(self.model, "main_input_name", "input_ids") │
│ 4247 │ │ │ inputs_decode = ( │
│ 4248 │ │ │ │ self._prepare_input(inputs[main_input_name]) if "inputs" in args.include │
│ │
│ /workspace/GLM-4-main/finetune_demo/finetune.py:90 in prediction_step │
│ │
│ 87 │ │ │ │ output_ids = inputs.pop('output_ids') │
│ 88 │ │ │ input_ids = inputs['input_ids'] │
│ 89 │ │ │ │
│ ❱ 90 │ │ │ loss, generated_tokens, labels = super().prediction_step( │
│ 91 │ │ │ │ model, inputs, prediction_loss_only, ignore_keys, **gen_kwargs │
│ 92 │ │ │ ) │
│ 93 │
│ │
│ /opt/conda/lib/python3.10/site-packages/transformers/trainer_seq2seq.py:354 in prediction_step │
│ │
│ 351 │ │ │ │ if self.label_smoother is not None: │
│ 352 │ │ │ │ │ loss = self.label_smoother(outputs, inputs["labels"]).mean().detach( │
│ 353 │ │ │ │ else: │
│ ❱ 354 │ │ │ │ │ loss = (outputs["loss"] if isinstance(outputs, dict) else outputs[0] │
│ 355 │ │ │ else: │
│ 356 │ │ │ │ loss = None │
│ 357 │
│ │
│ /opt/conda/lib/python3.10/site-packages/transformers/utils/generic.py:431 in getitem │
│ │
│ 428 │ def getitem(self, k): │
│ 429 │ │ if isinstance(k, str): │
│ 430 │ │ │ inner_dict = dict(self.items()) │
│ ❱ 431 │ │ │ return inner_dict[k] │
│ 432 │ │ else: │
│ 433 │ │ │ return self.to_tuple()[k] │
│ 434 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
KeyError: 'loss'

5 replies

1190201205 Dec 19, 2024

我和你遇到的问题一模一样也是4.47版本你有找到怎么解决了不只是评估出错如果你设置test集最后也会报这个错

wild0506 Dec 19, 2024

我和你遇到的问题一模一样也是4.47版本你有找到怎么解决了不只是评估出错如果你设置test集最后也会报这个错

降级到4.46.2版本无此报错了

1190201205 Dec 19, 2024

我和你遇到的问题一模一样也是4.47版本你有找到怎么解决了不只是评估出错如果你设置test集最后也会报这个错

降级到4.46.2版本无此报错了

我是降到4.44也没问题就是每次评估完验证集等的时间很久

Superminih Dec 19, 2024

我用的是4.44，你们在微调评估验证集的时候会不会遇到这个报错：
│ /home/zhengqing/anaconda3/envs/finetune/lib/python3.10/site-packages/rouge_c │
│ hinese/rouge_score.py:253 in rouge_n │
│ │
│ 250 │ ValueError: raises exception if a param has len <= 0 │
│ 251 │ """ │
│ 252 │ if len(evaluated_sentences) <= 0: │
│ ❱ 253 │ │ raise ValueError("Hypothesis is empty.") │
│ 254 │ if len(reference_sentences) <= 0: │
│ 255 │ │ raise ValueError("Reference is empty.") │
│ 256 │
╰──────────────────────────────────────────────────────────────────────────────╯
ValueError: Hypothesis is empty.

Acrzem Jan 8, 2025

我和你遇到的问题一模一样也是4.47版本你有找到怎么解决了不只是评估出错如果你设置test集最后也会报这个错

降级到4.46.2版本无此报错了

我是降到4.44也没问题就是每次评估完验证集等的时间很久

我降到4.46.2版本也解决了报错的问题，也卡在评估完验证集的等待时间，请问你解决了吗

zRzRzRzRzRzRzR · 2025-05-10T02:44:46Z

zRzRzRzRzRzRzR
May 10, 2025
Maintainer Author

我们已经更新了微调代码，并支持使用Llama Factory进行微调，请使用最新版本transformers

0 replies

微调的各种问题 #40

Uh oh!

zRzRzRzRzRzRzR Jun 5, 2024 Maintainer

Replies: 22 comments · 16 replies

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zRzRzRzRzRzRzR
Jun 5, 2024
Maintainer

Replies: 22 comments 16 replies