Fine-Tuning with Unsloth does not create any Modelfile #2653

dave-espinosa · 2025-05-29T22:06:18Z

dave-espinosa
May 29, 2025

Hello everyone. I am following the Tutorial: How to Finetune Llama-3 and Use In Ollama, but when running the comand:

model.save_pretrained_gguf("gguf_model", tokenizer, quantization_method = ["q4_k_m"])

The Modelfile is never created:

Same thing happens when this command is run:

model.push_to_hub_gguf(
    "daveespinosa-qs/gguf_model", 
    tokenizer, 
    quantization_method = ["q4_k_m",], 
    token = "hf_myHFtoken", 
    private=True
)

Which gets only:

I have seen that the tutorial mentioned above includes this workaround already, so I am not sure what else it could be.

What could be going on here?

BTW, I am running that notebook from a GCP Vertex AI Jupyter Notebook.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Fine-Tuning with Unsloth does not create any Modelfile #2653

Uh oh!

{{title}}

Uh oh!

Replies: 0 comments

Select a reply

Uh oh!

Uh oh!

Fine-Tuning with Unsloth does not create any Modelfile #2653

Uh oh!

dave-espinosa May 29, 2025

Replies: 0 comments

dave-espinosa
May 29, 2025