8000 llama: introduce good TinyLlama-15M model by renerocksai · Pull Request #174 · zml/zml · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

llama: introduce good TinyLlama-15M model #174

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

llama: introduce good TinyLlama-15M model #174

wants to merge 8 commits into from

Conversation

renerocksai
Copy link
Contributor
@renerocksai renerocksai commented Jan 28, 2025 •
edited
Loading

This change re-introduces TinyLlama-15M, but with HF tokenizer and safetensors weights.

@renerocksai renerocksai requested a review from Corendos January 28, 2025 16:44
@gwenzek
Copy link
Contributor
gwenzek commented Feb 3, 2025

when I tried it wasn't ouputing correctly either.
can you double check ?

@renerocksai
Copy link
Contributor Author

I had replied to you on slack what's going on. I'll repeat here:

If you run it, it works:

git clone git@github.com:zml/zml.git
cd zml/examples
bazel run //llama:TinyLlama-Stories-15M

...

info(llama): ✅ Prompt: Once upon a time,
there was a little girl named Lily. Lily was very tired and wanted to take a nap.

Lily's mommy said, "Lily, it's time to take a nap. I will give you a toy to play with and to be a good friend."

Lily was so happy and said, "I'm a good bunny. I'm so happy to have a toy to play with. I'm so happy and play with the toy. I'm so happy and said, "Thank you, mommy. I'm so happy to have a toy to play with. I'm so happy and said, "Thank you, mommy. I'm so tough to play with the toy, and her toy was the toy was a toy. 3-years, and her mommy was very tired, and she was a little, and she was a little girl was very tired and the to take a nap, and to take a nap, and to take a toy to the toy toy to play with. But the toy and to be a little, and so,
info(llama): ✅ Generated 249 tokens in 2.742e-1s: 907.987tok/s

Note that the prompt used above is Once upon a time,, the default provided in BUILD.bazel.

The prompt you used, the one that was also in the README, was Once upon a time, there was a cute little dragon. For some reason, that's not a good prompt for that model:

info(llama): ✅ Prompt: Once upon a time, there was a cute little dragon
. The c c c c, and the c to make the c c to make the c c c to make the c c c c to the c to make the c c c c to c to c to c to c to c to c to make a c c c c
info(llama): ✅ Generated 241 tokens in 2.505e-1s: 961.941tok/s

You can fix it by appending a comma at the end, like so: Once upon a time, there was a cute little dragon,:

info(llama): ✅ Prompt: Once upon a time, there was a cute little dragon,
who was very scary.

The little dragon was very scared and wanted to find a place to hide. He asked the big, shy, to help.

"I'm sc, the big, big, and the big, shy, the big, shy, and the big, shy,," the little dragon said.

"Don't be sc, the big, shy, and the big toy. It's a very nice and nice place to be. I'm the only to be the only one to be the only one.

"Thank you, little one. I'm so happy to be the only one. I'm the only one, and the toy was very happy.

"I'll be the only toy, and the toy, and the only the only the only the toy, and the only the to be the only the only the to be the toy, shy, and the toy, and the toy, and the toy, and the toy, and the toy, the toy, the toy.
info(llama): ✅ Generated 240 tokens in 2.602e-1s: 922.261tok/s

All in all, the model does not seem to be the best Stories-TinyLlama, judged by how quickly it gets into a repetitive state.

I also tested with a different prompt that does not end in a comma: Once upon a time, there was a little girl named Lily. She went into the woods:

info(llama): ✅ Prompt: Once upon a time, there was a little girl named Lily. She went into the woods
and saw a big, scary bear.

"Hello, little girl. What are you doing in the woods?" asked the bear.

"I'm just looking for a place to play. I'm very scared," said Lily.

"Don't be scared, Lily. I'll help you. I'll be a good friend and we can play together. I'll be a good friend and we can have a big, fun and fun.

"I'm a very brave and strong and I'm the best to be the best and the big, scary, and the big, scary, and the big, scary, and the big, scary, and the big, and the big, and the big, and the big, and the big and the big and the big, and scary, and scary, and the big, Lily, the big to do. He was so scary, and he was very scared to be there, and he'll. He was a little. "I'm a little a little one day
info(llama): ✅ Generated 235 tokens in 2.561e-1s: 917.764tok/s

@renerocksai
Copy link
Contributor Author

I updated the documentation so that it does not suggest that very prompt that confuses the model. Also changed the name back to TinyLlama-Stories-15M< 8000 /code>.

@gwenzek
Copy link
Contributor
gwenzek commented Feb 10, 2025 •
edited
Loading

I think your model is still corrupted. This is what I get with official weights from Karpathy (current zml/master):

bazel run --config=debug //llama:TinyLlama-Stories-15M -- --prompt='Once upon a time, there was a cute little dragon'

. He was very adorable and loved to play. One day, he was playing in the forest when he saw a big tree. He wanted to climb it, but he was scared.
The dragon's friend, a small bird, saw him and said, "Don't worry, I will help you." The bird flew up and helped the dragon climb the tree. They were very happy and played together all day.
But then, something unexpected happened. The dragon started to change. He became a big, friendly dragon! The small bird was surprised, but they still played together. They had lots of fun and became best friends.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Corendos Corendos Awaiting requested review from Corendos

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@renerocksai @gwenzek
0