Open
Description
Now, CasualLM in Keras_hub is different from ordinary LLMs, as it doesn't have a prefill step. This results in a lot of unnecessary waste for long prompt models.
So I think Keras_hub needs to consider introducing a prefill step for casualLM.
But this will be a major update, and we need to make different modifications to the current large number of models. Can the Keras team provide a relevant advancement plan? So that our community can submit contributions more conveniently.