A small project to finetune a pretrained gpt2 on poem dataset.
We crawled the awesome Ganjoor website. Then we cleaned the text to include a standard Persian character set only. Then we added poet name as a token at the beginning of each poem. Visit data preparation notebook for more detail on data preprocessing and data module for dataset implementation.
We used the pretrained Persian GPT2 accessible from here. We used this model's tokenizer with added special tokens.
Visit model module for more details of implementation using transformers package.
We used pytorch lightning as backbone. Visit main notebook for more details on training and generation.
We used streamlit to create an app for demo. run streamlit run demo.py