Welcome to the LLM repository, where we develop, train and fine tune large language models to push the boundaries of natural language processing and understanding.
Searching the web for resources on how to develop your own LLM's from scratch and then fine tune in an iterative process is overly complex I wanted to single place where I and others can learn and share is valuable knowledge, This repository contains the implementation and training code for a large language model, including model architectures, training scripts, and evaluation metrics. This repo is designed to assist in the learning of how Large Language models are trained on large amounts of text data and generate human-like language outputs. This is an ongoing effort as the AI world changes so must this resource
- Python 3.10
- PyTorch 1.9+ (see local readme for some restrictions)
- NVIDIA GPU (recommended)
- conda
- Clone the repository:
git clone https://github.com//llm.git - Change director to a project cd base/gpt2
- Follow Local Readme for a given project example
base/: Base LLM modes from scratchdata/: Datasets and data preprocessing scripts, limited bring your own datanotebooks/: Training, evaluation, and inference scripts
This repository is licensed under the MIT License.
Contributions are welcome! If you'd like to contribute to the development of the LLM, please fork the repository and submit a pull request.
Thanks to the following contributors, libraries, and resources that have made this project possible:
- check-ai.com
If you have any questions or need help with the repository, feel free to reach out to us at jeff@check-ai.com.