BadTransformer is a simplified and easy-to-understand transformer proposed by Shizhuo Zhang. It simplifies the encoder and decoder structures and changes the multi-head attention mechanism to a single-head attention mechanism, while still potentially achieving the same effects as the original transformer.
- Language Model miniGPT
...