VisBERT: Hidden-State Visualizations for Transformers
This demo visualizes the findings from our paper How Does BERT Answer Questions? A Layer-Wise Analysis of Transformer Representations. The tool consists of a Python (>=3.6.1) Flask app and does currently rely on three fine-tuned Pytorch BERT models.
Install requirements with pip:
pip install -r docker/requirements.txt
Install NLTK Punkt:
python -m nltk.downloader punkt
Run app with model directory as argument. The model directory must currently contain the models: 'squad.bin', 'hotpot_distract.bin' and 'babi.bin', which are available here:
python src/app.py {model_directory}
When using our tool, please cite the following paper:
@article{van_Aken_2020,
title={VisBERT: Hidden-State Visualizations for Transformers},
author={van Aken, Betty and Winter, Benjamin and Löser, Alexander and Gers, Felix A.},
journal={WWW '20: Companion Proceedings of the Web Conference 2020},
publisher={ACM Press},
year={2020}
}