This repo provides the source code & data of our paper: Multi-level Mixture of Experts for Multimodal Entity Linking(KDD2025).
- conda create -n mmoe python=3.7 -y
- torch==1.11.0+cu113
- transformers==4.27.1
- torchmetrics==0.11.0
- tokenizers==0.12.1
- pytorch-lightning==1.7.7
- omegaconf==2.2.3
- pillow==9.3.0
- Download the datasets from MIMIC paper.
- Download the data with WikiData description information from here and move it to the corresponding MIMIC datasets folder.
- Create the root directory ./data and put the dataset in.
- Download the pretrained_weight from clip-vit-base-patch32.
- Create the root directory ./checkpoint and put the pretrained_weight in.
sh run.shNote: We provide commands for running three datasets in run.sh. You can switch commands by opening comments.
Note: We provide logs of our training in the logs directory.
If you find this code useful, please consider citing the following paper.
@article{
author={Zhiwei Hu and Víctor Gutiérrez-Basulto and Zhiliang Xiang and Ru Li and Jeff Z. Pan},
title={Multi-level Mixture of Experts for Multimodal Entity Linking},
publisher="ACM SIGKDD Conference on Knowledge Discovery and Data Mining",
year={2025}
}
We refer to codes of MIMIC and MEL-M3EL. Thanks for their contributions.