This is the official code of the paper: "MaskSim: Detection of synthetic images by masked spectrum similarity analysis" Yanhao Li, Quentin Bammey, Marina Gardella, Tina Nikoukhah, Jean-Michel Morel, Miguel Colom, Rafael Grompone von Gioi.
This paper describes a semi-white-box method for detecting synthetic images by revealing anomalous patterns in the spectral domain. Specifically, we train a mask to enhance the most discriminative frequencies and simultaneously train a reference pattern that resembles the patterns produced by a given generative method. The proposed method produces comparable results to the state-of-the-art methods and highlights cues that can be used as forensic evidence. In contrast to most methods in the literature, the detections of the proposed method are explainable to a high degree.
Test passed on python 3.8. See requirements.txt for library details.
The pristine image datasets include Mit-5k, Dresden, COCO, HDR-Burst and Raise-1k. The synthetic image datasets include PolarDiffShield and Synthbuster. Detailed data scheme for training, validation and test is shown below:
| dataset | training | validation | test |
|---|---|---|---|
| Mit-5k | ✅ | ||
| Dreden | ✅ | ||
| COCO | ✅ | ✅ | |
| HDR-Burst | ✅ | ||
| Raise-1k | ✅ | ||
| PolarDiffShield | ✅ | ✅ | |
| Synthbuster | ✅ |
Use the following commands to prepare the data:
sh data_preparation.shThe structure of the processed_data/ folder should be like:
processed_data/
├──JPEG_Qrandom
│ ├──raise
│ └──synthbuster
│ ├──dalle2
│ ├──dalle3
│ ├──...
└──train
├──coco_train
├──coco_val
├──dresden
├──hdrburst
├──mit5k
└──newsynth
├──dalle2
├──dalle3
├──...
The pretrained model weights can be downloaded here. Then unzip the weight files in the checkpoints folder like below:
checkpoints/JPEG_Qrandom_w512/newsynth
├──dalle2-epoch=37-valid_auroc=1.00-valid_loss=0.065-low_loss.ckpt
├──dalle3-epoch=47-valid_auroc=1.00-valid_loss=0.036-low_loss.ckpt
├──...
python evaluate.py -Q random -w 512 --compression jpeg --img_q randomTo test the program on one single image:
python detect_one_image.py -i <img_path>python train.py -w 512 -b 8 -e 50 -p DnCNN -Q random --compression jpeg --progress@InProceedings{li2024masksim,
author = {Li, Yanhao and Bammey, Quentin and Gardella, Marina and Nikoukhah, Tina and Morel, Jean-Michel and Colom, Miguel and Von Gioi, Rafael Grompone},
title = {MaskSim: Detection of Synthetic Images by Masked Spectrum Similarity Analysis},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2024},
pages = {3855--3865}
}
Feel free to leave your comments at the Issues for any bugs found or any discussion 😇