This is the official code of the paper: "MaskSim: Detection of synthetic images by masked spectrum similarity analysis" Yanhao Li, Quentin Bammey, Marina Gardella, Tina Nikoukhah, Jean-Michel Morel, Miguel Colom, Rafael Grompone von Gioi.
Synthetic image generation methods have recently revolutionized the way in which visual content is created. This opens up creative opportunities but also presents challenges in preventing misinformation and crime. Anyone using these tools can create convincing photorealistic images. However, these methods leave traces in the Fourier spectrum that are invisible to humans, but can be detected by specialized tools. This paper describes a semi-white-box method for detecting synthetic images by revealing anomalous patterns in the spectral domain. Specifically, we train a mask to enhance the most discriminative frequencies and simultaneously train a reference pattern that resembles the patterns produced by a given generative method. The proposed method produces comparable results to the state-of-the-art methods and highlights cues that can be used as forensic evidence. In contrast to most methods in the literature, the detections of the proposed method are explainable to a high degree.
See requirements.txt
The pristine image datasets include Mit-5k, Dresden, COCO, HDR-Burst and Raise-1k. The synthetic image datasets include PolarDiffShield and Synthbuster. Detailed data scheme for training, validation and test is shown below:
| dataset | training | validation | test |
|---|---|---|---|
| Mit-5k | ✅ | ||
| Dreden | ✅ | ||
| COCO | ✅ | ✅ | |
| HDR-Burst | ✅ | ||
| Raise-1k | ✅ | ||
| PolarDiffShield | ✅ | ✅ | |
| Synthbuster | ✅ |
Use the following commands to prepare the data:
sh data_preparation.shThe structure of the processed_data/ folder should be like:
processed_data/
├──JPEG_Qrandom
│ ├──raise
│ └──synthbuster
│ ├──dalle2
│ ├──dalle3
│ ├──...
└──train
├──coco_train
├──coco_val
├──dresden
├──hdrburst
├──mit5k
└──newsynth
├──dalle2
├──dalle3
├──...
The pretrained model weights can be downloaded here. Then unzip the weight files in the checkpoints folder like below:
checkpoints/JPEG_Qrandom_w512/newsynth
├──dalle2-epoch=37-valid_auroc=1.00-valid_loss=0.065-low_loss.ckpt
├──dalle3-epoch=47-valid_auroc=1.00-valid_loss=0.036-low_loss.ckpt
├──...
python evaluate.py -Q random -w 512 --compression jpeg --img_q randomTo test the program on one single image:
python detect_one_image.py -i <img_path>python train.py -w 512 -b 8 -e 50 -p DnCNN -Q random --compression jpeg --progressprovide script for downloading datarelease preprocessing coderelease evaluation code of exprelease code for single imagerelease training code of exp
@InProceedings{li2024masksim,
author = {Li, Yanhao and Bammey, Quentin and Gardella, Marina and Nikoukhah, Tina and Morel, Jean-Michel and Colom, Miguel and Von Gioi, Rafael Grompone},
title = {MaskSim: Detection of Synthetic Images by Masked Spectrum Similarity Analysis},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2024},
pages = {3855--3865}
}
Feel free to leave your comments at the Issues for any bugs found or any discussion 😇