This GitHub repository contains the training code for data augmentation using blurry images, as detailed in the research paper "Improved modeling of human vision by incorporating robustness to blur in convolutional neural networks" authored by Hojin Jang and Frank Tong. For any inquiries, you can reach out to me at jangh@mit.edu.
Article
·
Data Repository
This study hypothesized that the absence of blurred images in the training sets of neural networks might lead them to overly depend on high spatial frequency information for object recognition, resulting in divergences from human visual processing. This hypothesis was systematically assessed by comparing different training regimes involving both clear and blurred images (i.e., standard training, weak blur training, and strong blur training). The results demonstrated that networks trained with blurred images outperformed standard networks in predicting neural responses to objects under diverse viewing conditions. Additionally, these blur-trained networks developed an increased sensitivity to object shapes and increased robustness to various types of visual corruptions, aligning more closely with human perceptual processes. Our research underscores the importance of incorporating blur as a vital component in the training process for neural networks to develop representations of the visual world that are more congruent with human perception. Based on our results, we recommend the integration of blur as a standard image augmentation technique in the majority of computer vision tasks.
Below is the main code for blur training employed in this study. It contains a custom function for adjusting the sampling weights corresponding to a range of sigma values. Additionally, we integrated the Kornia library, speeding the process of Gaussian blurring via Tensor operations. Users can also refer to the latest PyTorch version, which includes the torchvision.transforms.GaussianBlur() function. This function allows for the blurring of images with a sigma value that is randomly determined. Tensorflow also offers the function tfa.image.gaussian_filter2d() to achieve similar outcomes.
def add_blur_with(images, sigmas, weights):
blurred_images = torch.zeros_like(images)
normalize = transforms.Normalize(mean=[0.449], std=[0.226]) # grayscale
for i in range(images.size(0)): # Batch size
image = images[i, :, :, :]
weights = numpy.asarray(weights).astype('float64')
weights = weights / numpy.sum(weights)
sigma = choice(sigmas, 1, p=weights)[0]
kernel_size = 2 * math.ceil(2.0 * sigma) + 1
if sigma == 0:
blurred_image = image
else:
blurred_image = kornia.gaussian_blur2d(torch.unsqueeze(image, dim=0), kernel_size=(kernel_size, kernel_size), sigma=(sigma, sigma))[0, :, :, :]
blurred_image = normalize(blurred_image)
blurred_images[i] = blurred_image
blurred_images = blurred_images.repeat(1, 3, 1, 1) # Grayscale to RGB
return blurred_images
This research was supported by the following grants from the National Eye Institute, National Institutes of Health (NEI/NIH): R01EY035157 and R01EY029278 to Frank Tong, and P30EY008126 to the Vanderbilt Vision Research Center.