Fid Metric #2029
Fid Metric #2029
Conversation
|
Mypy error is cause scipy and torchvision do not exist in typeshed |
nvm.... found the fix. Updated mypy.ini file. |
ignite/metrics/gan/fid.py
Outdated
|
|
||
| class InceptionExtractor: | ||
| def __init__(self) -> None: | ||
| from torchvision import models |
There was a problem hiding this comment.
We should tnink better how to handle this dependency without making it as a hard requirement.
| ) -> float: | ||
|
|
||
| diff = mu1 - mu2 | ||
| import scipy |
There was a problem hiding this comment.
This is very unfriendly way to do things. This method is called after 1 epoch done and user get ImportError after some computational time. We should avoid this way of coding things!
There was a problem hiding this comment.
This just means that we have updated the mean and covariance based on all the training example given by user so far. Even if compute gives error as soon as user imports scipy they can rerun it and no computation done previously is lost.
There was a problem hiding this comment.
In the case where fid_score function is independently invoked, there is no extra computation so all cases are covered.
There was a problem hiding this comment.
Please use exception and a user friendly message whether scipy is missing.
ignite/metrics/gan/fid.py
Outdated
| from ignite.metric.GAN import FID | ||
| import torch | ||
|
|
||
| train, test = torch.rand(10,2048), torch.rand(10,2048) |
There was a problem hiding this comment.
Is it relevant as shape for the standard FID metric ?
Namings as train, test seem to be unconventional, we can put either put y_pred/y or fake/real
There was a problem hiding this comment.
FID is not just for image generation but also augmentation, style and feature transfer. In such cases real or fake is not appropriate. I will use y_pred/y instead.
ignite/metrics/gan/fid.py
Outdated
| self.model.eval() | ||
|
|
||
| def __call__(self, data: torch.Tensor) -> torch.Tensor: | ||
| if data.shape[1:] != (3, 299, 299): |
There was a problem hiding this comment.
If inception model is headless (no fully-connected layer), can't we use any size input. In the metric code we just have to ensure that we always have the same size...
There was a problem hiding this comment.
299x299 is minimum size of image needed and what is used for standard Inception model while calculating fid and IS. I can change it to >= 299 if needed.
There was a problem hiding this comment.
If inception model is headless (no fully-connected layer), can't we use any size input.
That's right !
In the metric code we just have to ensure that we always have the same size...
The user gives the feature size, so there is a test on shapes during iteration.
requirements-dev.txt
Outdated
| pynvml | ||
| clearml | ||
| scikit-image | ||
| py-rouge | ||
| nltk | ||
| pytorch_fid |
There was a problem hiding this comment.
You have it twice. See above.
|
Friendly advice:
For more info about running tests and running distributed tests: https://github.com/pytorch/ignite/blob/master/CONTRIBUTING.md#run-tests
Hope my advice helps you. |
Description:
First implementation and tests for the FID metric.
Check list: