8000 Adversarial loss · Issue #1 · erictzeng/adda · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Adversarial loss #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails 8000 .

Already on GitHub? Sign in to your account

Jump to bottom
Open
Yuliang-Zou opened this issue Apr 6, 2017 · 4 comments
Open

Adversarial loss #1

Yuliang-Zou opened this issue Apr 6, 2017 · 4 comments

Comments

@Yuliang-Zou
Copy link

Thanks for sharing the source code!

But I have some questions here:

  1. In adda/tools/train_adda.py line88-103: seems that you set source domain label as 1, and target domain label as 0? But here source distribution is fixed, while target distribution is updated. According to the original setting in the GAN paper, the changing one should have the zero label. And the equation (9) in the ADDA paper also shows that the ground truth label of source should be 1.

  2. How do you separate your encoder and classifier in the base model? In the paper, it seems that the encoder only contains CNN. But, in the code seems that you take the whole network as a encoder, then where is the classifier?

Thanks for your time. Looking forward to your response.
@BoChengTsai
Copy link

Hi, thanks for sharing the source code, too!

But, I faced the same problem as @Yuliang-Zou said above.
Therefore, I slightly modify the code to separate the encoder and classify by changing the feature representation from 'fc4' to 'fc3' which will feed into discriminator to do adversarial training in adda/tools/train_adda.py line96-97. And finally, it can work except for the experiment of SVHN->MNIST. Can someone share the parameter setting of this experiment if you succeed?

Thanks.

@aabbas90
Copy link

I am trying to implement the same (SVHN->MNIST) in PyTorch and I cannot get the target to converge, perhaps because of same issue as what @BoChengTsai and @Yuliang-Zou mentioned. Any suggestions?

@MInner
Copy link
MInner commented Jul 31, 2017

@Yuliang-Zou afaik, it does not make any difference; if you expand the loss expression, these two should be equivalent

it also took me some time to implement it in pytorch, here is a check list I wrote in the process, maybe it could be useful to some folks here

+ downscale everything to (28, 28)
+ SVHN reduce to 1 channel
+ same batch size 128
+ shuffle=True
+ learning rates for everything during adaptation (adam): 0.0002, 0.5, 0.999
+ lenet same shapes/kernels/strides, valid padding, 500 in hidden, relu
+ weight_decay on everything 2.5e-5
+ initialization with Xavier on everything
+ initialize target_embedding from source_embedding
+ adversarial classifier with shape [500, 500] + leaky relu with 0.1 gain
+ adversarial logits shape is (n, 2) # upd: binary loss also works
+ min cr_ent(y, logits) and min cr_ent(1-y, logits); y_i in {0, 1}

@sushant21
Copy link
sushant21 commented May 30, 2018 •
edited
Loading

@MInner could you share your pytorch code for SVHN->MNIST ? I tried with these hyperparameters but am unable to replicate the paper's accuracies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@MInner @Yuliang-Zou @sushant21 @aabbas90 @BoChengTsai
0