question about replace stygan2 decoder in other gan network

[NingNanXin]

Hi, thanks for your amazing work. I try to use mobileStyleGAN to replace the stylegan2 decoder in GFPGAN. To solve this:

1. The mapping network in GFPGAN is a u-net and output a feature with shape [batch, 16, 512]. So I replace the freq with style[1] which was style[2], so the freq share the style with hidden.
2. GFPGAN output an image with shape [batch, 3, 512, 512]. In mobileStyleGAN, I change the channels[1:] to channels[2:]
3. I fix the noise dataset to generate [3, 512, 512] image with torch.randn()

Then I successfully trained the network with batchsize=4, dataset length=10000. But after 1 epoch the kid_val is 0.0007 and never changed, the train loss oscillating around 0.5. I trained 20 epochs and use the 20st checkpoint to predict but get this result:

Another problem, I find the mapping network param is updating while training, but this network should freeze the params? Right?
Can you share your training loss? Or some training tips? Thanks for your help.