Has anyone tried training a model with two GANs one after another. I realize that it'll be computationally heavy but I wondering if it gives good results. Do let me know. TIA.

Comments

You must log in or register to comment.

TheRealSerdra t1_itfiwui wrote on October 23, 2022 at 7:15 AM

What exactly do you want to do that requires two GANs? And are you planning on just chaining the generators?

manli29 OP t1_itflbff wrote on October 23, 2022 at 7:48 AM

One GAN for colorization and one GAN restoration. I want see if that works better than a single GAN that does both.

kaarrrlll t1_itfyuqj wrote on October 23, 2022 at 10:53 AM

Having two gans is not a problem. Although for different purpose it exists since a long time (cyclegan). What's important is that your loss and/or other constraints must be precise to make sure to avoid one gan learns both and other learns identity. It's also double the concerns for instability during training. Good luck!

Yeinstein20 t1_itfpan2 wrote on October 23, 2022 at 8:43 AM

I feel like I've read a paper where they do something similar to this but I'm not completely sure. I'll try finding it.

Edit: maybe remind me of that in case I forget about it

manli29 OP t1_itfqo7q wrote on October 23, 2022 at 9:02 AM

Awesome thanks a lot

beingsubmitted t1_itht9o5 wrote on October 23, 2022 at 7:22 PM

I don't see how you would train them that way - you can't use the output of a discriminator as the input of a generator - that wouldn't get you what you want. You could train them in parallel, one network and discriminator doing only b&w restoration, and the other doing only colorization.

The way images work and the eye (part of the science behind why jpeg is so useful) is that we're much more sensitive to luminance information than color information. You could take the output of colorized image in hsl color space and replace the luminance with that of the generated restored photo. Doing it this way, you could force the separation of two generators using only one discriminator, as well - one generator only affecting the hue and saturation of the final image, and the other only affecting the luminance.

That said, with the more recent breakthroughs, it seems that networks are proving more successful as generalists than specialists. For example, it's believed that whisper performs better on each language because it's trained on all languages, as counter-intuitive as it may seem.