It's not controversial as long as you don't share it and make money with it, you are pretty much free to do whatever you want.

If you plan to share the output, meaning here what's generated, not just the code and checkpoints, or a training set that's under copyright, publicly then it's another question entirely and if you are serious about that I recommend seeking legal advice.