Nice. Thanks for taking the time to implement and to share. I get the feeling that this approach will be way more coherent for certain types of content and layout (mayhaps when prompting something like “a tiger and a bunny sitting side by side” and the “traditional” (is it old enough to have traditions yet?) approach will be better for other types, perhaps such as “a tiger and a bunny”, where the intent is what that approach often results in, some interpolation between the two objects; a chimera or mutant.

Or maybe another way to explain is that this MagixMix is akin to orchestration of multiple parts, and the present approach is more akin to a mashup of multiple parts; each has its own strengths and appropriate applications.

Thanks! I hope this helps!

Thanks for this implem, I'll try it out!

Indeed, there are so many natural methods to interpolate concepts, and I agree 100% that there some are better than others at certain tasks.
Compared to famous Img2Img, I understood this as a "generalized" method to interpolate. Since if you take \mu = 1.0, this becomes just Img2Img interpolation. You can read the paper to see the effect of \mu on interpolation, and it's quite interesting. Since this is more general approach, there are more things to tweak and figure out I guess...?

I only gave the paper a cursory look, but isn't this the same thing as prompt editing?

Prompt edit seems to be special case of MagicMix where Kmax = Kmin = T and nu = 0. MagicMix is more like Img2Img than sampling where ive understood it

It is prompt editing + prompt interpolation. So N steps of A, M steps of A transitioning to B, and then the remaining steps at B.

I am not in the field, but I am working on multi-materials architecture designs and one of the questions how can I started design such system to come up with new possibilities of architecture and new geometries. While reading the abstract in that paper it mentions "novel object synthesis" what this mean actually?

I am also struggling to understand what are the possibilities behind the explosion of diffusion models beyond art, sorry if that sound ignorant but I want to understand and deploy this in my work hopefully.

I would say that you should colab with someone in the field, doing this by yourself when you are not in the field might take a lot of time and a bit lower chance of success.

Thanks. I understood it partially, but your explanation made everything crystal clear and things all clicked in an instant.

I wish more papers has an "intuition" section like this.

Do you have any names you would recommend?

pretty sure 'novel object' means a image that is the combination of multiple objects so for instance - dog + coffee_pot = dog with some characteristics of a coffee_pot (in the image examples the head was short of coffee pot like). rabbit + tiger = rabbit with tiger charactistics. rabbit + sheep = rabbit with sheep characteristics (the example showed a rabbit with a wool like texture as opposed to rabbit fur texture).

You just need to find someone working in the ML/AI that you could work with. This person needs to understand your problem and convert it into a ML problem. If you don't mind, I can take a look. I am looking for someway to apply my skills anyway.

Found relevant code at https://magicmix.github.io + all code implementations here

--

To opt out from receiving code links, DM me

I've never been able to get a PC-based version of anything to run successfully, so if anyone makes or encounters a colab, huggingface, or similar implementation, I'd much appreciate a link!