[D] Hello, everyone. I watched an explanation on the use of diffusion models for creation of 2d images.

I just wonder, I think we are somewhat far away from 3d model generation. First, I think it would be much more computationally expensive. Second, I am not sure whether we have such a large set of training data. And third, the input and output that we have in 3d graphics is somewhat different from pixels, i.e. we are working with triangles in 3d graphics (maybe this is not as hard, as we can always start with vertices and then estimate triangles.

What's your take on that?

Comments

You must log in or register to comment.

a_marklar t1_jdx7ryn wrote on March 27, 2023 at 9:13 PM

In a limited sense, we're already there. For example, the microsoft avatar generation.

I'd guess that its very unlikely that generative models will use triangles. Point clouds, SDFs, parametric surfaces all seem to be better data formats for these types of things. Those can all be converted to triangle meshes if that's required.

konstantin_lozev OP t1_jdxahve wrote on March 27, 2023 at 9:31 PM

Thanks! That looks promising, indeed, if somewhat narrow.

NotDoingResearch2 t1_je7lqmi wrote on March 30, 2023 at 12:06 AM

How easy is it to convert point clouds or SDFs to triangle based meshes with useable topology?

a_marklar t1_je7ogfc wrote on March 30, 2023 at 12:26 AM

Depends what exactly useable topology means to you but my first stop is always CGAL

Sirisian t1_jdy84ph wrote on March 28, 2023 at 1:36 AM

https://dreambooth3d.github.io/
https://ryanpo.com/comp3d/
https://ku-cvlab.github.io/3DFuse/
https://lukashoel.github.io/text-to-room/
https://zero123.cs.columbia.edu/

There's so many papers released every week. If you used https://www.connectedpapers.com/ you'd probably find more. Some of these released at the same time mind you. So many teams are working on nearly identical projects.

slipknot25 t1_jdxn5ed wrote on March 27, 2023 at 11:01 PM

Something already here : https://instruct-nerf2nerf.github.io/