CMU School of Drama


Tuesday, November 22, 2022

3D for everyone? Nvidia’s Magic3D can generate 3D models from text

Ars Technica: On Friday, researchers from Nvidia announced Magic3D, an AI model that can generate 3D models from text descriptions. After entering a prompt such as, "A blue poison-dart frog sitting on a water lily," Magic3D generates a 3D mesh model, complete with colored texture, in about 40 minutes. With modifications, the resulting model can be used in video games or CGI art scenes.

5 comments:

Unknown said...

I personally am so excited for this 3d model generator to come out. My opinion on AI has changed over the past month as I have gotten more hands on with my VMD Stagecraft mini. I think that this text to 3d model AI software will make theater and design more accessible to those who do not have the years of experience it can take to become an artist. As someone who also uses 3d printers I am excited to see the implications behind more accessible 3d models in that realm too. More accessible 3d print designs could lead to cool props and models for both theater makers and cosplayers alike. I could see how more niche props could be created by ai as well as sculptures and art pieces. All in all I am really hopeful for the creative results of this new code.

Theo

Owen Sahnow said...

This is a very interesting idea - unfortunately I’m sure it’s going to be a minute before this technology makes it into the theater industry. They said the code wasn’t released along with the academic paper and the article doesn’t mention where it’s learning from (maybe thingiverse and google images?) That honestly would be pretty awesome if you could tell the AI to build a standard broadway style hardcover flat at certain dimensions and it would be able to spit that out. I imagine it would be more practical to write code that creates standard scenery like platforms and flats in 3D in autoCAD the same way that Al’s steel mill works. I wonder if there’s any call for this in the props world because although this frog is unique because it was generated by AI, you can probably just look at thingiverse for the same search terms and come up with something suitable.

DMSunderland said...

This is fascinating. I mean, I already and am still in shock at the intelligence and accuracy often displayee by AIs able to take text and turn it into a series of 2d images. But this? It requires an extra layer of context and scaling that I didn't think AI would have at this stage. I agree that is must be learning from thingiverse because it must be modelling these proportions from somewhere, and then coloring in the image based on the same bank of 2d images that it draws from for 2d images. There is certainly an implication that one day one might be able to use this to design structures using materials information in relation to the density, volume, and cross section of known materials in order to have it also test the structural properties of the structure that it has designed. This is a very cool thing that I am excited to hear more about down the line.

James Gallo said...

This is super interesting to me. I do a lot of 3D modeling and I can see this as a great way to enhance that. This also might be a great way to create some 3D props in a quick amount of time. With our 3D printer, this would give students the opportunity to very quickly come up with a model of something and have it in their hands in a few hours. I could also see this as a very good 3D drafting tool. If you are trying to model something more complicated that might include a variety of objects together, this is a great way to be able to quickly put things together and have a more fully realized product. As 3D modeling becomes part of more and more people’s routines, I see this as being a very useful tool in processes. I am excited to give this a try!

Sophie Rodriguez said...

This is so cool - to think that machines, artificial intelligence, can create accurate 3D models of something is absolutely insane to me. I can hardly even comprehend that AI can create 2D images from text, to be honest I only recently learned about that. Now I'm learning that AI can do the same thing but in 3D, it's interesting to think about considering how individuals like myself are still learning how to model things in 3D - and Artificial Intelligence can do it better than us, that's a weird concept to wrap your head around. Even weirder, one day this technology will develop even more and I'm sure it will do things that I can't even think of currently. I would love to see this type of technology make its way into our everyday lives to make our jobs just a bit easier although that would be in the very (very) distant future.