I have been watching a number of the NVIDIA GTC (GPU Technology Conference) talks on this week. I have mainly been watching talks related to Omniverse (their 3D rendering and simulation software). I was interested in automation of simple animated videos, such as how much can AI (such as large language models) reduce the effort … More Robotics for Character Animation?
How to render a 360 image for display in a VR headset, such as the Meta Quest 2? This is not a new problem, but I had a bit of trouble tracking down all the current steps. So I thought I would share in this blog how I created a 360 3D image in Omniverse … More 360 Images from NVIDIA Omniverse to Meta Quest 2
A personal side project is converting textual descriptions of camera positioning (for filmmaking) to a machine readable form. Screenplays normally do not include camera directions (that is left for the Director to do on set), but I am interested in creating 3D animations from just a screenplay and want to give the writer control over … More Text to Camera Shots with LLM AI
I have been exploring OpenUSD (formerly USD from Pixar) for creating computer generated animated videos. I create the characters, backgrounds, and animations as 3D models, then render it to a 2D video you can view on social media. Out of curiosity I just got a store demo of the Apple Vision Pro, exploring both 180 … More 3D Video vs 3D Animated Scenes
NVIDIA Omniverse includes a number of AI based tools, including Audio2Face and Audio2Gesture. Audio2Face takes an audio file and generates facial blend shape animations. Audio2Gesture can take the same audio file and instead generate upper or full body animation clips. But it’s not easy at first blush to combine the two results. In this blog … More Audio2Face + Audio2Gesture – Merging the Streams with OpenUSD
There was a recent Omniverse Discord thread that had some great links to videos by @Simplychenable. I am capturing the videos here for later reference. The following video on “Building Asset Pipelines” is my favorite at the moment. There is a lot of introductory content on OpenUSD. What is a prim? What is a prim … More Resources to help learn OpenUSD
I am interested in OpenUSD for portability of content. Instead of being tied to one rendering engine, my content can be rendered in multiple tools. At least that is the OpenUSD promise. But there are practical issues, one of which is materials. How to create materials that work across multiple tools? One project that gets … More OpenUSD UsdPreviewSurface Material Portability
Want to add a bit more “life” to your video voiceovers, but don’t want to show your own face? How about using a 3D avatar, synced with your facial movements? When to use an Avatar? Are avatars always the best option? No! If you are your brand (say, you are an individual business coach) then … More Animated Voiceovers
I sometimes listen in on the Academy Software Foundation (https://www.aswf.io/) sessions. The latest Web OpenUSD online meeting had a presentation by CloudZeta, OpenUSD for 3D Cloud – Content Distribution & Collaboration. 3D assets are large, so having a shared online storage area that 3D creators and consumers can share with web based preview would be … More CloudZeta – GitHub for 3D Assets
In my last post I introduced the approach of using Google MediaPipe to control VRM character facial expressions. Hasn’t this been done before? Yes! There are some excellent projects around, including XR Animator. But there is nothing like firsthand experience. In this blog post I describe my efforts to use TensorFlow.js to train a machine … More TensorFlow.js in the browser for expression prediction
Extra Ordinary, the Series 