Category: Behind the Scenes

AI Agents, Animation, and Minecraft

April 12, 2024

I have been exploring using Generative AI (LLMs specifically) to assist with going from a screenplay to an animated video, learning about relevant technologies along the way. (No, I have not got that far, but it’s fun trying and learning!) For example, at the recent GTC 2024 there were some interesting sessions on Robotics from Google and Disney. It reminded me of an older paper on Voyager – getting AI to play Minecraft. The GTC videos are just being published now, so I also just got to view Jim Fan’s presentation on “Generally Capable Agents in Open-Ended Worlds”, which also mentioned Voyager. This blog summarizes the highlights from this work that interest me. … More AI Agents, Animation, and Minecraft

Accessing the Omniverse Message Bus from Python

April 11, 2024

In this blog post I explore sending and receiving message bus events using the NVIDIA Omniverse framework in Python. I create two extensions: one to publish messages, and another to receive them. In an Omniverse Cloud project, the message bus will span microservices (using Redis I think read somewhere). In this blog, I use it … More Accessing the Omniverse Message Bus from Python

3D Product Rendering in Web Pages with Omniverse Cloud

April 3, 2024

Interested in high quality 3D product configurators for your ecommerce site? The Omniverse team livestream today recapped the recent GTC conference, and over half the video was talking about product configurators including web delivery! What is Omniverse? Sometimes it’s hard to keep up with everything that is happening with just Omniverse, let alone the rest … More 3D Product Rendering in Web Pages with Omniverse Cloud

Audio2Face + Audio2Gesture – Merging the Streams with OpenUSD

February 15, 2024

NVIDIA Omniverse includes a number of AI based tools, including Audio2Face and Audio2Gesture. Audio2Face takes an audio file and generates facial blend shape animations. Audio2Gesture can take the same audio file and instead generate upper or full body animation clips. But it’s not easy at first blush to combine the two results. In this blog … More Audio2Face + Audio2Gesture – Merging the Streams with OpenUSD

OpenUSD UsdPreviewSurface Material Portability

February 6, 2024

I am interested in OpenUSD for portability of content. Instead of being tied to one rendering engine, my content can be rendered in multiple tools. At least that is the OpenUSD promise. But there are practical issues, one of which is materials. How to create materials that work across multiple tools? One project that gets … More OpenUSD UsdPreviewSurface Material Portability

Animated Voiceovers

February 3, 2024

Want to add a bit more “life” to your video voiceovers, but don’t want to show your own face? How about using a 3D avatar, synced with your facial movements? When to use an Avatar? Are avatars always the best option? No! If you are your brand (say, you are an individual business coach) then … More Animated Voiceovers