Animation – Extra Ordinary, the Series

AI Agents, Animation, and Minecraft

April 12, 2024

I have been exploring using Generative AI (LLMs specifically) to assist with going from a screenplay to an animated video, learning about relevant technologies along the way. (No, I have not got that far, but it’s fun trying and learning!) For example, at the recent GTC 2024 there were some interesting sessions on Robotics from Google and Disney. It reminded me of an older paper on Voyager – getting AI to play Minecraft. The GTC videos are just being published now, so I also just got to view Jim Fan’s presentation on “Generally Capable Agents in Open-Ended Worlds”, which also mentioned Voyager. This blog summarizes the highlights from this work that interest me. … More AI Agents, Animation, and Minecraft

Robotics for Character Animation?

March 20, 2024

I have been watching a number of the NVIDIA GTC (GPU Technology Conference) talks on this week. I have mainly been watching talks related to Omniverse (their 3D rendering and simulation software). I was interested in automation of simple animated videos, such as how much can AI (such as large language models) reduce the effort … More Robotics for Character Animation?

TensorFlow.js in the browser for expression prediction

January 11, 2024

In my last post I introduced the approach of using Google MediaPipe to control VRM character facial expressions. Hasn’t this been done before? Yes! There are some excellent projects around, including XR Animator. But there is nothing like firsthand experience. In this blog post I describe my efforts to use TensorFlow.js to train a machine … More TensorFlow.js in the browser for expression prediction

Retargeting Skeletal Animation to USD

November 13, 2023

I am working on a project where I want to manipulate Skeletal Animation clips (for 3D computer animation) in JavaScript, then export to USD. Depressingly it’s not that hard. Why depressingly? Because I spent a LONG time working out what turns out to be pretty simple code. Luckily I am not BITTER about that at … More Retargeting Skeletal Animation to USD

Options for representing Animation in OpenUSD

September 27, 2023

I have been exploring NVIDIA Omniverse and the standard it is built on, OpenUSD (aka USD). It provides a number of ways to animate objects in a 3D scene. My personal interest in animation is I am trying to build a tool to simplify assembling your own animated video where the final result is exported … More Options for representing Animation in OpenUSD

Why I Stopped using Scene Activation in Unity

May 22, 2022

The Unity Sequences package support has a number of features to make it easier to create movies, game cutscenes, or in my case short animated video clips for including in my motion webcomic. When I first started using Sequences, I planned to use Scene Activation tracks, but over time I found a number of limitations … More Why I Stopped using Scene Activation in Unity

Generating iFacialMocap Blend Shapes in Unity for VRoid Characters

October 4, 2020

I have been exploring iFacialMocap, an iOS app that uses the iPhone “ARKit” library to convert facial expressions into a series of movements. These can be sent to Unity so your character inside Unity follows what is being recorded on your iPhone. A challenge however is the iOS ARKit library uses more blendshapes than come … More Generating iFacialMocap Blend Shapes in Unity for VRoid Characters

Extra Ordinary, the Series

Make the Extraordinary, Extra Ordinary in Your Life!

Tag: Animation