TensorFlow.js in the browser for expression prediction

In my last post I introduced the approach of using Google MediaPipe to control VRM character facial expressions. Hasn’t this been done before? Yes! There are some excellent projects around, including XR Animator. But there is nothing like firsthand experience. In this blog post I describe my efforts to use TensorFlow.js to train a machine … More TensorFlow.js in the browser for expression prediction

Getting Google MediaPipe to control VRM Characters

I am working on a web based 3D animation project where you can create an animated video using 3D character models, like a cut scene in a video game. Facial expressions in 3D are frequently animated using “blend shapes” (aka “morphs”, “shape keys”, or “facial landmarks”). Google MediaPipe can be used to live capture a … More Getting Google MediaPipe to control VRM Characters