Extra Ordinary, the Series

Ever wanted to quickly capture ARKit facial blend shapes for a OpenUSD project? There a number of free tools around, but I wanted to share one I have been working on as part of a larger project. The nice thing is there are zero software installs – it runs in a web browser!

Does it work in all web browsers? No. It is pretty CPU hungry. My iPhone X for example does not work (it stutters for a few frames then gives up), however my wife’s newer (no comments please) iPhone does work. My iPad also works. But the site is really designed for desktop use.

Currently the page only captures ARKit facial blend shapes (and head rotations). I plan to extend that to capture upper body movements in the future.

But given those limitations, how to use it?

• Go to https://ordinaryanimator.com/try
• Click the cog near the top right corner to make sure your preferred camera and microphone are selected. (It captures audio as well, although in Chrome it is saved as a webm file that you have to convert to mp3 using a utility like ffmpeg.)
• Turn on the camera in the top left corner.
• Wait. There is a delay while it starts up (another task on my to-do list).
• Once the character responds to movements, you can click the “Record” button in the bottom left corner.
• Click “Stop” to finish the recording.
• In the bottom middle, the “Playback” tab has a “Play” button if you want to review the recording.
• Select the “USD” tab.
• Click “Select output directory” and pick an empty directory to download files to.
• Click “Export” to download a set of files to the directory. This includes the character (with texture files), the animation clip, audio file, and a USD stage file linking it all together.

If you open the stage file in NVIDIA Omniverse you can then play the animation. What about other tools? Give it a try, but currently I am using Omniverse materials so they may not work on other platforms. (Another to-do item is to move to Material X, or at least USDPreviewSurface.)

The site is still under active development and I have a long list of tasks to do. But sharing this tool in case of use or interest to anyone. For example, for Halloween I took a model of a Pumpkin from Sketchfab, added ARKit blend shapes to it, then used a face recording to make the pumpkin talk. A bit of fun.