A personal side project is converting textual descriptions of camera positioning (for filmmaking) to a machine readable form. Screenplays normally do not include camera directions (that is left for the Director to do on set), but I am interested in creating 3D animations from just a screenplay and want to give the writer control over … More Text to Camera Shots with LLM AI
In my last post I introduced the approach of using Google MediaPipe to control VRM character facial expressions. Hasn’t this been done before? Yes! There are some excellent projects around, including XR Animator. But there is nothing like firsthand experience. In this blog post I describe my efforts to use TensorFlow.js to train a machine … More TensorFlow.js in the browser for expression prediction
Generative AI can be great for background images in animated videos. Want a single shot walking along a path? Creating a background image can take some time to assemble, so generative AI for creating 2D images with Stable Diffusion, Midjourney, DALL-E, etc. can be a real time saver. The following image was created using the … More Generative AI for 3D Scene Backgrounds
I love the progress Meta are making (https://dulucas.github.io/agrol/) on animating the lower body from just a VR headset and hand controllers. Personally I would love to use this as a poor man’s option for mocap (Motion Capture), to create custom animation clips for characters. There are a number of camera based mocap solutions around these … More AGRoL, Avatars GRow Legs at Meta
Just as generative AI has been stirring things up with 2D images, people are also looking hard at 3D models too. One great little video I stumbled across lists a number of platforms, some of which I had not heard of before. First, the video: Some of the sites mentioned: Some of the platforms tackle … More AI 3D Model Creation
NVIDIA Omniverse includes Canvas, a cool little tool for creating 360 background images with minimal effort. You draw color patches where you want trees or bushes, and it uses generative AI to create photo realistic artwork. For free. … More NVIDIA Canvas
Will video from Generative AI kill 3D CG animation? I don’t think so. I think it will have its place, but I think better quality will be achieved using AI tools to help with 3D model creation and animation. … More Generative AI Video vs CG Animation
There is a lot of buzz going on around AI generative artwork. Rather than art generation, I am personally interested in AI tools to speed up boring (non-creative) workflow tasks. For example, in this post I explore some work that has been done on how to create a 3D background for a film or cartoon … More 2D to 3D Background Artwork
6:15 PM I am exploring assembling scenes for computer animation from text based on AI. This is partly because I would like to speed up my workflow, and partly as a learning exercise to understand the challenges of applying AI to real life problems. This time I am looking at GPT to see how it … More Entity Extraction for Screenplays with ChatGPT
In my last blog I introduced this new series on applying machine learning to screenplays with the goal of automating some of the tedious steps of creating an animated cartoon using 3D character models and animation. Even if you are not in the animation space, you might find it interesting in an introduction to the … More Applying Machine Learning Classifiers to Screenplays for Animation Automation
Extra Ordinary, the Series 