Extra Ordinary, the Series

In this post I describe my current plans for animation of VRoid Studio characters in Unity (using animation clips). I might change my mind on the approach later as I learn more, but this is my current thinking.

Background

Animation clips in Unity capture position information for all the joints of a character. Animation clips can be used for standing still (idle), walking, running, jumping, fighting, and any other actions you want a character to take. Some animation clips are freely available, others can be purchased from places such as the Unity asset store, others you can create yourself.

Animation clips such as walking typically control the whole body. The body sways as it walks, moving legs and arms as a part of the walk cycle. Other animation clips may control only part of the body, such as waving an arm, but the better quality clips do tend to control the whole body. E.g. when waving to someone, the rest of the body often moves as well. If a clip does only control part of the body however then clips can be layered, where one clip overrides the part of the body it is designed to control.

There is a problem however. Not all character creation software uses identical bone structures. How then to reuse animation clips between characters? To help solve this problem, Unity supports the concept of an Avatar which you can add to humanoid characters. It identifies which bones represent common humanoid parts of the body such as legs, arms, the head, and so forth. Unity then does a pretty good job or “retargeting” clips to different characters by leveraging this Avatar information. VRoid includes Avatar descriptions in the character, so no additional effort is required here. (If you look in the “<character>.AvatarDescription” directory, you can open the Avatar description and it includes bindings from the VRoid character bones to all the Avatar standard bone names.)

With VRoid characters, all characters have identical bone structure (except for the hair), but there is still a challenge of finding clips and then applying them to characters.

Animation State Machine

Unity has the concept of an animation state machine. This can control the order in which different clips are played etc based on the characters state. Currently I am not planning to use this capability. I have found a “Locomotion” package on the asset store that can deal with walking on uneven surfaces (e.g. walking up a sloped surface). On top of that, I plan to use Cinemachine timelines to control when different animation clips are played in a scene. You just drag the animation clip into the timeline. As a result, I don’t think for my purposes I need the complexity of animation state machines. I am just going to use the Locomotion package and timelines to control character actions and movement.

Finding Animation Clips on Mixamo.com

Finding existing animation clips can save a lot of time. You can create new ones (or tweak existing ones), but a good animation clip is difficult to create. So reusing existing ones as often as possible is a good objective.

The mixamo.com (from Adobe) site is a good site for free animation clips. You have to create an account, but you can then pick a character from the site and then browse through the animations so you can view them applied to the character. (I am planning to use VRoid Studio created characters, so it is only the animation clips I am after.) You then download a FBX file that includes the character and the animation clip. (FBX appears to be a ZIP-like format for 3D animation.)

When you pick an animation you like, download the FBX file with Format set to “FBX for Unity(.fbx)” and Skin set to “Without Skin”.

Then drag the downloaded FBX file into Unity’s Asset directory tree. Unity will find the new file and load it. Expanding the FBX file in the asset view shows the contents of the FBX file. At the bottom is the animation clip. (I picked the character “Pearl” and animation clip “Orc Idle”. I dropped the FBX file into the directory “Assets/Mixamo/Orc Idle”.)

But you need to do a bit more. Click on the root “pearl@Orc Idle” node an look at the inspector panel.

Change the Animation Type to “Humanoid” and click “Apply”. This creates an Avatar for the animation. You can then use the clip with VRoid characters.

See also this video: VRoid anime characters to Unity for mecanim and facial blendshape animation.

If you want to edit the clip, you cannot modify it inside the FBX file. You need to make a copy and edit that copy. Single click on the “Orc Idle” animation clip inside the FBX file, selected Edit/Duplicate from the menu (control-D), and it will extract the animation clip into a separate “Orc Idle.anim” file in the same folder. The FBX file (“pearl@Orc Idle”) can then be deleted if you wish.

Repeat for all the animation clips you want to download.

Others Sources of Animation Clips

As you add other Unity packages to your project, you may find addition animation clips inside the packages. Unity-Chan is a free character from Unity for example (with some animation clips that look surprisingly like the ones built into VRoid), and the Locomotion package also has a sample character in it with some animation clips. There is also an animation clip marketplace, MotionLibrary.com, which also provides Unity integration.

I am thinking to build up a set of clips with a similar feel per character, where possible. I would like to have different idle clips for significant actors to give them a bit more individual personality. I plan to rename the clips to include the character name. For example, Mixmo exported the “Pearl” character with the “Orc Idle” animation clip as “pearl@Orc Idle”. That can make it easier to work out which clips are intended for which characters. For my Elanor character, I might rename this to “Elanor@Idle”.

I do not plan to have unique clips for all characters however – background scene filling characters for example I plan to have sharing animation clips for to save time.

Creating Your Own Animation Clips

Creating high quality animation clips takes effort. I believe many are created using “mocap” rigs (motion capture rigs) which track and record a real person’s body movements to create an animation clip from. These can cost thousands of dollars (much more for Hollywood quality), so are not a serious option for me. You can create your own clips by hand if you wish as well, but it is a lot more work.

Creating and editing animation clips inside Unity is not the best experience, but is possible. To create a new animation clip asset, right click in a folder and select “Create / Animation”. This will create a new .anim asset file. One approach to edit animation clips is to add an “Animation” component to the VRoid character, add the animation clip(s) to be edited to the Animation component (specify the number of clips in the “size” property), select (single click) the character in the scene hierarchy, then open the “Animation window”. The Animation window will include a drop down list of all the animations in the Animation Component for the selected character.Once there, you must add properties to the animation clip. It is important however, as VRoid characters are humanoid, to only use the “muscles” under “Animator”. Do not use any of the other properties. (I got errors about “legacy clips” when I did.) This video at around 30 minutes in has a quick example: Unity3d GTA style multiplayer game tutorial #12 (finishing series, animations, options, stars).

You can use this approach to create some simple animations if desired. Personally I am looking forward to maturation of some of the webcam based mocap solutions – capturing movements just using your computer’s camera. While these may be of lower quality than a full mocap suit, it does offer an option that may still be more natural looking than hand created animation clips.

(Other interesting approaches to mocap are to use VR headsets and controllers (e.g. Final IK and VR MoCap – HTC Vive Motion Capture) and devices such as LeapMotion to capture hand gestures.)

Using Animation Clips

To use animation clips, create an empty GameObject in the scene, open the “Timeline” window, click the “create” button (this will add components to the GameObject to turn it into a real timeline), save the timeline asset to asset hierarchy (I have a “Timelines” folder under Assets), drag the character from the scene hierarchy to the timeline window, select “add animation track”, then drag and drop animation clips into the track. The following is an example with two overlapping animation clips – Unity will automatically blend from one clip to the other.

The above was admitted a whirlwind example. I plan to go into more detail after getting a few more topics out of the way.

Conclusions

Animation clips are an important aspect of creating an animation using VRoid characters in Unity. Using existing animation clips can save a lot of time, if you can find the animation clips you need. Sites such as mixamo.com and motionlibrary.com look like promising sources to build up a base set of animation clips.

Next, I am planning to describe the “Locomotion” package by Runevision on the Unity asset store. This package has been around a while, but looks like a good solution allowing characters to walk over non-flat surfaces. You can start with just a standing and walking animation clip, with running added if appropriate. You move the characters around the scene, and the package does the best job it can to decide if walking or running, making sure the character’s feet are on the ground even if on a slope, steps, and so on.

After that comes the 2020.1 Animation Rigging preview which includes abilities such as doing simple IK (Inverse Kinematics) based animations to override animation clips. For example, hand waving instead of working out the angles and strength for each muscle, you can animate the position of the wrist and the IK support in the animation rigging package will work out how to bend the arms etc to achieve the desired target. This makes it easier to add little tweaks to a scene such as waving a hand.