I recently shared a blog post on a Unity script I am experimenting with to add ARKit compatible expressions to VRoid characters. Of course the day after, another tool became available that could have saved me quite a bit of hair pulling! Lol! I thought I would write up my brief experience with Hana_Tool and compare some of the expressions done by my script versus Hana_tool.
TL;DR:
- My goal is to create lots of expressions in Unity using animation clips – I am not trying to enhance a VRM file for use by tools such as PerfectSync or Luppet. So my personal needs are a bit different.
- Hana_Tool has additional blend shapes my script does not do, but there are a few that my script does that Hana_Tool does not. I may create a new version of my script to add blend shapes on top of Hanna_Tool, so I can combine them together in new and additional ways.
Hana_Tool_v2
To learn about Hana_Tool, I suggest you first read the blog post, Implement with a click! Perfect Sync BY HANA Tool. (I used Google Translate to read it as it is in Japanese.) The tool is available on Booth for a small fee. The Google translated instructions were fairly easy to understand, but to repeat in English the steps I took:
- I extracted the downloaded ZIP file from Booth.
- I dragged and dropped the “HANA_Tool_v2_9_1_VRM.unitypackage” file into my “Assets” folder in Unity. This made a “HANA_Tool” menu appear in the menu bar.
- I imported a VRM file exported by VRoid Studio 11.2. (The blog says this version is required for compatibility reasons.)
- I dragged my VRoid character into a scene.
- I selected “HANA_Tool” / “Reader” from the menu. That brings up a new window.
- I dragged the “Face” object from the scene character into the “SkinnedMeshRenderer” box (where it says “None”).
- I picked the “Female” checkbox (because my character was female).
- I then clicked “Add” so it was dark grey (instead of light grey).
- Finally I clicked “Read BlendShapes” at the bottom.
Then I waited 5 minutes for it to add all the blend shapes to my character.
Just kidding!
My script takes 5 to 10 minutes (no idea why – I need to speed it up), this tool was instant!
I then hooked it up with IFacialMocap and away it went! Easy.
Check out other posts from the same author for information about an application called “PerfectSync”. I have not used that application myself as I am trying to create animated cartoons in Unity. But if you are a VTuber, that might be a useful tool. The blend shapes are designed to work with PerfectSync.
Comparison
So which tool is better? Mine of course!
Again, just kidding!!!
I prefer many of the expressions in Hana_Tool – it does more lip movements than I do. It was interesting to see that Hana_Tool also skips a number of the blend shapes that I skipped as well. I don’t think these blend shapes matter so much for VRoid characters.
First the neutral expression.
“cheekPuff” is a bit hard to see here, but it is noticeable when turned on/off. I did not attempt “cheeckPuff” in my script, so this one is a clear benefit in Hana_Tool.
“noseSneer” was interesting – it changes the eyebrows instead of the nose. You cannot see changes in the nose normally, so disdainful eyebrows might be a good alternative solution. The curve of the eyebrows is interesting as well – really gets a reverse curve on the brow.
For “mouthFunnel”, here first is the side profile of the neutral expression.
With “mouthFunnel” on, the lips move forward, but not open. The ARKit image showed it to be open, but for my purposes this is probably fine. I can always use the “U” and “Ah” etc mouth shapes from the original character set of blend shapes.
The “mouthPucker” expression narrows the mouth. For puckering for a kiss, this might need to combine with some other blend shapes, such as “mouthFunnel”.
“mouthRollUpper” at 100% by itself just looks weird to me!
Reducing to 30% “mouthRollUpper” looks much more normal and useful. Closing the lips more tightly.
“mouthRollLower” for my character frankly seemed strange because it went into the mouth and out of the skin near the nose, but combining with “jawOpen” it could be useful.
“mouthShrugUpper” I am not sure is much different than “mouthRollUpper”. “mouthShugLower” might be useful. It narrowed mouth, lowered in middle a touch.
“mouthClose” closes the lips together more tightly.
The “mouthSmileLeft” and “mouthSmileRight” look pretty good.
“mouthFrownLeft/Right” is pretty good too.
“mouthDimpleLeft/Right” is much like other expressions, but with less vertical movement. All up, does the job well.
“mouthUpperLeft/Right” control the upper left/right lip – but it appears to work by just cutting off half way across the mouth, so you get the strange appearance near the middle of the mouth. My script did a better job of smoothing into the middle of the mouth here I think.
Here is “mouthLowerDownLeft/Right”, which is the same but for the lower lip.
While separately the lips look a bit strange, when Left and Right combined you can create some more useful expressions.
My script looks a bit different here due to trying to not move the middle of the mouth.
“mouthPressLeft/Right” are supported in Hana_Tool.
“mouthStretchLeft/Right” looks good.
“tongueStickOut” I did not have, so is a real bonus. There are additional “tongueUp” (curl tip of tongue up), “tongueDown” (curl tip down), “toungueLeft/Right” for further tongue movement.
“irisMoveBack” is similar to shrinking the iris.
One big difference is how “jawOpen” is done. In ARKit, “jawOpen” lowers the jaw stretching the mouth open wide. Then the teeth can be moved separately: see “teethUpperUp/Down” and “teethLowerUp/Down”. Making the jaw open alone looks, um, well, do you have grandparents with false teeth?
However, combining “jawOpen” with the teeth positions can create some good expressions, although I am not sure this case it is strictly following the ARKit definition of what the blend shape should do.
This is one case where I want to keep my script definition of “jawOpen”, which expands the jaw outline of the face downwards. I may however separate out the “Oh” mouth shape as it can be added manually later.
I think Hana_Tool does “browDownLeft” and “browDownRight” correctly. The eyebrow moves directly up and down (no twist as in mine). I suspect you use “browInnerUp”, “browOuterUpRight”, and “browerOuterUpLeft” together to get different eyebrow angles.
Problems
The biggest problem I have had with VRoid is closing of the eyes. I think this is because of the degree of variation and control VRoid Studio gives over eye shapes. But it can make it hard to get the eyes right. Hana_Tool on the character I tried does not fully close the eyes. The default VRoid eye closed blendshape worked better in this case. But this has not been true for other characters – the default blend shape closes the eyes too far. This might be one area where each character has to adjust the strength to get it right for that particular character.
Conclusions
Do I plan to include all the Hana_Tool expressions in my C# script? No. Instead I am thinking to pick the blend shapes of my script I want to keep, rename them, then add them in addition to running Hana_Tool.
It is important to remember that I am not trying to create a set of ARKit based blend shapes. I am not trying to use tools such as PerfectSync or Luppet to be a live streaming VTuber. Rather, I am trying to have a good set of primitives to create animation clips from for creating animated cartoons. So I am considering adding my extra blend shapes to the end with different names so I can pick and choose between my blend shapes and those from Hana_Tool.
Extra Ordinary, the Series 