Implementing Captions in Unity using UI Document

Here is my current implementation of captions in Unity, the writing you see at the bottom of the screen that shows what the character is saying. You might do this when the original movie is in a different language, or in my case, I am using captions to avoid having to worry about voice actors for some simple animated cartoons I am working on. (One day I might try voice synthesis, but so far they have lacked the emotional depth for what I want.) This blog post describes how I implemented captions using Unity’s UI Toolkit.

Here is an example caption:

The black background text is to help make the text easier to read (it improves contrast of the text with the background it is on). The background is resized to match the size of the text.

I have previously been using speech bubbles, like comics. I am exploring captions as an alternative. For speech bubbles you have to find a good position for the speech bubble, make sure the text fits in well, point the tail towards the mouth correctly, and so on. If you want to translate the text to multiple languages, it becomes harder as the amount of text is different in different languages.

Canvas and Panel

The first step was to create Canvas and Panel game objects in the scene hierarchy. (Creating the Panel from the Unity menus created the parent Canvas game object for me.) I renamed them to make their purpose clearer.

The important setting for the Canvas is to have “Render mode” set to “Screen Space – Overlay”. This keeps the text in a constant position on the screen (it does not move as the camera moves).

A child of the Canvas is the Panel for displaying the content. The default Panel created had an image in it. I disabled this component in the screenshot below so you can see what it was (you can remove the component). I then added a UI Document component that I use for positioning content on the screen. I also added a custom component that updates the document to update the caption text. I touch on snippets of this component at the end of this post.

The Panel Settings asset I created, but did not do much to. It has settings such as “Scale with Screen Size”.

UI Document

The UI Document is where the interesting settings are. If you double click on the UI Document asset, it will open up a “UI Builder” window. My document is pretty simple – I added two labels for captions at the top and bottom of the screen.

The most interesting thing in this post is the settings for those two labels to make sure they behave correctly. In my case, I wanted white writing (by default) on a semi-transparent black background to make the writing easier to read. I wanted the rectangle as small as possible to match the text. The captions are positioned relative to the top and bottom of the screen (whatever size it is). (I normally display captions at the bottom of the screen, but sometimes I have found showing a caption at the top of the screen useful.) The text should not expand too wide (it should not reach the left and right edges of the screen).

Here are the settings for the top caption label.

I ended up starting with Visibility set to off as when there is no text, small dark patch was still visible. So in my script when I set the text to display, I also set Visibility to true; when I clear the text, I also set Visibility to false.

Next is the position (and alignment).

The position is important. The top text has position set to “Absolute”, with Top set to 0 (so it sticks to the top edge of the screen with an offset of zero). I set the alignment to centering. The view port section of the UI Builder shows the component outline stuck to the top as follows.

If you click on the Margin property, it shows the padding and margins I have set up. Orange is margins, green is padding.

The Margin & Padding values are as follows. The top margin is what leaves a space above the text at the top of the screen. The left and right margins stop the text expanding too close to the left and right edges of the screen.

Here is an example with some text added. The left/right edges have not been reached yet, so no line wrapping has occurred. The green is the padding around the text.

Here is longer text after the margins hit the edge. Notice the wrapping of text kicks in.

Without the coloring, you can see the padding affects the size of the background fill (to make the text easier to read) and the margin stops it from getting too close to the left and right edges of the screen.

Moving on, the text properties control the font, point size,  color of text, wrapping, etc. I used a True Type font (the “Font” property), but to get it working it was also necessary to create a Font Asset that references the font. It includes metadata such as line height. I found I had to adjust the settings in the Font Asset for my font to display correctly (the line spacing was wrong in my case).

The final interesting properties are the Background properties. I set the color to black and the transparency, the “A”  value of RGBA (Red, Green, Blue, Alpha) in the color picker, to around 66% (161 to be precise). (You may notice I also adjusted the paragraph spacing, but I don’t expect to use that often with captions.)

The Bottom Caption is set up almost identically of the top caption (I copied and pasted the Label, then adjusted it). The only difference is the Position property sets the Bottom to 0 instead of Top, and the bottom Margin is set to 32px instead of the top margin. This is so it sticks to a small offset away from the center of the bottom of the screen.

Setting Caption Text

The final step is then to set the caption text. In my case, I have a custom Timeline track implementation that takes the text and controls the mouth of the character so the lips move (a simple Lipsync solution). So I extended this code to also display caption text by updating the UI Document via a script.

Here is the start of the code of my captions component.

public class OrdinaryCaptions : MonoBehaviour
{
    public UIDocument uiDocument;
    private Label topCaption;
    private Label bottomCaption;

    private void Init()
    {
        if (uiDocument == null)
        {
            uiDocument = GetComponent<UIDocument>();
        }
        if (topCaption == null)
        {
            topCaption = uiDocument.rootVisualElement[0] as Label;
            bottomCaption = uiDocument.rootVisualElement[1] as Label;
            topCaption.visible = false;
            topCaption.text = "";
            bottomCaption.visible = false;
            bottomCaption.text = "";
        }
    }

To set the caption text I added

    public void ShowCaption(bool atTop, string speaker, string dialogue)
    {
        Init();
        if (atTop)
        {
            topCaption.text = MakeText(speaker, dialogue);
            topCaption.visible = true;
        }
        else
        {
            bottomCaption.text = MakeText(speaker, dialogue);
            bottomCaption.visible = true;
        }
    }

The text I included the speaker in yellow text, which is possible if the “Rich Text” property is enabled on the Label components in the UI Document.

    private string MakeText(string speaker, string dialogue)
    {
        if (speaker != null && speaker != "")
        {
            return $"<color=yellow>{speaker}:</color> {dialogue}";
        }
        return dialogue;
    }

Here is example code to wipe all captions to make them disappear.

    public void ClearCaptions()
    {
        topCaption.text = "";
        topCaption.visible = false;
        bottomCaption.text = "";
        bottomCaption.visible = false;
    }

My custom Timeline track code then finds the captions component using

    var captions = FindObjectOfType<OrdinaryCaptions>();
    captions.ShowCaption(…);

Black Outlined Text

There is an alternative to having a transparent black background rectangle behind the text, which is to draw characters with a black outline. The black outline around the white characters helps make the white text stand out from the background image.

The Label component supports a few methods of doing this. The first is outing the “Text Shadow”. Setting a Blur Radius of around 5 shows some outline, but it is not very strong.

There is also an Outline Width setting you can use with an Outline Color. The problem is that the outline traces the edge of the characters, both extending outwards and creeping into the middle of the characters. The white areas get thinner and harder to see. Here is an outline width of 2. (With an outline width of 5, there is no white left in the characters.)

So how to maintain the full white text, but have a solid black outline? One approach that is a bit crude but appears to work is to have two labels, one in front of the other. The label in front is pure white text. The other has an Outline Width of 5 of color black. Because the white label is in front, you don’t get the same bleed problem as above. This is what the above screenshot used.

Of course you can mix and match a bit. For example, you could have the transparent black rectangle behind all the text in combination with the above. But right now, I am leaning to the cleaner look of no background rectangle and using the two layered labels.

Conclusion

I think speech bubbles look more interesting visually, but one of my personal goals is to create content faster. Speech bubbles do take extra time to position manually, and you have to worry about camera panning or moving characters. These problems go away with captions at the bottom of the screen. I have not yet decided which way to go on my own project.

But I hope you found this run through on how you can use a UI Document in Unity useful for creating a captioning solution. It is not particularly difficult, but getting the properties right to get the black background to match the text took a bit of experimentation, hence this post.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s