Extra Ordinary, the Series

USD (Universal Scene Description) is an file format originally developed by Pixar for describing 3D scenes. A scene can include a collection of sets, characters, and props. In this blog I give a quick introduction, including how it is used to compose a hierarchy of sequences and shots similar to the Sequences package in Unity. My interest is whether using USD is a better approach than using the native Scene support in Unity for my solo computer animation projects.

What is USD?

USD originally came from Pixar, but is now an open source project (openusd.org) supported by a wide range of 3D CG tool developers. USD is used to describe a hierarchy of “prims” (primitives), similar to Game Objects in a Scene in Unity. Prims can be nested and referenced using a path much like a file system.

A basic USD file can be considered similar in concept to an XML document. There is a hierarchical structure with nesting, where nodes have a series of properties (similar to XML attributes). Multiple encodings of files are supported, including text and binary. 3D assets can be large, so binary can be faster to process.

Where USD becomes more interesting is that it defines semantics for combining multiple files. For example, you may decide to put each character in a separate file, then reference the character files from a scene file. You can also have one file override parts of another file, allowing “non-destructive updates”. You can also define sets of variants, selecting the appropriate variant when you reference the file. (I could imagine having different versions of a character.)

USD also supports the concept of schemas, where a set of strongly-typed supported properties can be defined. For example, a schema is defined for a sphere which has the standard attributes to represent a sphere, such as radius and color. USD supports defining of additional custom schema types.

Note that while USD is a file format, the project also includes a Python API for creating USD file contents. Many examples are provided as Python code to create the file contents rather than the final markup. This makes dealing with the more efficient binary format more feasible, with the plain text form being more useful for debugging purposes.

What is special about USD?

To understand the power of USD, its useful to think back about the problems it was created to solve. Pixar develops movies with large development teams. Different teams are responsible for different parts of a large production. One team may be responsible for developing characters, a specialist teams for fur or hair, another for developing locations, another for animation (include mocap), and so on. Pixar needs to enable each team to progress on a project in parallel, without the pain of merging different streams of work into the final product.

Pixar starts creating an end-to-end cut of a movie from the very start of the project using storyboards and staff as voice actors. They then iteratively improve sections of the movie, replacing a storyboard with a rough cut animation, then final animation – but you can watch the full movie at all times, and see the impact of bringing in different developments on the overall feel of the movie.

To merge the efforts of different teams with minimal pain, USD uses separate files where different teams own different files. These separate files are then merged into a final complete scene description. USD defines complex rules on how one file can override what is defined by another file, based on paths and priorities. This allows each team to work independently (on different files), with a final render according to the contents of those files.

For example, to implement different sequences of a movie, a project may define a file naming standard, such as giving files a prefix of “S01” for sequence 1, then “_002” for shot two in the sequence, then the purpose of the file. Some files are global for all shots in a sequence so leave out the shot number.

For example, sequence 1, short 2, might have files such as:

S01_characters.usda
S01_props.usda
S01_lighting.usda
S01_scenery.usda

S01_002_cameras.usda
S01_002_animation.usda
S01_002_lighting.usda

S01_002.usda

The final file references the previous files, where rules are defined as to which parts of files can override what is defined by other files. The global defaults for a sequence (such as the location and global lighting ) come first, then shot files can override parts of the global default. For example, a shot may change the position of characters in the shot. (The USD documentation often refers to this layering approach of files similar to layers in a PSD file – upper layers can hide/override the contents of lower layers.)

USD does not (at present) define a concept like the Unity Sequences package, where multiple shots can be defined in a hierarchy within a single project. USD uses a different set of files per shot. This keeps each shot consistently isolated, simplifying parallel work on different shots as well as different layers of a shot. Again, it is all about having multiple people work on a project where files can be allocated to individuals, simplifying the problem of merging changes by different people.

Who is using USD?

There are many products that support USD, such as  Maya, 3ds Max, Houdini, Katana, Nuke, Modo, Sketchup, Substance Designer, and Substance Painter. NVIDIA Omniverse has a nice list of products with USD support. Products of particular note for myself are:

• The NVIDIA Omniverse suite of tools uses USD as its native file format. (These tools are free for individuals!)
• Unreal Engine can read USD files directly from disk into a project.
• There is a Unity package for importing USD files as well.
• Even Apple is using USD.

When is USD useful for projects?

The real power of USD is its ability to handle complex projects with many developers working in parallel. By defining complex file loading and override semantics, each team can take ownership of a specific set of layer files (for example, all animation layer files) without concern of colliding with other team members.

It enables workflows such as starting with a crude mockup of characters initially, then iterative refinement of characters during the project (e.g., better quality facial expressions or hair physics) in parallel to the development of the rest of the movie.

USD supports rich markup and semantics and is being widely adopted as a standard supported by a range of tools. Due to this standardization, it makes it easier to keep a project independent of a specific technology. For example, creating assets in USD and then importing them into Unity allows the project to later switch to Unreal Engine or NVIDIA Omniverse for rendering without need to rebuild all the assets.

Another aspect that is interesting is using tools to create USD files. For example, can I define my own USD schemas that no rendering solution supports, but rather I feed into custom scripts I write that convert from one USD format to another. For example, I might define a USD schema for lines of dialog a character says, then use a program to read that file, send the text to a remote text-to-speech engine, and replace the file with a new format file that just references audio files. This could allow experimentation in mapping between the formats, or swapping text-to-speech engines.

So is USD suitable for my own projects?

I am not sure if I, as an individual creator, need all the complexity of layers that USD supports. Using USD also means I need tools that allow me to edit USD files. For example, how do I animate the position of a camera for a panning shot? If I do this inside Unity, I can use Cinemachine cameras which are not supported in USD at present. I would have to define my own custom schema to use this capability, and my own tools to create content using those schemas. Using a specific platform (like Unity or Unreal Engine) opens up the usage of all the native tools that the platform supports.

So for me the jury is out. Using something like Unity and its native tools feels less work. USD might be more interesting if I can both import into and export out of Unity. For example, can I use Unity’s Timeline for clip edit blends between animation clips, then export the result back to USD?

My main concern with USD is I use features in Unity that are not currently in USD. I mentioned Cinemachine cameras, but there are other features like HDRP Volumetric Clouds, which are not in USD. It begs the question of whether having a platform independent format is important enough for me. This becomes a question of how hard is it to define my own USD schemas for exporting custom information so I can roundtrip with Unity.

But there is no doubt USD is already a major standard, supported by a wide range of tools.

Want to read more about USD? The Houdini project has a nice introduction to USD. I found their description more readable than most. The openusd.org site is quite formal in comparison, and can therefore be harder to understand.