Extra Ordinary, the Series

Omniverse have announced Omniverse Cloud, “a fully managed platform-as-a-service offering for developing generative AI-enabled OpenUSD applications.” While full details are yet to be published, browsing the Omniverse documentation, they describe how to build microservices using Omniverse Kit (their application building framework) with support for cross service API calls and a message bus. Omniverse Enterprise is described as helping enterprises to “Develop Generative AI-Enabled 3D Tools and Applications for Industrial Digitalization”. This made me wonder what enterprise applications using OpenUSD and Omniverse might look like. This blog post is my blue sky thoughts.

What is Omniverse?

Omniverse is a 3D scene assembly and rendering software from NVIDIA. You can create 3D scenes (much like gaming platforms such as Unreal Engine and Unity) with high quality rendering. It is based on the OpenUSD format for representing such models. (It is not for creating models, like Blender.)

It then goes further with tools such as robotics simulation, allowing you to simulate robots in a virtual space. Beyond just simulation, this can be used to train AI models to teach the robots particular skills. Previously I have mentioned how Disney built custom robots to entertain guests at Disney parks, which were trained to walk inside Omniverse Isaac SIM. Design the robots, generate lots of virtual scenarios, use physics simulations for the robots to learn from, before building the first bit of actual robotic hardware. It greatly reduces costs.

Omniverse is focusing on the “industrial metaverse”, not a social or gaming metaverse. This includes domains like architecture, factory layouts, robotics, and simulation. Got a factory? Why not build a model and simulate your new production line before building the actual factory. Did you leave enough safe traveling zones for humans to move around the factory safely? Is there sufficient access to maintenance hatches on equipment to service it safely? With a digital twin, it is easier to explore such issues in software and identify problems before physical construction. This is the main differentiation between Omniverse and platforms such as Unreal Engine and Unity – they are targeting different problem spaces.

“But I thought NVIDIA was all about AI?” Obviously NVIDIA started first building GPUs for graphics acceleration. But Omniverse does fit into NVIDIA’s AI plans. It is a useful tool to generate synthetic data for training AI vision systems. Want a camera on your production line to spot damaged goods? Train a model with both real world and synthesized data.

What is OpenUSD?

OpenUSD is a format for capturing 3D assets. It originally came out of Pixar for creating animated movies. It can however be used for any 3D content. Scenes are constructed using a hierarchy of “prims” (primitive elements) where prims are identified by paths (like a file system). Each prim can have a schema type that defines the semantics of the prim and its properties. For example, a 3D object mesh can be defined by a prim with a “Mesh” prim type. Prim schema types are organized into an inheritance hierarchy, allowing customization and extension.

Example snippet of a text rendition of OpenUSD content from learnusd.github.io.

Note: General advice from the OpenUSD community is not to go crazy defining your own custom schema types. Start with defining the meaning of custom property names added to prims using existing schema types. Then define a schema once the new type is well understood and has proven to be valuable across multiple teams. Schemas are more commonly used for industry level standardization allowing consistency across tool vendors, but I can see custom schemas being a useful tool also for internal enterprise-wide standardization of important data types even if not in common practice today.

What makes USD more interesting than other 3D model formats is it does not only define the format of inside a single file – it also defines how files can be combined. Files can be split into sublayers where the final result is formed by flattening multiple layers with overrides. This allows, for example, customization of an existing model by defining changes to apply to the existing model. The original model file is not modified. If the original model is updated, the changes will flow through to all the other dependent models. OpenUSD includes well defined semantics for different ways to layer and reference other files, resulting in a lot of flexibility in how projects can be put together.

That is, OpenUSD provides basic schema types for 3D models, but also allows you store additional information in the content. It also provides a way for different contributors to take ownership for creating and maintaining different parts of the data, combining appropriate subsets into useful resources for use in different use cases. One team could be responsible for capturing the shape of 3D meshes, another responsible for adding textures to make them look attractive, another responsible for adding physics information, and another for performing simulations. This was important for Pixar as different teams contributed to different parts of a movie (character creation teams, rigging teams, animation teams, location creation teams, lighting teams, and so on.) The flexibility of OpenUSD makes multiple teams contributing to a larger initiative feasible for an organization.

Enterprise Data Modeling

The flexibility of OpenUSD raises interesting questions for an organization. How to model OpenUSD data for use across multiple enterprise applications?

It made me think about the evolution of microservices and their data. Originally relational databases were all the rage. Then along came NoSQL which brought about different ways of thinking about data to be persisted. OpenUSD is another novel format with new interesting questions about standardization – of prims and how to organize files. With OpenUSD, do you build a single shared scene that the whole organization uses? I suspect the correct answer is “no”.

With microservices, it is generally a best practice for microservices to communicate via well-defined interfaces, not the storage layer. This allows each microservice to pick the most appropriate persistence technology for that service, and then later change the storage representation in the future. As long as the external interface does not change, other systems in the organization do not care. I think the same is true for OpenUSD content.

A part of the reason for this is I mentioned that OpenUSD files can reference other files. This can be using relative paths or absolute paths. Those absolute paths can be formats such as a URL. This means any microservice can return USD content that builds upon USD content published by other microservices. In my mind, an OpenUSD microservice may use traditional inter-service communication patterns such as API calls and asynchronous message buses, but it may also publish its OpenUSD content with endpoint URLs that other services can depend on. Each service can then contribute to the greater OpenUSD graph of content, allowing upstream applications to combine (or omit) the contributions of different microservices for the particular need.

Versioning of published data may also come into the overall picture. One team may be working on creating new assets, sharing the assets between team members for creation and review. When completed, the team may publish the new data. This could be via a version management system (such as git, svn, etc) allowing references to either the “latest” version of content or to a specific previous version. Identifying specific versions allows the consumer of content to decide when it is appropriate to update the latest content.

Conclusions

OpenUSD is a flexible format that can capture hierarchical data. You can think of it a bit like an extended version of JSON in that there is a hierarchical nesting of content. But OpenUSD also defines schemas that define the semantics of the content (with predefined schemas for common 3D assets). The data types for properties are also richer than JSON (with types including matrices of numbers as primitive types). OpenUSD also has well defined rules for combining multiple files, meaning when data modeling you need to design not only the content of files, but conventions on how to structure files so they can be combined in useful ways.

Is this model supported by Omniverse? That is something I am still exploring and learning. They provide a framework for building applications out of extensions and then into microservices. Omniverse also includes Nucleus for storage of OpenUSD content. By having teams set up Nucleus servers per project (or even just different subdirectories for projects) the ownership of content management can be clearly delineated. Nucleus also supports versioning of content. So I think the answer is “yes”, but I would rather see a few successful implementations. 3D models are a more complex data format than other more traditional forms of data. I think it will take a bit longer for most enterprises to need projects that span departments.

But I do think OpenUSD is flexible enough to support enterprise use cases in ways other 3D data formats today cannot.