RealityKit – Playing multiple animations in USDZ file

Question

Has anyone found a workflow to create multiple animations for a skeletal mesh packaged in a USDZ file and playback the animations using RealityKit?

I have a skeletal mesh with two animations (idle & run). I want to package them into a single USDZ file (or even multiple USDZ files if I have to) to be used in RealityKit.

I have been able to create an FBX for export of my skeletal mesh and the animations, and ship them up to sketchfab for a valid USDZ export that RealityKit can understand. I do not know how to package the second animation into a single USDZ file and then use SWIFT to playback the specific animations based off of specific events.

There seem to be a lot of posts from about a year ago on the topic with no real answers and little activity since. Any pointers would be appreciated.

Answer 1

Storing and extracting multiple Model's animations in RealityKit

In RealityKit you have no possibility to play multiple animations from single .usdz model out-of-the-box. Look at this post and this post. There is just one animation accessible by default:

let robot = try ModelEntity.load(named: "drummer")
let anchor = AnchorEntity()
anchor.children.append(robot)
arView.scene.anchors.append(anchor)

robot.playAnimation(robot.availableAnimations[0].repeat(duration: .infinity), 
                                              transitionDuration: 0.5, 
                                                    startsPaused: false)

---

When you extract second or third elements from collection, your app crashes:

mainModel.availableAnimations[1]      // crash
mainModel.availableAnimations[2]      // crash

However, you can append second, third, etc. animations to the main .usdz file from other USDZs.

model_02.availableAnimations[0].store(in: mainModel)
model_03.availableAnimations[0].store(in: mainModel)

And here you can find one more elegant solution.

Answer 2

As of April 2023, in RealityKit 2.0 there are two ways to load multiple animations, as described in the WWDC 2022 talk: Dive into RealityKit 2, in order to bypass the limitation of just one AnimationResource per USD file.

Have one USD file per animation clip

The first way is to have one USD file per clip. We can load each USD as an entity and get its animations as AnimationResources. The AnimationResource can then be played on any entity, as long as the names of the joints in its skeleton match the animation.

Create multiple AnimationViews from the same AnimationResource

Another way to load multiple animation clips is to have them in a single USD on the same timeline and then use AnimationViews to slice this timeline into multiples clips.

This requires knowing the timecodes between each clip.

Each AnimationView can then be converted to an AnimationResource and used exactly the same way as the previous method.