Animation in RealityKit

Jun 23, 2022

Basic Animation

To animate the rotation, scale or translation of an entity is quite straight forward. The Transform component has a move method. The code below moves an entity 0.5m along the X axis.

let transform = Transform(scale: .one, simd_quatf(), translation: [0.5, 0, 0])
entity.move(to: transform, relativeTo: entity, duration: 1, timingFunction: .easeInOut)

To do something at the end of the animation you add a subscription to the scene's publisher:

scene.publisher(for: AnimationEvents.PlaybackCompleted.self, on: entity).sink(receiveValue: { event in
    print("Animation finished")
})

Example App

You can download the code for the app here. When you tap on the cube, it will move in a random direction and change color. When the animation has finished, it will change back to its original color.

private var subscription: AnyCancellable?

private func moveBox(_ modelEntity: ModelEntity) {
    // If the animation is in progress another animation is not allowed to start
    guard subscription == nil else { return }

    // Set the color of the entity to blue at the start of the animation
    modelEntity.model?.materials = [SimpleMaterial(color: .blue, isMetallic: false)]

    // In the transform the translation is just affected so scale and rotation are set to their identity values
    let transform = Transform(scale: .one, rotation: simd_quatf(), translation: randomVector(length: 0.08))
    modelEntity.move(to: transform, relativeTo: modelEntity, duration: 1, timingFunction: .easeInOut)

    // The returned AnyCancellable from sink is stored in a member variable so when the method finishes, the subscription won't be deallocated and cancelled
    subscription = scene.publisher(for: AnimationEvents.PlaybackCompleted.self, on: modelEntity).sink(receiveValue: { [weak self] _ in
        // Set the color of the entity back to red at the end of the animation
        modelEntity.model?.materials = [SimpleMaterial(color: .red, isMetallic: false)]
        // Set subscription to nil so another animation can start
        self?.subscription = nil
    })
}

The subscription is stored in a member variable so that it doesn't get deallocated and cancelled when the method finishes. Once the subscription has returned a value, the member variable is set to nil. weak self is used in the closure to avoid a retain cycle. You generally use weak self in closures when the class instance holds a reference to the closure.

Importing Animations from Blender

You would normally do animations in a 3D software application like Blender. A lot of 3D file formats allow you to embed the animations. Blender allows you to have multiple animations using the Action Editor. For example you might have one animation for when the character is walking while another for when they jump. But due to a limitation in RealityKit it can only access the first animation. Blender can export USDA (text) and USDC (binary) but RealityKit is not able to load the animations from the files it exports. Reality Converter can import GLTF (text)/GLB (binary) and FBX files and convert them to USDZ though I found the animations weren't accurate with GLTF/GLB so FBX seems to be the best option. For Reality Converter to import FBX you need to install Autodesk's FBX SDK and FBX Python SDK. The latest version doesn't seem to work but version 2020.0.1 does. The packages don't seem to be notarized so you get a message when you try and install them. After double clicking on the package, open System Preferences > Security & Privacy and under the General tab at the bottom press Open Anyway. Create your animation in Blender, go to File > Export > FBX and make sure Bake Animation is checked. Open Reality Converter and import the FBX file and then export to USDZ. Drag the USDZ file onto the left inspector in XCode, making sure the target is checked under Add to targets.

In the code below a USDZ file is loaded. The animation was created in Blender, exported as a FBX file, imported into Reality Converter and then exported as a USDZ file. When the model loads the first animation is played. You can download the full code from here.

subscription = ModelEntity.loadAsync(named: "box").sink(receiveCompletion: { loadCompletion in
    switch loadCompletion {
    case .failure(let error):
        print("Unable to load model: \(error.localizedDescription)")
    case .finished:
        break
    }
}, receiveValue: { entity in
    anchorEntity.addChild(entity)
    let animation = entity.availableAnimations[0]
    entity.playAnimation(animation.repeat(duration: .infinity))
})

Animating Properties

To animate a property you use a concrete implementation of AnimationDefinition. From that you create an AnimationResource which you can then play on the entity e.g:

let animationDefinition = FromToByAnimation(to: Transform(translation: [0.1, 0, 0]), bindTarget: .transform)
let animationResource = try! AnimationResource.generate(with: animationDefinition)
entity.playAnimation(animationResource)

The property you animate is specified with the bindTarget. The options for bindTarget are:

.transform: Animate translation, rotation or scale
.parameter: Animate a parameter of the entity e.g. .parameter("someParameter")
.path(BindPath): You can target an entity in the scene graph e.g. .anchorEntity("anchor").entity("box").transform. This will animate the transform of the entity with name "box" who is a child of an anchor entity with name "anchor".
.jointTransforms: Animate the entity's joint transforms.

The property you animate needs to adopt the AnimatableData protocol. This includes Transform, JointTransforms, Float, Double, vectors (SIMD2, SIMD3, SIMD4) and quaternions (simd_quatf).

Another common property in the animation definition is blendLayer. It will apply animations in ascending order of this property. So if two animation definitions animate the same property the definition with the higher blendLayer value will overwrite the one with the lower value.

The concrete implementations of AnimationDefinition are:

FromToByAnimation: Animates a property by specifying the initial value, the final value or an amount to change by.
SampledAnimation: You can specify an array of values and it will animate between them. It's kind of like keyframing.
OrbitAnimation: Animates an entity around the point 0, 0, 0.
BlendTreeAnimation: Blends two or more animations.
AnimationView: Creates a variation of an existing animation definition. For example you can delay or change the speed of the animation.
AnimationGroup: Runs multiple animation definitions at the same time.

Below are examples of each of the above:

FromToByAnimation

Moves an entity 0.1m along the X axis.

let animationDefinition = FromToByAnimation(to: Transform(translation: [0.1, 0, 0]), duration: 1.0, bindTarget: .transform)

let animationResource = try! AnimationResource.generate(with: animationDefinition)
entity.playAnimation(animationResource)

SampledAnimation

Moves an entity to different random positions.

var transforms:[Transform] = []
for _ in 1...10 {
    transforms.append(Transform(translation: [Float.random(in: -0.1..<0.1), 0, Float.random(in: -0.1..<0.1)]))
}
let animationDefinition = SampledAnimation(frames: transforms, frameInterval: 0.75, bindTarget: .transform)

let animationResource = try! AnimationResource.generate(with: animationDefinition)
entity.playAnimation(animationResource)

OrbitAnimation

Orbits an entity around the origin. You need to move the entity away from the origin with startTransform parameter for this to work.

let animationDefinition = OrbitAnimation(duration: 2, axis: [0, 1, 0], startTransform: Transform(translation: [0.1, 0, 0]), bindTarget: .transform)

let animationResource = try! AnimationResource.generate(with: animationDefinition)
entity.playAnimation(animationResource)

BlendTreeAnimation

Blends two animations by different weights. One animations moves the entity 0.2m along the X axis while the other moves it -0.1m along the Z axis. Unlike FromToByAnimation and SampledAnimation you need to specify the type of the value you want to animate after the class e.g. BlendTreeAnimation<Transform> where as with the other two, Swift can guess the type from the bindTarget parameter.

let animationDefinition1 = FromToByAnimation(to: Transform(translation: [0.2, 0, 0]), duration: 1.0, bindTarget: .transform)
let animationDefinition2 = FromToByAnimation(to: Transform(translation: [0, 0, -0.1]), duration: 1.0, bindTarget: .transform)
let blendTreeDefinition = BlendTreeAnimation<Transform>(
    BlendTreeBlendNode(sources: [
        BlendTreeSourceNode(source: animationDefinition1, weight: .value(0.25)),
        BlendTreeSourceNode(source: animationDefinition2, weight: .value(0.75))
    ])
)

let animationResource = try! AnimationResource.generate(with: blendTreeDefinition)
entity.playAnimation(animationResource)

AnimationView

Delays an animation by 1 second and halves the speed.

let animationDefinition = FromToByAnimation(to: Transform(translation: [0.1, 0, 0]), duration: 1.0, bindTarget: .transform)
let animationViewDefinition = AnimationView(source: animationDefinition, delay: 1, speed: 0.5)

let animationResource = try! AnimationResource.generate(with: animationViewDefinition)
entity.playAnimation(animationResource)

AnimationGroup

Moves two entities in opposite directions along the X axis.

let animationDefinition1 = FromToByAnimation(to: Transform(translation: [0.1, 0, 0]), bindTarget: .transform)
let animationDefinition2 = FromToByAnimation(to: Transform(translation: [0, 0, -0.1]), bindTarget: .anchorEntity("anchor").entity("blueBox").transform)
let animationGroupDefinition = AnimationGroup(group: [animationDefinition1, animationDefinition2])

let animationResource = try! AnimationResource.generate(with: animationGroupDefinition)
entity.playAnimation(animationResource)

I couldn't find a parameter other than transform to animate (There doesn't seem to be a way to animate material properties). So I animated the transform property of another entity. I've create a demo app that demonstrates each of these classes. Just tap on Animation Types at the bottom of the screen to run an animation.