Emotion in Four Ways¶

The same emotion can reach a character four ways, and the one to use depends on where you are working and what decides the moment.

A line you have written carries its emotions with it, baked onto the clip, so they play the same way every time the line plays. A scene you are cutting in Unity's Timeline takes them as clips on a track, beside the audio and the camera. A moment the game decides, a flinch at an explosion or a character growing angrier through an argument, comes from your own code. And the mood a character sits in while it waits comes from a demeanor, which needs no moment at all.

However it arrives, an emotion behaves the same way on the face. Emotions from different routes play together rather than replacing each other, so a line's own emotion, a Timeline clip and a call from your code can all be on the character at once, combined by the rules in Overlapping Emotions.

Baked Onto a Clip¶

The usual route. An emotion baked with a line travels with it, plays back the same way every time, and needs nothing at runtime beyond the clip itself. There are two ways to put one there, and they meet in the middle:

From the transcript. Wrap the words in a tag, <Proud>…</Proud>, and the baker places the emotion over those words with a sensible envelope. See Emotion Tags.
In the Cue Clip Composer. Place and shape it by hand against the waveform, or open it and type exact times. See Emotion Cues.

A tag produces an ordinary, editable cue, so a line tagged in the script can be reshaped in the composer afterwards, and a cue placed in the composer looks no different from one that came from a tag.

On a Timeline Track¶

For a scene you are already cutting in Unity's Timeline, an emotion track puts the same cues on the sequencer, where they sit beside the audio and the camera. See Timeline.

From Your Own Code¶

For anything driven by what is happening in the game, the Emotion driver on the character takes calls directly. Play plays one emotion. PlayEmotions plays a whole set of emotion cues, the same kind a bake writes. Both hand back a handle, which keeps your code in charge of what it started.

Playing One Emotion¶

public IEmotionPlayHandle Play(
    string emotion,
    float intensity = 1f,
    float durHold = -1f,
    float rampUp = -1f,
    float rampDown = -1f,
    EmotionDriver.BlendOverride emotionBlend = EmotionDriver.BlendOverride.Inherit,
    EmotionDriver.BlendOverride repeatBlend = EmotionDriver.BlendOverride.Inherit,
    bool clearResidualMood = true,
    bool leaveResidualMood = false)

Play plays one emotion as a one-shot: it rises, holds, falls away and cleans up after itself. It is layered over whatever else the character is doing, so a character can flinch at an explosion halfway through a line without the line's own emotions stopping.

Parameter	What it does
`emotion`	The emotion to play, by a name the library knows it by, in any casing.
`intensity`	How strongly the character feels it, from 0 to 1.
`durHold`	How long it holds at full once it has risen, in seconds. Leave it out for a hold of three seconds.
`rampUp`	How long it takes to rise, in seconds. Leave it out for the emotion's own rise.
`rampDown`	How long it takes to fall away, in seconds. Leave it out for the emotion's own fall.
`emotionBlend`, `repeatBlend`	How it meets other emotions, in place of the character's own settings. See Choosing How It Meets Others.
`clearResidualMood`	Clears a residual mood still on the face, as a new line does. Pass `false` to play over the mood and leave it standing.
`leaveResidualMood`	Leaves a residual mood of its own when it ends, as a line does. Off unless you pass `true`, since a moment in the game is not usually a mood the character carries on with.

var emotion = GetComponent<EmotionDriver>();
var handle = emotion.Play("Surprised", intensity: 0.8f);

A name the library does not carry plays nothing: Play returns null and says so in the Console, so check the handle before you use it.

Playing a Set of Cues¶

public IEmotionCuesHandle PlayEmotions(
    EmotionCueData emotionCues,
    IAudioTimeSource timeSource = null,
    EmotionDriver.BlendOverride emotionBlend = EmotionDriver.BlendOverride.Inherit,
    EmotionDriver.BlendOverride repeatBlend = EmotionDriver.BlendOverride.Inherit,
    bool clearResidualMood = true,
    bool leaveResidualMood = false)

PlayEmotions plays a whole set of emotion cues at once, each with its own start, end and intensity, the way the emotions baked onto a line play. It is the route for emotion you load or build yourself.

Parameter	What it does
`emotionCues`	The cues to play. `EmotionCueData.Load(string json)` reads them from JSON, in the same format the baker writes.
`timeSource`	Where the set gets its time from. Leave it out and the cue times count from the moment you call, so a cue starting at two seconds starts two seconds later. Pass one to set the time yourself, see Setting the Time Yourself.
`emotionBlend`, `repeatBlend`	As for `Play`.
`clearResidualMood`, `leaveResidualMood`	As for `Play`. A set clears the mood from the call, even when its first cue comes later.

Left to itself, a set ends on its own once its last cue has faded, the way a one-shot does. A gap between two cues, or a first cue that starts late, does not end it early. Release it to cut it short: the cues already under way finish their fall, and any still to come do not start.

From a Baked File¶

Load the emotion .json the baker wrote beside a line, and play it. StopReacting shows the release that cuts it short.

[SerializeField] EmotionDriver emotion;
[SerializeField] TextAsset reactionCues;   // an emotion .json the baker wrote

IEmotionCuesHandle _reaction;

public void React()
{
    var cues = EmotionCueData.Load(reactionCues.text);
    _reaction = emotion.PlayEmotions(cues);
}

public void StopReacting() => _reaction?.Release();

Building Cues in Code¶

The cues do not have to come from a bake. Fill an EmotionCueDataRaw with EmotionCue entries and pass it through the same loader. Times are in seconds from the start of the set, and a cue given no rise or fall takes the emotion's own.

var raw = new EmotionCueDataRaw
{
    cues = new[]
    {
        new EmotionCue { emotion = "Worried",   start = 0f, end = 2.5f, intensity = 0.6f },
        new EmotionCue { emotion = "Surprised", start = 2f, end = 3f,   intensity = 1f },
    }
};
_reaction = emotion.PlayEmotions(EmotionCueData.Load(JsonUtility.ToJson(raw)));

Setting the Time Yourself¶

Left to itself, a set starts the moment you call PlayEmotions and plays through at normal speed. To decide where it is yourself, pass a SettableAudioTimeSource as the timeSource, and set its AudioTime to the moment you want, in seconds from the start of the set. The set shows whatever moment you give it. The usual reason is audio you play yourself: give it the audio's position, and the emotions stay with the voice even when the audio is slowed down or skipped.

A set on a time source plays until you release it. It cannot end on its own, because it never knows the time has run out: you might move it back to any earlier moment, and its cues would play again from there. So release it when you are done with it, as the example below does when the voice stops.

Until then, set the time on every frame, even on frames where it has not changed. A time left unset for half a second is taken to mean the set was forgotten, and it fades out with a warning in the Console.

[SerializeField] EmotionDriver emotion;
[SerializeField] AudioSource voice;

readonly SettableAudioTimeSource _time = new SettableAudioTimeSource();
IEmotionCuesHandle _line;

public void Speak(EmotionCueData cues)
{
    voice.Play();
    _line = emotion.PlayEmotions(cues, _time);
}

void Update()
{
    if (_line == null) return;

    // the line has finished, let its emotions go
    if (!voice.isPlaying)
    {
        _line.Release();
        _line = null;
        return;
    }

    _time.AudioTime = voice.time;   // every frame, even when it has not moved
}

The Handle¶

Play hands back an IEmotionPlayHandle, and PlayEmotions an IEmotionCuesHandle. Both have these in common:

Member	What it does
`bool IsActive { get; }`	Whether it is still playing. False once it has ended on its own, and from the moment you release it, even while it is still falling away.
`event Action<IDriveHandle> Finished`	Raised once, when the emotion has left the face, whatever ended it. The handle is passed to your handler.
`void LeaveResidualMood(bool leaveResidualMood)`	Whether it leaves a residual mood behind when it ends. Starts as the call's `leaveResidualMood`.
`void Release()`	Lets it go gracefully. It falls away along its own fall from wherever it is, rather than cutting.

On top of those, each has one control of its own, because the two play different things:

Member	What it does
`void SetIntensity(float intensity)`, on the handle from `Play`	How strongly the character feels the emotion, from 0 to 1, changed while it plays without starting it over.
`void SetWeight(float weight)`, on the handle from `PlayEmotions`	How much of the set reaches the face, from 0 to 1, across every cue in it. Each cue keeps its own intensity, which makes this the control for fading a set in or out.

A set has no single intensity to change, since each of its cues carries its own, so it takes a weight instead.

Knowing When It Is Over¶

IsActive and Finished answer two different questions. IsActive says whether the emotion is still yours to change, and it goes false the moment you release it. Finished says the emotion has actually left the face, which after a release is once its fall is over. To start something when an emotion is done, start it from Finished:

var surprise = emotion.Play("Surprised", intensity: 0.8f);
surprise.Finished += _ => emotion.Play("Amused", intensity: 0.5f);

Finished is raised from the character's own update, never from inside the call that ended the emotion. So subscribing straight after Play never misses it, and a handler can safely play or release anything. It is not raised if the character is destroyed first, or if its Emotion driver is switched off, which ends every emotion on it at once. IsActive going false is the signal then.

The handle outlives the emotion. Once it has finished, the handle does not become null. It stays yours, IsActive reads false, and anything you call on it does nothing. To play the emotion again, call Play again for a new handle.

LeaveResidualMood can change until the emotion starts to fall away, which is when the mood takes over, as it does after a line. A one-shot starts to fall the moment you release it, so turn it on before Release.

using Firecrest.Systems.FaceCue;
using UnityEngine;

public class Argument : MonoBehaviour
{
    [SerializeField] EmotionDriver emotion;

    IEmotionPlayHandle _anger;

    // the argument starts, and Angry holds for as long as it lasts
    public void Begin() => _anger = emotion.Play("Angry", intensity: 0.3f, durHold: 30f);

    // each insult pushes it further, without starting it over
    public void Escalate(float level) => _anger?.SetIntensity(level);

    // someone apologises: the anger falls away, and leaves a trace behind
    public void Settle()
    {
        if (_anger == null || !_anger.IsActive) return;
        _anger.LeaveResidualMood(true);
        _anger.Release();
    }
}

Choosing How It Meets Others¶

Either call can also say how its emotion meets others on the face, in place of the character's own blend settings. Code has no clip to draw a crossfade on, so this is how a moment fired from the game asks to carry across, or to dip. For a set, the repeat choice also decides how the set's own cues combine where it holds the same emotion twice. Leave it out and the character's settings decide. See Overlapping Emotions.

emotion.Play("Happy", intensity: 0.6f, emotionBlend: EmotionDriver.BlendOverride.RequestedMax);

As a Demeanor¶

The one route with no moment attached. A demeanor is a palette the character drifts through while it waits, and there is one for idling, one for listening and one for awaiting a dialogue choice. They are set on the character and play underneath everything above. See Demeanors, and Speaking & Listening for where the character's state comes from.