Speaking and Listening¶
How FaceCue knows where a character sits in a conversation, what changes when it does, and how you tell it.
This is a small amount of machinery with a large effect, because a great deal of the performance keys off one value.
Two Components, Deliberately Symmetric¶
The speaker side watches audio playback. It is what knows a clip is playing, which clip it is, and how loud it is at this instant, and it is where cues are resolved from a registry. If a character speaks, it needs this.
The listener side watches the character's own state instead. It is the route by which something that is not audio, a game event or a change in the world, reaches the face.
They are built as mirrors of each other on purpose. One sources what the character is doing from sound, the other from state, and everything downstream takes them the same way.
Both are required peers of the brain, and Unity adds them with it. Even a character that will never be spoken to carries the listener side, because the state it resolves has to come from somewhere whether or not anything is currently driving it.
Conversant State¶
Everything above resolves to one value: Talking, Listening, or Idle.
That single value is read right across FaceCue, and it changes more than you would expect.
The eyes carry most of it. How often the character blinks, how often its gaze breaks away and how far it travels when it does, and how much the head sways underneath. Breath shapes its cycle differently while listening. Nothing else reads the state directly.
The gaze part is the one worth understanding, because it is not a single dial turned up and down.
An idle character makes small movements very often, the constant scanning of eyes with nothing in particular to hold. A character in a conversation makes larger movements much more rarely, breaking away from a face and coming back. So idle eyes move most and travel least, and a talking character breaks away roughly twice as often as a listening one.
Listening Is Not Quiet Talking
The three states are genuinely different behaviours, not three amounts of one behaviour.
Blink rate is the clearest case. Talking raises it. Listening deliberately does not change it at all, because the evidence says attention does not move it. Somebody went looking and found nothing there.
What listening does change is stillness. People hold noticeably still when they are paying attention, so the sway is gated right down and the eyes break away far less. A listening character reads as attentive instead of as one waiting for its turn.
Connecting Your Own Game¶
FaceCue ships the interface, not the game state. It has no idea who is talking to whom, and no opinion about how your dialogue system works.
What it offers is a small interface to implement on whatever you already have. Tell it who is speaking and who is being spoken to, and the state above resolves itself.
Conversation is one of five things your game can tell a character, alongside what it is looking at, how hard it is working, and how bright the scene is. All of them work the same way, and they are set out in World Inputs.
Until you wire any of it up, a character that plays audio still resolves to Talking on its own. That is enough for a great deal, and it is why a character set up with the wizard already behaves sensibly before you have written any integration code. What you gain by wiring it up is Listening, which is the state a character cannot infer from its own audio, because nothing is playing.
Where the Value Goes¶
Worth knowing, because it explains behaviour that otherwise looks like it came from nowhere.
- Gaze takes almost all of it: how often the eyes break away and how far, how often the character blinks, and how much the head sways.
- Breath builds a different cycle while the character is listening.
None of those are things you switch on. They follow from the state being right, which is why wiring the interface up is worth doing even though the character works without it.