Live¶
Driving the mouth from audio that was never baked: a player's microphone, voice chat, a line generated while the game runs, or a clip with no bake behind it. Two drive modes do it, Live Viseme and Live Amplitude, and this page covers both.
A live line is heard as it plays. Nothing tells FaceCue which words are coming, so the mouth is shaped from the sound itself. That is what separates it from a baked line, and it is why some of what Baked describes has no live equivalent.
What a Live Line Needs¶
Audio FaceCue can read. Both modes read the samples as they play, so the clip's Load Type has to hand them over. Decompress On Load does. Streaming does not, and the mode is passed over. See What Runs Where.
The character's own Audio Source. FaceCue follows the Audio Source set on the brain, the same as for a baked line. A voice played anywhere else is not heard.
For a microphone, something to capture it. The package includes an example, the Live Speaker Microphone component, which records the microphone into the character's Audio Source and tells FaceCue to follow it. Use it as it is, or as the starting point for push-to-talk or voice activity of your own. A browser build has no microphone of its own, so it needs a microphone solution you bring.
Live Viseme¶
Live Viseme recognizes the mouth shapes in the voice as it plays, and drives the character's visemes from them.
It plays the same viseme map as Baked Phoneme. Nothing extra is authored for it. A character set up for baked lines is already set up for Live Viseme, and each shape is moved whole, the same way.
Where the recognition comes from. Viseme Source, under Mode, picks it. Meta uses Meta's recognizer, which ships inside FaceCue and has builds for 64-bit Windows, Intel Macs, Android and iOS. On Linux, in a browser, and on an Apple silicon Mac outside Rosetta it cannot run, and the character moves on down its Fallback Chain. What Runs Where has the details, including what a Windows build requires. Manual takes a viseme source of your own instead.
What it keeps from a baked line:
- Shapes that run into each other. The recognizer blends one shape into the next, and FaceCue follows it.
- Loudness opens the mouth. A louder voice opens each shape further.
- The lips close where a word needs it, even while an expression is competing for the mouth. Use Labial Constriction Enforcement holds them closed, and it starts on.
- A base pose under each shape, when Use Minimum Spoken Pose is on, so quiet phonemes do not go slack. See Base Pose.
- Small living movement, so a held phoneme does not freeze. What it goes without. It has no delivery styles, and it has no reading of which phonemes were stressed. It is a very good mouth for speech nobody wrote down, and a baked line remains the more deliberate one.
The Mouth Leads the Sound¶
In real speech the mouth sets up each shape a moment before the sound comes out. On a line played from a clip, Live Viseme does the same: Use Anticipatory Offset, under Timing, starts each shape ahead of its sound by Offset (ms). It is on by default, so the character reads as saying the line rather than reacting to it. A microphone cannot be heard ahead of what it has picked up, so a microphone line has no lead.
Live Amplitude¶
Live Amplitude reads only how loud the voice is, with no recognition at all. It opens the mouth in proportion to the loudness, and the louder the voice, the further through its shapes the mouth goes.
It plays its own map. Instead of visemes, the amplitude map holds a set of recipes, each tied to a band of loudness, from a small opening on a quiet sound to a wide one on a loud sound. The OneClick Wizard fills it in, and The Amplitude Map sets out how to shape it.
It needs almost nothing. No bake, no model, no recognizer. It runs anywhere audio can be read, which includes the places Live Viseme cannot go, such as a browser build.
What you set. Master Gain, under the Live Amplitude settings, is the overall strength of the mouth. The silence threshold decides how quiet the voice has to be before the mouth settles closed.
What it goes without. It has no idea which phoneme is being made, so the lips never close for a word and no vowel gets its own shape. A mouth driven this way reads as talking rather than as saying words, which is exactly right for a crowd, a distant character, or a line where nothing better can run.
Choosing Between Them¶
- Live Viseme when the platform runs it and you want the mouth to show the words: a microphone in a desktop or mobile game, voice chat, generated speech.
- Live Amplitude when you want the least setup, when the platform cannot run Live Viseme, or for characters where reading the words does not matter.
The default Fallback Chain already puts them in that order, so a character asked for Live Viseme on a platform without it drops to Live Amplitude on its own.
The Rest of the Face¶
A live line moves more than the mouth. The head and the brows follow the voice as well, in their own live way, covered on the Emphasis & Prosody Live page.