Performance¶
What decides how much a FaceCue character costs, and what you can do about it. This is the one page that carries the cost picture, so the rest of the documentation points here rather than repeating it.
FaceCue is built to be cheap by default. A character with everything running is a small fraction of a frame on a desktop machine, and the cost is set by things you control: how the rig is built, how often the character is calculated and written, and how much of the face a distant or unseen character is still doing. The sections below take those in turn.
A Slow Editor Is Not a Slow Game
Everything on this page is what a character costs while your game runs. A FaceCue inspector is a large surface to draw, so an editor that feels heavy with a character selected is usually the inspector rather than the face. See The Editor Slows Down With a Character Selected.
Where the Cost Comes From¶
Calculating and Writing¶
A character costs in two ways. Calculating it, deciding what the face, the eyes and the head are doing, depends on what the character runs, the eyes most of all, and on how often it is calculated. Writing the result depends on how many blendshapes the rig moves and how often they are written. The writing is what makes one character cost more than another, since rigs differ far more than what runs on them. Each of those is yours to set.
The Rig Decides How Many Blendshapes Move¶
FaceCue writes a blendshape only when writing it would change the face, so the cost you pay is never wasted: every write is fidelity you can see on screen. A shape that has settled is left alone, and on a quiet character most of them have, which is why an idle crowd costs so much less than a talking one. What differs between characters is how much fidelity the rig offers. A high-fidelity figure such as a Daz Genesis 9 has close to 400 blendshapes assigned across all of FaceCue's channels, a Daz Genesis 3 has around 50, and a VRoid character around 25, and the first costs more than the last because it is doing more. That is a choice you make when you pick the rig, and it is the right choice for a character people will look at up close.
What you should not pay for is a blendshape that moves nothing. A prepared Daz or Character Creator figure often carries the entire blendshape set on every renderer, so a mouth blendshape exists on the boots as well as on the face, and most of those copies do nothing at all. The Blendshape Census on the brain finds them so they are never driven. Run it on every character that came through one of those pipelines. The live copies that remain, on hair, outfits and accessories, usually reach the face through the catch-all renderer group, and Auditing It for Performance covers trimming them.
Every Character After the First Costs Less¶
Characters share much of their work, so a crowd costs well under one character multiplied by its size. In our tests each character added after the first cost about half as much as the first, and eight cost about five times one, on a PC and on a Quest alike. The saving is larger when the characters share a skeleton and smaller when they do not, so a crowd of one rig is cheaper than the same crowd of eight different ones.
Spreading Across Cores
Every character is calculated on worker threads, in parallel with the main thread, and the result is written to the rig just before the engine skins the meshes. On a desktop machine 40 to 55% of what FaceCue would cost the main thread runs there instead, and on a standalone headset 55 to 75%. A crowd gains the most, since each character's work runs alongside the others rather than after them. Solve Schedule has the two rows that decide where the work runs and where in the frame it lands.
Emotion Is the One Part a Scene Can Grow Without Limit¶
Speech is naturally bounded, since a character speaks one line at a time. Emotion is not: a layered emotion moves the most blendshapes of anything on the face, two can play at once, and a residual mood keeps playing after its cue. On a busy scene it is the thing to ration, and the Emotion page says how.
For Scale
On one Windows machine, in a built player, a high-fidelity character comparable to a Daz Genesis 9, talking with its gaze tracking a moving target and its emotions playing, measured around 0.35 ms per frame, and 8 of them around 1.75 ms. Your rig, platform and scene all move those figures. They are here for a sense of proportion, not as a budget.
The proportions travel better than the figures. We measured the same scenes on a PC and on a Quest, and while the absolute numbers differed, how the cost divided up, and how it grew as characters were added, came out about the same on both. What you learn about a character's cost on your desktop holds on the headset.
The Two Rates¶
The largest levers are on the brain, under Facial Timestep, and they are set out on One Clock. The Calculation Rate is how often the character is calculated, and the Output Rate how often the result is written. Both default to 60, and both are per character, so a hero can keep higher rates while a crowd runs lower.
A lower output rate makes the character coarser and cheaper. In our tests 30 looked almost indistinguishable from higher rates, though how much you see will depend on your characters and your scene, and Display Smoothing keeps even lower rates smooth. For the calculation rate, we suggest keeping it around 60, and not above 120. Setting it to Output keeps the two together, so lowering the output lowers both.
Display Mode¶
On a headset, we recommend setting the output rate to Display instead of a number. The face then writes on a share of the game's own frame rate, every frame, every second frame or every fourth, chosen by the Display Divisor, so each write lands on a frame. Half is the usual choice. Display Smoothing keeps the motion smooth wherever the writes and the frames do not line up, and when the game's frames call for it, Clock Adaptation moves the rate on its own, for SpaceWarp's generated frames and for a game falling behind. What each of these does is set out under Display, Display Smoothing and Clock Adaptation on One Clock.
Solve Schedule¶
Its own section on the brain, under Facial Timestep and just above Optimization. Two rows decide where the character is calculated and where in the frame the result is written. The character looks the same whichever is picked. Solve is where the saving is: on Worker Threads the calculation runs on worker threads, in parallel with the main thread. Apply At, shown under Worker Threads, is where the result is written, and the later values leave the most room for that work and make the rig final later in the frame for a script that reads it.
| Solve | What it does |
|---|---|
| Auto | FaceCue picks. |
| Main Thread | Everything inside the character's own LateUpdate. |
| Worker Threads | Calculated on worker threads, in parallel with the main thread, and written where Apply At says. |
| Apply At | What it does |
|---|---|
| Auto | FaceCue picks. |
| At Character LateUpdate | Written in the character's own LateUpdate. |
| After Every LateUpdate | Written once the project's other late updates have run. Also where At PostLateUpdate lands when the engine's loop cannot carry its step. |
| At PostLateUpdate | Written early in Unity's PostLateUpdate phase, just before the engine prepares its meshes for drawing. |
If the GPU is what holds your frame rate down, moving FaceCue's work off the main thread will not raise it. What you gain is CPU time to spend elsewhere, on animation, physics, AI or the rest of your own code.
Under them the brain says when a character cannot use the value it was given, and why. A character whose Seed Mode is Shared runs on the main thread whatever Solve says, and the section shows it in amber. Below them, the Conflicts section finds other code that reads or writes the same bones and blendshapes at the wrong moment, measured while you play and predicted from the open scenes. Finding Conflicts explains it.
Optimization¶
The brain's Optimization section holds two policies that spend less on a character nobody is looking at. Visibility Freeze is on by default and Distance Rate Limiting is off, and both are per character.
Source is how the character decides whether it is being seen, and both policies read it:
| Source | Seen means |
|---|---|
| Renderers | Unity's own visibility flag on the character's meshes, from any camera. Free and exact, and it leans toward seen, since a shadow or reflection camera counts. |
| Main Camera | The main camera's view contains the character, with a margin. Also yields the distance the table below needs. |
| Custom Cameras | The same test against a list of cameras, for mirrors and second views. The camera that shows the face biggest gives the distance. |
| Animator | Unseen only when Unity would cull the body's animator. A body set to always animate is never unseen, and the face keeps going with it. |
Visibility Freeze¶
Enabled stops the face of a character that has gone unseen for Unseen Frames To Freeze frames in a row. Nothing is simulated and nothing is written, so the character costs essentially nothing, and it comes back the moment it is seen. A character within a metre of the camera is never frozen, whichever way the camera faces, so one standing right beside the player keeps moving. That holds on the Main Camera and Custom Cameras sources. It does not snap. It fades onto the body's own pose over the Freeze Blend on the clock, and fades back in from wherever the body left the head. A frozen character stops listening to its audio too, a microphone included, and picks the line up where the audio has got to when it comes back. What that looks like is on Working With Your Animation.
Distance Rate Limiting¶
Enabled spends less on a face the farther it is from the camera, through a table you can edit. Bands run outward from the camera, and each band has one cell for each thing the character can be doing: Speech Active, Gaze Tracking, and Idling, which is where listening lands. A cell says how often the face writes, as a fixed rate or as a share of the screen, and whether the character stops entirely out there. A share also has a Minimum, the slowest it may fall to on a slow screen.
Switching it on gives you a suggested table to start from, in writes a second:
| Up to | Speech Active | Gaze Tracking | Idling |
|---|---|---|---|
| 4 m | 60 | 60 | 60 |
| 12 m | 60 | 30 | 30 |
| 25 m | 30 | 15 | 15 |
| beyond | 15 | frozen | frozen |
Distances are counted through the camera's lens, as if the camera had Unity's default field of view, so a camera zoomed in on a far character counts it as near and keeps it at the nearer band's rate. A VR headset counts the real distance. The suggested cells are fixed rates, so each one costs and looks the same on every screen, while a share of the screen follows whatever screen the game runs on. The near band equals the default rates, so a character in front of the camera is untouched. The suggested table never freezes a speaking character, at any distance, because a voice can be heard from further than a face can be seen.
In our measurements a character standing around cost about a third less at 30 than at 60, and about half as much at 15.
Under the table, Floors are the slowest each part of the face still reads at, in writes per second. A cell never takes a part below its floor, so a distant character's blinks stay blinks.
Where to Spend¶
Three rules of thumb, each set out on its own page:
- Ration emotion. Give layered emotion and demeanors to the characters that carry the scene, and keep the crowd on Solid. See What It Costs.
- Keep Biomechanical for the characters people look at up close. Simple is cheaper, but not by enough to give up the fidelity on a lead. See Which Engine.
- Reach for the rates before anything else. They cut every driver at once, they are set per character, and an output rate of 30 still looks smooth. See The Two Rates.
The Live Paths¶
Baked cues are the cheapest way to move a mouth, because the work was done before the game ran. The two live modes add work at runtime, and only while a character is using them. A live mode a character has stopped using, such as one a fallback line started, stops reading the audio shortly after, instead of running on through the lines that follow.
Live Amplitude reads the loudness of the audio it is given, and it is close to free.
Live Viseme runs a small speech-recognition model on the device while it listens, and its cost depends on the device far more than the rest of FaceCue does. Which mode a character uses, and how it falls back, is on Speech.
Where Live Viseme Runs¶
Live Viseme can recognize the voice off the main thread on every platform it supports. A line recognized there takes little from the frame, and getting the recognizer ready for each line moves off the main thread with it, so a line starts without a hitch. This covers lines played from a clip. A microphone is recognized as its audio arrives.
Recognize On, under Live Viseme Setup in the Speech driver's Optimization section, decides where it runs:
- Auto, the default, recognizes in the background on Windows and on Android, Quest included, where stress testing showed it stable. On Intel Macs and iOS it stays on the main thread until it has been measured there.
- Background recognizes off the main thread on any platform.
- Main Thread recognizes as part of each frame on any platform.
Whichever it runs on, measure it on your own hardware before shipping a scene that depends on it, above all on a platform where every millisecond counts.
When Recognition Fails¶
Background recognition checks its own health. When it stops working, FaceCue restarts it and the Console says why, and a character whose background recognition fails again before a line finishes goes back to the main thread until it is loaded again.
Anything those checks do not catch falls to a last-resort safety net. A character whose voice keeps playing with no mouth shapes coming out has its recognizer restarted, and moves on down its Fallback Chain if that does not help. Either step logs an error with an FCR code, and we would like to hear about it.
Loading the Voice Model¶
Live Viseme's voice model takes a noticeable moment to load, once for the whole application, and whatever is on screen waits for it. Voice Model Warm-Up, in the Speech driver's Optimization section, decides when that moment comes. It is one setting for the whole project, so changing it on any character changes it for all of them.
- At Scene Start, the default, loads it as a scene starts that has a character able to use Live Viseme, so the first spoken line does not pay for it. A character counts when Live Viseme is its Speech Mode or is switched on in its Fallback Chain, which it is by default, so almost every scene with a FaceCue character loads the model.
- At Chosen Scene loads it as a scene you pick starts. A loading screen or a title menu suits this, anywhere a pause is already expected. If the scene is not in the build, the first scene loads it instead.
- At First Line waits until a character first speaks a line that needs it, and that line's opening moment pays for the load.
The Voice Model Stays in Memory
Once loaded, the model stays in memory for the rest of the run, about 16 to 20 MB. Where Live Viseme runs off the main thread, the load happens there too, so for most projects it is a reasonable price for live lines that start without a hitch. Whether it is worth it for yours is your call.
If you are sure a project never uses Live Viseme, switch it off in your characters' Fallback Chains, and no scene loads the model. Setting the warm-up to At First Line also holds it back, until a line first needs it.
From your own code. Two calls on FaceCueRecognizer:
public static void WarmUpModelInBackground()
public static void WarmUpModel()
WarmUpModelInBackground starts the load and lets the scene keep running. WarmUpModel waits until the model is loaded, which suits a loading bar that counts it as one of its steps. The model only ever loads once, so calling either again, or leaving the setting to load it too, costs nothing. The package includes an example, the Warm Up Voice Model On Load component, which does it as its scene starts.
Load It Where the Player Already Waits
The best moment for the load is one nobody notices, such as a loading screen or a title menu. Either pick At Chosen Scene and choose that scene, or load the model from your own loading sequence and leave the setting on At Scene Start as a fallback, so a scene reached some other way still loads it before anyone speaks.