Recognizers

The choices at the top of the baker that decide how your audio is read, and what each one does with it.

The Recognizer section is where a bake is actually configured. Everything below it is about where files go. The first control in it, Strategy, gates all the others, so start there.

Strategy

Three routes. Which ones are open to you depends on the language, and the picker only offers what is available.

Accurate runs the recognizer built for your language against your transcript. It is the only route that gets a language's own refinements, and it is the best result FaceCue produces. It needs one of the ten tuned languages. Choosing it reveals a Language picker and a Recognizer picker.

Multilingual (Rough) runs the multilingual model on its own, without those refinements. It is the same model the Accurate route starts from, so it is a real step above reading the audio blind, and it reaches far more languages. Quality varies from one to the next. Choosing it reveals a Mode picker instead.

Universal (Crude) reads mouth shapes straight from the audio, with nothing but the sound to go on. No transcript, no language, no model to download, and nothing it cannot take. The accuracy is crude and it always produces something. This option appears only when the OVRLipSync plugin is present, since that is what does the reading.

Which Route Covers Your Language

The first chip is the best Strategy available for that language. The other two say whether the baker can write the transcript or synthesize the audio, since both are offered from this same window and neither has the same coverage as the bake. Explained in full under Language Support.

Setting Up an Accurate Bake

Language is the language spoken in the recording, and it has to match the audio. It is not a preference, it selects the model that will be listening.

Recognizer picks how that language is read, and the options fall into two kinds.

A transcript decoder is given the words and only has to work out when each one was said. Half the problem is solved before it starts, which is why it is the more accurate of the two and the one to reach for whenever you have the script.

A free decoder is given audio and works out the words as well as the timing. Getting a word wrong means getting its mouth shapes wrong, so it is less exact, and in exchange it needs nothing from you at all. That is a genuine trade rather than a lesser option, and there is a case below where it is the better choice outright.

Next to the recognizer sits Setup, which opens the window where models are downloaded, and a Backend row covered in Backends. If a model is missing the baker says so here, and the bake stays disabled until it arrives.

Setting Up a Multilingual Bake

Multilingual replaces the language and recognizer pickers with one Mode control, carrying the same two kinds in a different shape.

Free decode reads the audio alone. It takes no language at all, which is the point of it.

Transcript refines that same reading against your script. More accurate and slower, and it needs a language as well as the text.

The Language Picker Detects for You

Transcript mode has to know which language it is reading, and making you say so would be tedious for exactly the projects that reach for this route. So the Language field detects it, reading the transcript text and not the audio, and re-detects whenever that text changes.

The field says	What it means
Auto - Japanese	It read the transcript and that is what it found
Auto - Pending	There is no transcript yet, so there is nothing to read
Japanese	You pinned it yourself, and detection is off

Click the field to search for a language and pin it, or to hand it back to detection.

Detection is per clip, not per batch. Point the baker at a folder of mixed-language recordings and each one is detected from its own transcript and baked in its own language, with no sorting first. The field shows the distinct languages found across the batch, with a count when there are more than it can fit.

Clips That Change Language

A single clip that switches language partway through is its own problem, and which route you are on decides whether it is a problem at all.

The routes that take no language handle it by construction. Universal and Multilingual Free decode are never told what they are listening to, so a line that starts in French and ends in Korean gets mouth movement throughout without being told anything. If you have code-switched dialogue and no scripts for it, free decode is not a fallback here, it is the straightforward answer.

The routes that take a language need telling. Accurate and Multilingual Transcript are set up for one language per clip, so a stretch in another one matches almost nothing the model knows. That does not produce wrong mouth movement, it produces none, which is far easier to miss in review than an error would be.

The fix is to mark the stretch in the transcript, which also gets you the recognizer built for that language and not merely some movement. Marking it is one of several things transcript tags do, and it is covered with them in Transcript Tags.

Generating a Transcript

If you have audio and no script, you do not have to settle for decoding free.

FaceCue can transcribe the recording first, and then bake with a transcript decoder. That puts you in the more accurate case, and it gets you something better besides: a script sitting in a field you can read and correct before the bake runs, rather than a decoder's guesses going straight into the phonemes where you will never see them.

Open Generate Transcript under the Transcript section. It offers its own model choice:

• Specialized is English only and about twice as fast, and produces the same English transcripts.
• Multilingual covers every supported language.

Its language can detect per clip too, so a mixed folder transcribes without sorting.

One Button, Three Behaviours

The big button at the bottom of the window reads what you have given it and changes accordingly.

With audio and a transcript it says Bake Phoneme Cues.

With audio and no transcript it says Generate Transcript & Bake, and does both in one go.

With a typed transcript and no audio it says Generate Audio & Bake, synthesizing the speech, assigning it, then baking it.

So an empty field is not an obstacle. Fill in either one and the button offers to fill in the other.

Which to Pick

Two words are worth pinning down first, because no language is unsupported and it would be easy to read this table as saying otherwise.

A tuned language is one of the ten with a recognizer of its own. A covered language is one the multilingual decoder reaches, which is over fifty more. Everything else still bakes, on the universal route.

Your situation	Route
Tuned language, script in hand	Accurate, transcript decoder
Tuned language, no script	Generate the transcript, then Accurate
Covered language, script in hand	Multilingual, Transcript mode
Covered language, no script	Multilingual, Free decode
Several languages in one clip, no scripts	Multilingual, Free decode
Neither list, or nothing else works	Universal