Recognize Speech¶
A transcript from a recording. The same recognition the baker offers inline, with the tidying and the safety checks laid out where you can see them.
You want a transcript for one reason above all others: a bake with one is more accurate than a bake without one. The baker can do this for you as part of a bake, and this tab is where you do it deliberately, over a whole folder, and look at the result before anything is baked from it.
Input¶
Single takes one clip, either an AudioClip from your project or a path to a file on disk. Batch takes a folder, and skips clips that already have a transcript, so running it again after adding recordings only does the new ones.
Language and Model¶
Two controls that belong together: which language the clip is in, and which model listens to it.
Language¶
Pick the language, or leave it on Auto to have it detected.
Auto is reliable on a clean recording of one language. It is worth naming the language when you know it, because a short clip gives the detector little to go on, and a wrong guess produces a transcript that looks plausible and is not.
Recognition covers a wide set of languages, wider than baking does. Check yours in Language Support.
Strategy¶
Two models ship, and Strategy chooses between them.
| Strategy | Covers | Speed |
|---|---|---|
| Multilingual | Every supported language | The baseline |
| Specialized | English only | About twice as fast |
They produce the same English transcripts. Specialized is not a lower-quality shortcut, it is a model trained for one language and therefore smaller and quicker at it. If your project is English and you are transcribing a lot, the difference is worth having.
Multilingual is what a new install uses, because it is the one that works whatever language you point it at. Specialized is a download, offered in the Setup window alongside the other models.
FaceCue Offers the Swap When It Would Help
Set the language to English on the multilingual model and the window says so, and offers either the switch or the download.
It only appears when you have named English yourself. On Auto the language is not settled until the clip has been listened to, so there is nothing to suggest yet.
Cross-Language¶
Some lines change language partway through. A character switches into French for a phrase, or a name is said the way its owner says it. Read with one language's model, the stretch in the other language matches almost nothing that model knows, and it comes out with the mouth barely moving over it.
Cross-Language is the first section of the tab's settings pane, and it has the transcript pass listen for the change. Switched on, each clip is checked for a stretch in another language, every stretch is read with the model for its own language, and the pieces are joined back into one transcript. A clip that never changes language comes back exactly as it would have anyway.
It is off until you ask for it, because it adds a pass over the audio and a line that stays in one language has nothing for it to find. It also needs its own model, the Cross-Language Detection pack, offered in the Setup window like the others. Switch the feature on without it and the tab says so and offers the download. Transcripts still generate in the meantime, read in the single language chosen above.
Sensitivity¶
How short a stretch of the other language to look for.
| Sensitivity | Finds | Trade |
|---|---|---|
| Clause | A whole clause or sentence in another language | The surer of the two. On a line that never changes language it stays quiet |
| Phrase | Shorter stretches too, down to about a second | Occasionally marks one on a line that did not change language |
Neither finds a single word. A word is too little sound to name a language from, and a word or two of another language is usually better handled as its own clip.
Write Tags¶
Whether what was found is marked in the transcript.
It is on, and it is the point of the pass. The marks are the same language spans you would write by hand, <French>…</French> around the stretch, so the bake reads each part with the right voice and you can see, check and correct where the change was placed. Turn it off and the pass still reads each part correctly, but the transcript keeps no record of where the language changed.
What You Get Without Touching Anything Else
A tagged transcript is all the bake needs. The baker's own Language Spans handling takes over from there, and its default reads every stretch with the recognizer built for its language, which is the most accurate path there is. A recording that mixes languages goes from folder to performance with nothing marked up by hand.
When a change is found, the console names the clip, the language and the seconds it spans, so a batch can be checked from the log alone.
Polish¶
Three things happen between the model's raw output and the file, and each can be turned off.
Text Cleanup¶
Speech models produce text that reads oddly: spacing before punctuation, expanded contractions, inconsistent casing.
Tidying is per language, because what counts as tidy is. Some languages have rules FaceCue knows and applies, and the rest are left exactly as heard. Applying one language's conventions to another would damage it, so a language without rules gets none.
Auto uses the rules for the language when there are any. Forced On applies them whatever the language, and Forced Off keeps the transcript exactly as the model produced it.
Long Clips¶
A clip too long for the model to take in one pass is transcribed in pieces.
The split looks for a pause instead of cutting at a fixed length, so no word is broken in half. Shorter pieces give it more places to choose from, at the cost of more passes.
Skipping Bad Pieces¶
Speech models produce confident nonsense when given something they cannot transcribe. A piece that is silent, that comes back as the same phrase over and over, or that the model itself is unsure about is dropped instead of written into the transcript.
That filtering is on, and it works piece by piece. The rest of the clip is unaffected.
Rejecting an Invented Transcript¶
There is a stronger guard, and it is off until you ask for it.
Speech models are trained largely on subtitled video, so faced with silence they fall back on the lines that fill subtitles: "Thanks for watching" and its relatives. The whole transcript is a sentence the model has learned rather than anything in the audio, and no per-piece filter catches it, because nothing about the sentence looks broken.
Reject Invented Transcripts checks the finished transcript for those and throws the whole clip out when it finds one. A rejection is reported in the console with the text that was thrown away, so you can check the call was right.
A Rejected Clip Is Left Blank on Purpose
A rejected clip gets no transcript at all, rather than one nobody trusts. That is the useful outcome: an empty transcript sends the baker down its free-decode route, which reads the audio directly and is the right treatment for a clip whose words could not be established.
A blank row in a batch is therefore a result, not a failure.
Output¶
A .txt beside the clip, or rows written into a CSV index with clip_id, an optional language, and the transcript. The CSV is the one to use for a batch, because it keeps a folder of dialogue in one file you can read, edit and hand to somebody else.
Either way the transcript pairs with its clip by name, which is the same rule the baker uses to find it. See Recognizers.
FaceCue keeps a number of backup versions when it rewrites a transcript, so an accidental re-run over hand-corrected text is recoverable.
Hardware¶
The tab runs on whichever execution provider the baker is set to, so a machine set up for GPU baking transcribes on the GPU too. See Backends.