Universal 3.5 Pro brings fast, multilingual pre-recorded STT (Full Transcript)

[00:00:00] Speaker 1: Last week, we introduced Universal 3.5 Pro Realtime. This week, we're excited to introduce that in the pre-recorded API. So for meeting note-takers, post-call analysis, and transcription workflows, the pre-recorded model and API is going to provide you the most accurate transcription across accent in English, and all of the languages that we now support out of the box. The model will code switch natively, and you can drive the transcription via contextual prompting. Let's walk through a few examples where we can see multilingual and code-switching in action. I'm going to switch the prompt to the code-switching prompt, transcribe this recording with multilingual speech. We'll drag in an audio file, English and French, and hit transcribe. So with pre-recorded, we're going to be sending an audio file, and get the transcription back within a couple of seconds.

[00:00:47] Speaker 2: I said something like, j'ai dit un message tout jeune. It's time to really pay attention to what the idea of code-switching is.

[00:00:58] Speaker 1: We can see here we've got the transcription back. It mixes English and French together, and we can see that the turnaround time was under 10 seconds. Very fast for the type of audio that we're going to be sending. Let's look at another example.

[00:01:11] Speaker 3: Mera rukne ja toh bahut maan hai, but I have an exam to give tomorrow.

[00:01:16] Speaker 1: Awesome. Let's see how that is transcribed by the model. So we can see here that this was just one turn, but the beginning of the turn is transcribed as Hindi, and the end of the turn is transcribed as English, matching the code-switching that the user is doing.

[00:01:29] Speaker 3: Mera rukne ja toh bahut maan hai, but I have an exam to give tomorrow.

[00:01:32] Speaker 1: Let's see one more example in action here. This one's going to be English and Mandarin.

[00:01:36] Speaker 4: Wo bu gong guo le. You can see the le at the end of the sentence indicates that the situation now, the current don't work. It's different from what it was before.

[00:01:51] Speaker 1: Amazing. So we can see the transcription we got back seamlessly mixes the accented English with the Mandarin. We can even see that the one word that they said between you can see the, and at the end of the sentence is properly identifying the character and transcribing that accordingly. This is pretty incredible for our model and allows truly global transcription products to be built and serviced around the world. Let's look at one more use case. Now, this is not multilingual, but it is contextual prompting. So if we listen to the audio.

[00:02:24] Speaker 4: In solo queue, I better leave.

[00:02:27] Speaker 1: Very hard to hear what that makes out. So let's see what it's transcribed without any prompt. In solo queue, I better leave. So that is definitely not what they are saying. So let's add a prompt. This is about League of Legends. League of Legends roles. We're giving the model context that this is about League of Legends roles, and we're going to transcribe this again. Great. So now let's play back the audio.

[00:02:52] Speaker 4: In solo queue, I ban Azir.

[00:02:55] Speaker 1: Right. So now the model is transcribing this. In solo queue, I ban Azir. Now, this is a common phrase and character in League of Legends, and the model is now accurately transcribing that with the context. The contextual prompting is way more powerful with this model. You don't need to be instructional in how you guide the model on formatting and punctuation. The model is going to infer and do that out of the box. But now the context you provide it is going to accurately steer it towards the types of conversations you're having and what those conversations are about. This will help identify the entities, the names, and hard-to-pronounce words that the model typically used to struggle with and that generally speech-to-text models struggle. This is what's going to make or break critical post-processing use cases across meetings, summaries, robust notes, and any downstream processes that you want to pass along from the transcription. We're really excited to put this new pre-recorded model in your hands. We can't wait to see the improvements that it provides for your products. Thanks.