Prompt TTS for Better Pacing, Tone, and Readouts (Full Transcript)

[00:00:00] Speaker 1: Hey everyone, Mart here from Assembly and I just wanted to quickly go through some prompting tips that I have for our new text-to-speech model. We'll be covering the fundamentals of generating speech like pacing, tone and normalization and this video is going to be packed with tips but just to put it up front I also have in the description a markdown file full of all of the advice that will be in this video really for your agent. You can just send that over to your agent and it would be ready to go with all of these fundamentals that we'll go through but I did also want to go through them on the dashboard just to help you with your testing to find which voices you like and for what purpose. So first up we want to adjust pacing. Sometimes you want the agent to stutter over certain words or add pauses and silences in the middle of speech. So for starters what you should be able to do is just add some hyphens between the character. We go ahead and generate that.

[00:00:53] Speaker 2: What would you do if you could clone yourself? Maybe you'd pick up a new hobby?

[00:00:58] Speaker 1: Yeah there you go. Wonderful. And let's just change over to say Kyla. We could also add if we add three dots here we would get a pause.

[00:01:12] Speaker 3: What would you do if you could clone yourself? Maybe you'd pick up a new hobby?

[00:01:18] Speaker 1: Alright we couldn't hear it in that one. We can also just add maybe here.

[00:01:26] Speaker 3: What would you do if you could clone yourself? Maybe you'd pick up a new hobby?

[00:01:31] Speaker 2: So notice that paused right after it said you.

[00:01:34] Speaker 1: We can also add emphasis. We can emphasize you.

[00:01:44] Speaker 3: Maybe you'd pick up a new hobby?

[00:01:46] Speaker 1: And we can add another sentence maybe. Think about it and ending in an exclamation mark.

[00:01:51] Speaker 3: What would you do if you could clone yourself? Think about it. Maybe you'd pick up a new hobby?

[00:01:56] Speaker 1: So you can see how the voice kind of changes what it emphasizes on depending on the punctuation and the capitalization in the sentence. A question also ends in a different tone at the end of the sentence and we can add dashes, ellipses, capitalizations to kind of help with the pacing and the emphasis in the sentence. If you have any filler words that you want to add you'd have to spell it out and what we found is that it's enough to just have two letters. You don't have to write out the entire word.

[00:02:30] Speaker 3: What would you do if you could clone yourself? Think about it. Maybe you'd pick up a new hobby?

[00:02:36] Speaker 1: So right there I did H and M dot dot dot and it came out as like a hmm and you can do this for other filler words too like um and uh. You can also add a false start so if you put the word next to a hyphen and a space that would be a false start.

[00:02:54] Speaker 3: What what would you do if you could clone yourself?

[00:02:58] Speaker 1: Yeah so right there she said what twice the first one was a false start. That was lots of fun for a first demo but in the real world voice agents are more likely to be reading out phone numbers, addresses, order confirmation numbers and so on and so that's what this next section about normalization is all about. So right here you can see I added a currency amount at the start and then I also added in my email address. Let's see how the model handles that.

[00:03:25] Speaker 3: Your final amount will be $12.13. You can submit your proof of payment to martin at assemblyai.com.

[00:03:33] Speaker 1: So right away you can see because of that dollar sign $12.13 is spoken as $12.13 and my email address is always pronounced as you know martin at assemblyai.com. You can also play around with how dates are read out.

[00:03:53] Speaker 3: She was born on October 24th 2026 at 430 p.m. Her hospital was on 54 Davis Street.

[00:04:00] Speaker 1: So here we can see that it read out the date as October 24th 2026 and the time is usually formatted this way and the street address is always read out in full regardless of whether you put SC or street. These are just automatic ways that our models pronounce these key terms in a sentence. Let's check and see what happens if I reverse the dates and put it in like British format rather than American.

[00:04:31] Speaker 3: She was born on October 24th 2026 at 430 p.m. Her hospital was on 54 Davis Street.

[00:04:39] Speaker 1: So there even though I reversed the month and the day it stayed the same so maybe it changes if I change the accent. She was born on October 24th 2026.

[00:04:50] Speaker 4: She was born on October 24th 2026.

[00:04:54] Speaker 1: Yep so dates are always pronounced with the month first followed by the day. Common abbreviations like USA or MIT are spoken in the expected way.

[00:05:06] Speaker 4: I studied at MIT in the USA.

[00:05:09] Speaker 1: Whereas more uncommon abbreviations need a hyphen between letters.

[00:05:14] Speaker 4: Please meet me at the ETO.

[00:05:17] Speaker 1: Versus say if we remove the hyphens.

[00:05:20] Speaker 4: Please meet me at the ETO.

[00:05:22] Speaker 1: Yes so there we can see the hyphen in between the letters in the acronym causes more emphasis on the individual letters as well as a bit of a longer pause between them. So let's say your voice agent wants to read out a series of numbers. There are different ways that you can normalize and punctuate a number that will result in a different result.

[00:05:45] Speaker 4: So for example please call the emergency hotline at 1-800-64-645.

[00:05:51] Speaker 1: So there it read out the whole number and we can add hyphens to put the number in pairs instead.

[00:05:57] Speaker 4: Please call the emergency hotline at 1-800-646-45.

[00:06:04] Speaker 1: So there it automatically grouped up the numbers and read them in the individual groups. So if we just put 4321 the model will read out 4321. But to read this out as a number like 4,321 we can just add a comma in the number. So those are just some of the fun ways that we can play around with pacing, tone and normalization without text-to-speech model. I hope that was helpful for you in engineering voices that meet your needs and as usual if you have any questions don't be afraid to get in touch with our team. We'd be more than happy to help. Bye.