Speaker 1: We push the button, we push the button. We are alive. We push the button. Hello. We made it. I need to go check everywhere that it actually made it. I'm excited for the stream today. We're going to do fun things.

Speaker 2: This feels like a normal insert day of the week for me here.

Speaker 1: I'm like, we're basically doing what we do offline. This is what we do online. Yes, we we we co-work through this to make sure we get this done. Sabin, what up?

Speaker 2: Sabin, what's going on? Hope I'm saying that right.

Speaker 1: I hope I'm saying that right, too. If I'm not, Sabin, correct me. I don't know if we've ever like, Sabin. I'm really bad with names.

Speaker 2: OK, Sabin, got it right.

Speaker 1: Cool. Yes. OK, well, we are going to work on. Transcribing stuff and then what to do with transcriptions for a talk I'm giving in Melbourne. Yeah, in. Two, two and a half weeks.

Speaker 2: Yeah.

Speaker 1: Luckily, I've given the talk before, but I don't want to do it about your content this time. I'd rather do it about teach gen tech. Works for me.

Speaker 2: So I am I am just excited to see that work that I have done is migrating and moving into something different, which is good. Like, that's a good thing. So I feel like if you're just copy and pasting all the stuff that I'm doing, that doesn't I mean, it's nice, but it's not like great.

Speaker 1: Yeah, I would say it was doing this and what up, Damien? Doing this really helped in. Being able to learn and build off of someone else's content to be able to make yourself is probably one of the best ways I think I've learned because of open source.

Speaker 2: Yeah, it was this.

Speaker 1: This all came about of, hey, Jen, you have to give a talk at this like meet up. Give a talk on this on this workshop. And I was like, oh, OK, let's go. Let's do this. Let's do this. And I gave the talk and I felt pretty comfortable doing the talk. And I got to learn a lot more about open search and rag patterns and link chain, but I didn't get to do all the transcripts. And I was like, that is part of the process that I want to understand more. So, yeah, that's what we're doing.

Speaker 2: So the one downside to doing all the transcripts, though, is they just take forever, which is I'm not going to lie. So there's a there's a credit on this repo and the credit goes to the person who did the original transcriptions for me because I was just like, I have, you know, 80 episodes at this point and doing one is fine, but I don't want to I don't want to like just sit there and run it. And then do you do like all 80 at one time?

Speaker 1: I'm going to be that person. I have like 100 episodes.

Speaker 2: Okay. So what I would suggest is for the stream, don't try to do all 100 at the same time.

Speaker 1: Wasn't planning on it.

Speaker 2: That's good.

Speaker 1: That's because do I think this is a great follow-up question. Do I have to download all of the videos or can I use it off of like a YouTube link?

Speaker 2: Oh dear. You're doing it from video.

Speaker 1: Which I mean, I can do it from audio. It's just that causes more steps, you know? So I just have to think about those steps too.

Speaker 2: It's designed to work from audio. So that's kind of important. Luckily, I have, you have the RSS feed for your podcast, right? Well.

Speaker 1: Well, it's on YouTube. It's technically not a podcast. It's technically on YouTube. They're YouTube videos.

Speaker 3: Okay.

Speaker 1: So, but I don't, I don't know. Let's, let's go explore. I will share my screen and see.

Speaker 2: You can, you can technically make any YouTube playlist a podcast now.

Speaker 1: So this is my YouTube.

Speaker 2: Just showing everybody all the things.

Speaker 1: Yeah, I'll show you all the numbers. I don't, I don't mind. They're improving. You can, if anybody was curious. They, all the numbers are going up. And the analytics are, we're getting there.

Speaker 2: Hashtag growth.

Speaker 1: Hashtag growth. Let's go. Go subscribe to my YouTube. Not what I was going for.

Speaker 2: But do it anyway, though.

Speaker 1: Yes. Yes, please do. Please do. Lerner. Hello.

Speaker 3: Hey, Lerner.

Speaker 1: What's up? 12.30 in the morning. You're up late. Okay, what am I doing? We are going to content.

Speaker 2: So like we ultimately we have a couple of options. Like you're gonna have to get this into audio form somehow. I'm sure something like a YouTube DL could also work where you do like a YouTube downloader and then you just download the. Save and it's on Twitch. It's also on YouTube and it's also on my YouTube channel. So I don't know. You have options.

Speaker 1: I think most people are on Twitch right now. Probably.

Speaker 2: Well, Damien's on Twitch.

Speaker 1: Lerner's on Twitch. The people talking are on Twitch is what I'm hearing.

Speaker 2: Yes.

Speaker 1: And I will link the Twitch. Yeah. Standby.

Speaker 2: So the, the challenge that you're going to have is how do you get this and how do you get it into audio form? Some form of audio only format, which I'm guessing is either. If you set it up as a podcast, it'll give you an RSS feed with the RSS feed. I have a tool that you can use that I made way back in the day that will get the RSS feed. It'll download the audio file for you and then it'll start the transcription. Otherwise, you will need to do something like YouTube DL, which will download it as an MP4. Then you'll need to convert that MP4 or FLV into an MP3. Yeah.

Speaker 1: A big reason I want to do these transcripts too is yes, of course, for the, for that, but like, oh, hey, we still have the just is not having all the transcripts. It's I, I want to be able to ask my own transcripts. This is, this is why I want to do it. I must say, I want to ask my transcripts how I did something so I can refer back to it. Because Anthony and I did something, uh, last year, when did we stream this? Sure. And there's a gist convert MP3 to wave. So, all right. Well, there's some things.

Speaker 2: Yeah, this is, we're going to, we're going to be using Python for this. I don't know what Anthony was doing, not to say that it wasn't, it wasn't like good or anything. Just saying.

Speaker 1: No, he did it in Python.

Speaker 2: What is this whisper.cpp thing he's doing then? Okay. Wait, wait, wait. Hold on. Scroll down. Okay. So he's converting the MP3 to wave.

Speaker 1: Let me, let me explain this to everybody. Hold on.

Speaker 2: He's running Whisper. Yeah. We're not doing half of that.

Speaker 1: This is also, things have changed.

Speaker 2: Yeah. I was gonna say Whisper has changed so much now that that's not that big of a deal.

Speaker 1: But this is what we did back then. And it would have been nice to be able to, you know, like have used my transcripts to go search what I used to do or have done.

Speaker 2: I agree with you. All right. So shall we get started on this?

Speaker 1: Yes, I'm guessing you want audio.

Speaker 2: Yeah, we got it. We got to get the audio thing. But again, I think the fastest way to do this is to convert your thing into a podcast. Are all of those episodes in? Are all of these things in? Oh, no, don't do it this way. No, no, no, no, no. Just make it a podcast. It's so much faster. I'm guessing all of the videos that you want are inside of a playlist, correct?

Speaker 1: Not all of them, but they can be.

Speaker 2: They should be. So let's.

Speaker 1: Add to playlist. We're going to create a new playlist. We are going to call it a playlist.

Speaker 2: New podcast.

Speaker 1: New podcast.

Speaker 2: Yeah.

Speaker 1: Transcribe.

Speaker 2: Oh, well, don't call it transcribe because that's what's going to be called like in podcast players everywhere. Just call it Teach Jen Tech.

Speaker 4: Okay.

Speaker 1: What up, Anthony? I just shared the gist, by the way. Cool. We. Oh, a thumbnail is required. Hold. Look at this.

Speaker 2: I was about to say, Jen. I know if anyone has a thumbnail.

Speaker 1: It's me.

Speaker 2: It's somewhere.

Speaker 1: Okay, we have added this 30 to. I think we added it to it.

Speaker 2: Cool. Maybe those 30 videos.

Speaker 1: Yes. And just while I'm here, I'm going to do the other eight on this one.

Speaker 2: Sure.

Speaker 1: Add to playlists, podcasts, save.

Speaker 2: This is this is where we find out if the thing that I built five years ago works for YouTube podcasts. I'm sure. Yay.

Speaker 1: Okay, we're to the first 30 live streams. Sweet. The next 30 live streams. Done. And the very first of the of the show, though, the first. 2324. Oh my goodness. All of these. Are now. In the podcast. I want to see all 122 videos.

Speaker 2: I want to see how this shows up. It should show it up by date. So I'm guessing that you won't have a problem. Like if it didn't, then like that's just going to be even more funny.

Speaker 1: Connect to RSS feed.

Speaker 2: Oh, I just realized. Yeah. What's the RSS feed for that?

Speaker 1: Well, it just said connect to RSS feed.

Speaker 2: Yeah, connect to RSS feed. Do that.

Speaker 1: I mean, we're going to have to do that to make this work, but I don't have one to connect to it. So will it? Oh. Yes. Hopefully I don't mess anything up.

Speaker 2: I'm hoping this isn't saying like connect to it. Oh, no, no, no, no, no, don't do that.

Speaker 1: Yeah, it's that's connecting to an existing RSS feed.

Speaker 2: Oh, that's kind of cool how that works. I don't know how that works, but that's kind of cool how that works. Okay. So how would I they have to be generating their own RSS feed? All YouTube channels have RSS feeds by default. Yes, you are correct that all YouTube channels have RSS feeds by default. The problem is, is the thing that I'm doing that will grab the audio from the RSS feed is expecting it a certain way. So if we just grab the YouTube RSS feed, it may not have the correct stuff for it, basically. And that's why that's why I really want the YouTube based RSS feed. But now I'm just realizing that that could be a situation on. Let's see. What is the YouTube podcast RSS feed? I really thought you already had these is like audio.

Speaker 1: No, I didn't think about that because when I transcribe stuff, I do it from on an expensive app. But it does it from video. So I didn't even think anything of it.

Speaker 2: Yeah. No, like I said, it's ultimately not a problem. Learn how to edit podcast details. Learn how to reupload episodes. Okay. So connect your RSS feed to upload to upload podcast. Let's upload podcast to YouTube. Don't care about that. Can you check the? Is there like a settings thing inside of that podcast?

Speaker 1: Not that I'm releasing. Let's connect to an RSS feed not create a RSS feed. That's just the channel settings.

Speaker 2: Okay, so what we can do in this case, do you have the ability to download? Yeah, you use you to YouTube DLP to transfer downloads from anywhere into the right way file. Yeah, that's what I was about to suggest is in that case. Let's start with one because we need to show you how to do one will skip the downloading of things and do you have one of two options? Do you have one of two options either something like YouTube DLP or just something to download a YouTube file? Actually, wait, you own the video. Just download the video. Yeah, what am I thinking? Just yeah, pick a video any video video doesn't really matter which one. Whichever one you want. I was so that we can develop off of it.

Speaker 1: Yeah, I was just thinking that I should do a videos. Okay. Why am I like? This is weird date. Like I have a three-minute video that I'm looking for.

Speaker 2: Okay download just grab a video.

Speaker 1: Yeah, because if it's three minutes that it's not going to take forever to go through the process.

Speaker 2: Sure. I like that. Um, okay. So next you need to go to the github repo. We need to fork the repo.

Speaker 5: Okay. Scariness.

Speaker 1: Yeah, I don't know scary for me.

Speaker 2: I'm the one that wrote all the code. So people are going to be like, oh, look at all this ugly code.

Speaker 1: I'm just going to search you. There's you but can I just like users I do this. And then I do this. Hey, look at you.

Speaker 2: That's me. Everybody go on github follow star all the things. Because apparently that's social currency now conduit transcripts right there.

Speaker 1: Well, we'll start it. There you go.

Speaker 2: Oh snap. I made it a template even better. Use this template. You don't have to fork it. Oh, I made it a template so that other people could basically create their own repose to it and make the changes that they need and then have it not be directly connected to my stuff. So you can call it TG TJT transcripts or whatever. Also, if you want, you could probably figure out how to make it so that it auto downloads from YouTube using some other module. Or just using whatever YouTube's RSS stuff is.

Speaker 1: I feel like I can go back and look into that. Yeah, at the moment. I'm like, well, yeah, you're wording but not I'm wording but not making sense.

Speaker 2: Yeah, I'm saying things with the things are making sense.

Speaker 1: Yeah, yeah, no problem.

Speaker 2: Okay, so let's let's dive into what is happening here. Obviously, we have a readme. We have that whole avian service start thing. Let's start with the avian service start thing because that's going to get replaced by maybe maybe gin gin odd one day with just Terraform because why

Speaker 1: not learning Terraform?

Speaker 2: Yeah, we're doing Terraform thing.

Speaker 1: You guys haven't seen it, but we're doing Terraform things offline. It's fine.

Speaker 2: Yeah, instead of instead of doing like this buck wild like services map thing that assigns it basically looks for everything that has a tag and then turns it on and off. We can just define specifically the things that we want using Terraform and then say have at you. Also, there might be a GitHub action thing working. I don't think it's going to work. It's probably going to fail. You're welcome. You can just ignore that file an issue. So yeah, I just wanted to mention that this is a thing that we're using services were specifically specifically using Ivan services. We work for Ivan cool now. It's sponsored. I've been give us some money. Oh, wait, you do. So now the things that we're going to actually do post transcription is are these were going to live in a Postgres instance. Can I come out really quick?

Speaker 1: Yes, I'm just curious if I could get like a poll of the audience of like who uses JQ because like I just started learning about JQ and it does like so many magical things and so now J is working on a like what are they called cookbooks? Yeah, a cookbook of how to use JQ and I was just curious if everybody else knew.

Speaker 2: Well, it's not how to use Jake. It's not how to use JQ. It's how to use JQ specifically with Ivan. I did that but so what I'm more excited about is how I'm building the cookbook because anyone with a basic understanding of how JSON works could template what I'm doing and build their own cookbooks. So I've there's a it's a python installable package. It's not on pip because I don't want to own python done thing like python things but what I do want to own is this like cookbook generator that basically has a textual interface to it. So people can say like create new chat like create new section call this section this and it's basically writing JSON stuff in the background and then applying that to whatever you want. So if you want to create your own cheat sheets or have your own section of like how does terraform work or what are my most commonly used terraform commands then you could do that or same with kubernetes docker whatever whatever challenging computing project that you have is there that's in a future stream though. We're not doing that right now.

Speaker 1: Yeah. No, we're not. But I just it reminded me to say that it was fun and cool to learn about.

Speaker 2: Yeah, I learned jq to avoid learning terraform and then I had to learn terraform anyway. So like here we are but now I know both which is great. Okay. So yeah, like I said, we don't need that shell script very much. We do have a contributing document a license. We don't have a code of conduct that is on me. I am filing an issue as we speak to correct that. Unless it's in the dot GitHub check the dot GitHub. No, I'm horrible. No code of conduct. I fail and then we have transcripts. If we go into transcripts. We have all of these transcripts. These are episodes of conduit. You're not going to want episodes of conduit. You're going to want whatever your podcast is. So later on in this, you know, one issue would be delete the old files. That's actually something I should figure out how to make in the template so that it just doesn't download all of my transcripts here.

Speaker 1: Maybe just do one as an example.

Speaker 2: Well, yeah, I was going to say have like a neck 0 0 negative 1 sample or whatever. But yeah, and then let's go to tools and tools. Are really where we're going to be working doing a lot of our work. So we have OS ingest and PG ingest. Don't worry about those. Those are just like hey, how do I how do I load the stuff that I have into those data stores? We have quick upload which basically is the script that I do that is I need to. Download all this stuff and make it work. We have URL finder which you can look at URL finder. We're not going to use it, but it's probably worth looking at. So I originally and maybe still was scraping the website for the podcast network that I'm on relay to pull down the updates, but for some reason I just blanked out and forgot that. Yeah, I did create this whole like pod downloader thing that will just download the audio file from a podcast RSS feed. So yeah, my my simple self did more work than was absolutely necessary. So yeah, we can get out of that. But that's what that's doing. It's just using gazpacho which if you've used beautiful soup gazpacho is well gazpacho means cold soup. So it's like a lighter version of beautiful soup, which is a web scraping tool. And then we have transcribed pie. This is the thing that you want. This is this is the thing that you you are ultimately excited about. And yeah, as you can see from URL finder import get audio URL from episode number. We're going to we're going to do some surgery on that. So this file is going to change a lot. So let's talk about what's happening. First. We're using typer typer is a command line tool. It just allows us to run this from the command line and provide some of the inputs that we want. So you can say like download by title or download by date or something like that. I mean, you'd have to program that logic in but it would make it easier for you. If you know the name of the episode, you could just say like transcribe dash dash title and then paste the title in and then it would search it by the title and then download it. Then we're using whisper. Yeah, you'll need whisper for the stuff that you're doing and whisper is the LLM powered like transcription tool that takes in audio and then outputs words. I'm actually interested if whisper has like a particular like format because I think that you can do whisper against other stuff. I don't think it has to be.

Speaker 1: A Anthony loves whisper. So I'm just curious if Anthony knows.

Speaker 2: Let's see. We'll transcribe speech and audio files using the medium model looks like flack mp3 or wave. It doesn't look like it'll do video which is fine. That was just going to save us a little bit of time. Yeah, you're going to need to convert your video into audio to make this work.

Speaker 1: Yeah, which I mean if we want to go through that step we can but if I go into StreamYard, which is just kind of fun that StreamYard does this is I have the audio files. You can download just audio only.

Speaker 2: Oh, yeah, do that. Why not?

Speaker 1: So StreamYard also does transcripts, but I don't think that's a very good process long term because then you're very reliant on this tool.

Speaker 2: Yeah, it's also giving you a VTT file, which it can do text. But okay, I was gonna say have fun reading a VTT file.

Speaker 1: It's been interesting. I've been trying it and Anthony said has to be wave at a specific bitrate.

Speaker 2: You sure about that Anthony? And I'm not saying that as an I don't believe you. I'm just reading the docs or I'm just reading their GitHub repo and their GitHub repo says it can be mp3 file. So I don't know. I'll leave that up to you to determine. So yeah, that's the whole thing. And I guess also I have the ability to post this to the chat. So I will.

Speaker 1: Yeah, you use. Okay.

Speaker 2: Yeah, I'm talking about github.com slash open AI slash whisper that python module cool. So let's scroll down a little bit here.

Speaker 1: It didn't download yet. Hold on.

Speaker 2: That's fine.

Speaker 1: We don't you know, it's thinking it's taking a long time.

Speaker 2: But yeah, let's go here. All right. So the next thing that we're doing is we're creating a recursive character text splitter. Oh.

Speaker 1: Which so I was looking at this and I was kind of curious. I was like we have like the link chain and the text splitter in here already. I think it yet because we're going to be doing that during the next steps with this.

Speaker 2: This was a readability thing. So if you go to the very last transcript. In the transcripts file actually go to the one before it. Go to like 80. Okay, do you see how it's all like on one line there just kind of ran together. You can scroll down like it. I mean, oh, yeah. The yeah, it's just a giant wall of text. Okay. Now go to 81. Oh geez. Okay.

Speaker 1: Well, never mind then you I've seen I've I wonder if you've yeah, we did one.

Speaker 2: I probably did do read. Yeah, so it used to be that it used to be like straight up all one line of code, which was not helpful. So it's most most of these are going to be like one line of code. And then we got here to where it's it's splitting things up. It's putting them on new lines actually go to raw because I think it is still just click on raw. No, yeah over to the right.

Speaker 1: Anthony said LRC is pretty readable output format.

Speaker 2: Yeah. Okay. So as you can see here, it's doing some stuff which actually I haven't looked at this. It's kind of interesting because it looks like it's like duplicating some things. Unless that's just me. Being very No. Okay. Apparently I said words like a couple of times in a couple of places. Cool. All right, so all this is doing is breaking it up on new lines that makes it slightly more readable. That's what the segmentation was doing. It will hallucinate repeated words and phrases. Okay, cool. So that's that's possibly what has happened here is that it's it's for it's repeating a couple of things. I never said this was perfect. I just said that it was done. Perfect in that case, so that's that's all that that segmentation is doing because otherwise it just throws it up on the screen and it's done versus hey, if we chunk it and have it actually chunk out, then it will be slightly more readable and maybe we add two splits in the future so that it adds more more spacing between there. Now, you did mention a good point of hey, we're chunking this on 300 characters with the overlap of 20 and all the standard separators. That's a thing that we do in our rag pattern already. If you go to OS ingest pie. Scroll down a little bit. Yeah, you see that same recursive text splitter here. So because we're doing the the chunking already this should actually speed up that process for you because it's just means you don't have to do it. You don't have to do it again. You can just split by new line and embed do all of your chunked embeddings just based on new line.

Speaker 5: Wait, what?

Speaker 2: Okay, so the splitter the the splitter characteristics that we did were split chunk it to 300 characters overlap of 20 characters use the standard separators. We did the same thing for both inside of your actual open search ingestion as well as after we generate all of our wall of text because we've done that. That means we don't have to do it the second time. So in OS ingest instead of doing all of the generating your embeddings and chunking them at the same time since it's already chunked. You can just take your text split it by new line and run. You don't have to you don't have to do that step again. So you're right. I get what you were asking. You're asking like hey, we do it over here. Do we need to do it here? It's like well, you're correct that we don't need to do it twice, but I would actually flip how you're doing it.

Speaker 1: So do it here instead of in transcribe pie do it in transcribes in

Speaker 2: transcribe pie instead of here.

Speaker 1: Yeah. Well, I can fix that when we yeah, we'll fix that when we start

Speaker 2: actually working on it. Let's go back to transcribe pie. So we split it. We check the first character. This was just a thing that I noticed before where it likes to do this weird thing where it'll chunk it on the punctuation instead of like after the punctuation. So if I look in my open search like right now all my open search stuff has like it starts with a period or it starts with a question mark and that's because of how it split. There's just some logic here that fixes that in code here instead of so that I don't have to play with it over there. Oh, no, nothing's going to run for six hours. I mean most of my audio files are like an hour and a half to two hours long and they take like eight minutes to run at max.

Speaker 1: Oh, no, he's saying that any length of something can be done. Like if a show was six hours, it was more like I was worried that I was like, what if we get stuck on it? Just like running all the time and then you don't realize that I got stuck on that line because you walk.

Speaker 2: Oh, I don't think that'll happen here.

Speaker 1: 1% is a pretty pretty decent odds, you know.

Speaker 2: In part that one in part we're streaming our audio through so we would I think we would see it if it was happening. Okay, so then we have a text splitter. So we split a block of text into paragraphs. This is all just logic to do the chunking thing that we're doing. So don't worry too much about this. All this is saying is hey, take that chunk. Check the first character. Make sure that first character is not a punctuation like Mark and if it is punctuation, then just apply that punctuation to whatever the previous chunk was and then we're good and then just return it as clean segments. Let's scroll down. Um, and then yeah, we have this whole download audio file thing. The download audio file thing was using HTTP X to get the actual raw audio from the web and then load it in again. There are ways to get the URL. We're going to we would do this. But since you already have the file downloaded on your local machine, we don't have to do that in the future. If you want to do this, you totally could. You could use the RSS feed from YouTube. You could also use some other things. And then we get to the actual scroll up a little bit. Okay, that's fine. Yep, scroll back down. Then we get to transcribe audio file. It's just doing whisper things. Hey model transcribe cool. And then do it do the thing. Sweet and then transcribe from episode number. This is these are basically just like wrappers around what we're doing to do all the this is the CLI command. If I give you an episode number, I want you to get my metadata and the URL from the audio based on the episode number that I gave you that runs that function that then scrapes the web. Well, it doesn't scrape the web. It scrapes a specific website that I have told him that I have permission to scrape and then it grabs all the metadata from that web page that we need and then it downloads the actual it gets the actual audio URL the link to the mp3 file. Then it downloads that audio URL and we transcribe it and then we return that transcription to our output our output file so we can put like in my case, I would say, you know transcribe pie episode 83. It's going to go to relay.fm slash conduit slash 83. It's going to get the mp3. It's going to download that mp3 and then it's going to start the transcription on that audio file and then it's going to put it automatically in the transcripts folder when it's done. So what we've got to do is we've got to figure out how to make it work for your content and that means we got to get out of a web browser and get into a code editor. Okay, well,

Speaker 5: here we go. Go back to code. Code code VS code. I don't know. I always do that. I always do that.

Speaker 2: You know, you can do all of that from VS code, right? Okay, so command palette time command shift P. Get clone. Clone from GitHub. Give it a path. Yep, perfect. Oh, you pasted something in. Do it again. Command shift P. Get clone. Clone from GitHub. And then you're going to put in right there first option. Yeah, then no, put it in the code folder wherever code folder you wanted to be in. Yep. There you go. Boom open.

Speaker 1: There we go. We got there. The entire reason I have a code folder is because of Anthony when two, oh my gosh, two and a half years ago when I started the stream. That's crazy to think that it's been wild. So long ago.

Speaker 2: Yep. That's a lot. Okay, so let's let's go back to tools and transcripts. Pie. Or transcribed up. I you got it.

Speaker 1: I love how like that's where you go. And I was like, oh we get to delete all the transcripts now like that.

Speaker 2: I mean, you can you can do that too. That's fine. Like

Speaker 1: we don't need to pop quiz time.

Speaker 2: Jen, how would you remove all of the files from the transcripts folder using the command line?

Speaker 6: I wouldn't.

Speaker 2: Well, how would you though?

Speaker 1: Because it's much more satisfying to do it.

Speaker 2: Do it. No, do it command line time. Open the command line. By the way, this is this is how it goes when we're working to it's just like Jen pop quiz time. Do this thing. Okay, so you should be in that folder yet.

Speaker 1: You are in the folder, but I need to go to CD trans.

Speaker 2: No, you don't. I mean, why not?

Speaker 1: I mean you can you don't have to why wouldn't I want to be in that folder

Speaker 2: just do rm space transcripts slash star. Make sure you do the star. The inner

Speaker 1: Oh, I don't want to I mean I can.

Speaker 2: What's this? What's this prompt noise? Don't prompt me to delete all my files. I'll I'll I'll rm dot if I want to you can't I was going to say for I and

Speaker 1: this may not be the best way to I was going to delete like all but one to go back to it. At least that's how I go through rebuilding things in Astro because I'm like I need to see if this works first and if it works then I will delete but if it doesn't work, I need to remember how something was made or at least try to figure it out.

Speaker 2: It's fine.

Speaker 1: Which I mean, I'm pretty proud like eventually my goal is to make this work on the website. But first I need to finish building it but for those who have been around forever, I am further along in my Astro site than I have been ever and I'm very excited about that.

Speaker 2: It still has a lot of CSS work to do time to blow it all away and build it in Django. Oh, I'm kidding.

Speaker 7: No, not until I get this brutal thing working.

Speaker 2: I'm kidding. Like I do not care what your website is built in. You can build it in just raw HTML and CSS and I will still be like wow, that was impressive. It's probably not the best use of time but like yeah, it's an Astro and

Speaker 1: tight. Well, it is in TypeScript and JavaScript, but it's just don't write it in

Speaker 2: PHP. That's a I'm just I'm going to put it up on the screen.

Speaker 1: Yes. Yes.

Speaker 2: I love that.

Speaker 1: Yeah, that would be a struggle. That would be a struggle.

Speaker 2: I mean you can write it in Laravel. Which is PHP not bad.

Speaker 1: No, I'm okay. I'm okay.

Speaker 2: All right. So we've deleted the transcripts.

Speaker 1: That was fun.

Speaker 2: Use use the add code of well, oh wait, never mind. No, because you loaded it from a template. You didn't fork the code. So never mind. You couldn't backport that go to conduct change if you wanted to. Sweet. All right, so you need to install all of your requirements first. Do you want to do it the Python way or the VS code way?

Speaker 1: Python way within my terminal in VS code.

Speaker 2: Okay, that works. Then you're going to do Python dash MB and V dot V and dot V and B words. Python that I like burps hiccups and like stuttered at the same time. I don't know what you call that. I call it an epic fail.

Speaker 7: It's like you short-circuited. I did.

Speaker 2: I very much did. All right, so Python space dash M space V and V Victor echo November Victor Victor echoed November Victor face dot V and V Victor echo November Victor. Yeah, enter. Okay.

Speaker 1: What I have to do Python 3.

Speaker 2: You need to fix that. There needs to be a whole stream dedicated to just fixing that and you say, hey, we created a new environment variable. Do you want to select it? Just hit? Yes.

Speaker 1: So I think you and Anthony would love to just do an entire stream because Anthony and I have done entire streams of trying to fix I've been on different computers and I still seem to break this. Yeah, so just this is just blow it up and start over.

Speaker 2: Just erase the drive and start over like you do it again.

Speaker 1: I have done that many times. Yes.

Speaker 2: Okay, so I mean everyone's done it and get before years. I get this. It didn't work. I'm just going to reclone the thing. Okay, so hit enter inside of the command the terminal again. I just want to make sure that it that's in the code.

Speaker 1: Well, I didn't click up there but it went up there type in which Python

Speaker 2: three I guess or which well what happens if you type in Python now, if it's in the virtual environment? No, no, I'm just saying like if you just type okay, I didn't find Python. Anyway, yeah type in which Python three.

Speaker 1: Which Python just three like that.

Speaker 2: Okay, it's on the library framework. Okay. Yeah, so you're not you're not mapped to your homebrew Python thing. That's weird.

Speaker 1: I mean I can homebrew Python really quick.

Speaker 2: It won't take long brew install Python.

Speaker 1: I feel like I've done this before.

Speaker 2: I'm pretty sure you have I'm pretty sure I've been on the call with you when you've done it. Apparently not though because it's doing stuff. Well, maybe you did it before Python 312. So it just needs to update the newest version.

Speaker 1: I've done a lot of things on this before because like my code dot would not open my VS code for a very very very very very long time

Speaker 2: type in Python type in Python and see what happens. I think you got to restart the terminal restart the terminal.

Speaker 1: I have a feeling it's still not going to work probably not but hey,

Speaker 2: we tried.

Speaker 7: Oh, it did work.

Speaker 2: Now eventually we'll get you into pie environment and then you'll will install 17 versions of Python on your machine and then teach you how to test things with like talks. I'm kidding. We're not going to do any of that stuff right now. Don't worry about it. You're not mad at having to like change nodes.

Speaker 1: I'm like having to do. All I saw was no and I got anxiety. What?

Speaker 2: No, pie environment. No one's doing pie him. We're not doing pie him things. I've got I probably got like seven versions of Python installed on my machine right now. Why because I have different things that I run that support different versions of Python and I mean, that's just what poetry is doing. Don't let poetry lie to you and be like, oh, it's poetry to solve the problem like and JavaScript solve that for me. Oh JavaScript is something something lock files something something Python will get there eventually. All right. Okay.

Speaker 1: I just got to save my lock files on my Astro thing is just destroyed and I have to recreate my lock files. Almost every time I try to edit anything on my Astro lock lock files

Speaker 2: do be locking.

Speaker 1: Okay, files are evil.

Speaker 2: Let's go. Let's go to transcribe. I were there now. Oh, you still need to install your your stuff. You're going to need to exit out of Python there. Nope. Okay, I'm gonna control D. How would I have guessed that like could have also typed in exit?

Speaker 1: I tried escape. I tried it as a function. I tried control C. I know Nancy like is anybody supposed to keep these straight? There was just like go with one exit for everything. Not what is Vim? What has been Vim is like QW.

Speaker 2: No, no colon WQ. But that's that's insert. That's go into command mode. W stands for right in Q stands for quit.

Speaker 1: They need to all be the same thing.

Speaker 2: I mean you can you can quit you can quit them. However, you want to quit it. But do you want your stuff saved is the question?

Speaker 1: I just want to quit to get out of this shit when I'm like just trying to keep coding and they're all different.

Speaker 2: Uh-huh. I got you turn off your computer. That'll work. Yep.

Speaker 1: That is how I always exit out of them.

Speaker 2: I use I use tmux. So whenever I quit, I just start a new session. It's like, well, here we go. New and then I don't ever do control BS because what I love that. It's control BS. I think I set mine to it used to be control SS, but I set my leader to control B because my keyboard is one of these weird things. And then the the B is closer to the control than the SS. So and then I get to do control BS. So, you know, you get to laugh, right? All right, let's install the things that need to be installed all of our requirements. You do that by Python dash. M space pip. Space install. Space dash R. Space requirements dot txt. And you don't have to do the Pippin, you don't have to do the Python dash M part, but we're going to do the Python dash M part because it's important to make sure that we're installing this in the right space.

Speaker 1: I will say every time that we do this and I have to do it locally, but in the Ivan workshops, we do Jupyter notebooks and code code spaces. I'm like this. I get why we do code spaces.

Speaker 2: Mm hmm.

Speaker 1: Yeah.

Speaker 2: I mean, if you want to contribute a code space to this project, I'm not going to stop you.

Speaker 4: Like it's frustrating. All right, install everything. Don't mind me.

Speaker 2: So some of these things we actually aren't going to need, but we're going to we're going to keep in here in case you change your mind. Did it not go to requirements dot txt? I feel like I'm missing stuff from. No, it's all in there. Cool. All right. It's in there. It just wasn't. Maybe I could alphabetize that. It'd be easier to read. I'm going to transcribe dot pie. Cool. Oh, cool. Now they're gone. Oh, laying chain. Wait, laying chain isn't in there.

Speaker 1: I thought I saw a thing. Laying chain hugging face.

Speaker 2: Oh, yeah, you need laying chain as well. Pippin saw laying chain. Note to self add that to the requirements dot txt file an issue. I need you to take a transcript of this and every time I said file an issue just like super cut that.

Speaker 1: And then send it to, you know, I'll practice this. Yeah, well, I'll just the transcript of this. Well, I'll practice it when we're done. Yeah, maybe not today. Probably tomorrow. But okay, look at that. No more squigglies.

Speaker 2: Love it. Okay, scroll down. What do we need? What do we not need? We need all of that. We need that. Yep. We're going to need all that. Yeah, you hear that. So download audio file, you're not going to need because you already have your audio file downloaded. Don't just delete it. Don't just delete it.

Speaker 1: Not. I'm going to comment it out. I'm going to do what I do, which is this. The command backslash best friend and VS code comment it out. Don't delete it. And then when you figure out you got it working, then you can go back and delete stuff.

Speaker 2: I mean, yeah, that's it.

Speaker 1: That's that's how I run the show.

Speaker 2: That's a VS code hack right there. Okay, so transcribe audio file. We obviously need and then we have the audio file path of path, which is good. So transcribe from audio URL. We don't need because that's just not what we're doing.

Speaker 1: But we'll need this back in case we get the RSS feed.

Speaker 2: Yeah, so model transcriber turn over split text. Okay, cool. I think honestly, we just need transcribe audio file and we're done. So instead of and then transcribe from episode number, we don't need either either. Let's just let's just see like can we make this work using the audio file that we have? Oh, no, you need the if name dot main stuff. We're doing that. Cool. All right, so let's change that type or command. We had gotten up down to the very bottom. Down to the very bottom. We have typer dot run transcribe from episode number. Just change that to whatever the one that we have is like was a transcribe audio file. Something like that. Yeah, that one perfect. All right, cool. And then let's go up to transcribe audio file. Okay, you know how we go. I was going to say find. I also feel bad because I part of me was like command or command F transcribe audio file enter. But then I was also like, wait a minute. I use vim mode. So I was kind of afraid to like give you certain commands. I don't know them. I don't know if they're been mode stuff, but or not. I think that one that's just fine. Okay, so transcription equals model dot transcribe audio files audio file. The verbose is false. And then return text split trend. Okay, cool. So. Instead of. Instead of just returning the text because that doesn't help anybody. Let's save it to a file. So maybe we do.

Speaker 1: Well, it'd be nice to return the text to be able to verify it quickly and make a file.

Speaker 2: Okay, that's fine. Let's create a new function. We'll call it preview and save. I mean, technically, you know, Python clean code one function only does one thing. Our function is going to do two things. It's going to preview and it's going to save. And then we need to pass in an audio file. And we're going to return a string.

Speaker 3: Yep.

Speaker 2: Jen, what does that colon path lib dot path do?

Speaker 5: Stuff.

Speaker 2: It tells VS code as well as it needs to get it.

Speaker 1: Yeah, it tells it where it's looking for that audio file and how to find it.

Speaker 2: It's just telling it what type of input that that's expecting. Close. Very close, though. And then the arrow STR. What is that? What does that tell you?

Speaker 1: Telling you that it's a string.

Speaker 2: Well, it's going to return a string, correct? Now, is it going to return a string?

Speaker 1: It's going to return a string and save a file.

Speaker 2: Okay, add that to the comment, the doc string. Press enter, add three. Yeah, that transcribe audio text to file. That's called a doc string.

Speaker 4: And scribe and audio file and save file to save.

Speaker 1: Yep. Save transcript file, save.

Speaker 2: However you want to, however you want to name it. But naming is important. So let's name it correctly. All right, cool. So now what we need to do is we need to say transcription equals transcribe audio file audio file.

Speaker 1: Script transcription. Equal audio file.

Speaker 2: Well, no, the transcribe audio file. We're going to call the function above. Oh, because that's doing all the work. Yeah.

Speaker 1: Transcribe audio file.

Speaker 2: We're going to save its outputs to a variable called transcription. Which is trans. So that's why we had transcription equals transcribe audio file. Oh.

Speaker 1: Yes, we don't have transcription defined anywhere, though. Well, this is where we're defining it.

Speaker 2: Oh, yeah. Also, are you using to change your spacing? Wait, I'm confused because it says that it has spacing is four, but that's definitely not four. That's two. That's two.

Speaker 7: It's four. Okay. One, two, three, four.

Speaker 2: Interesting. Okay. It just looks small. Okay. So now that we've, and also transcribe audio file is a function. So we got to, we got to call the actual function with, that's where you put the parentheses around it. Nope. Put the parentheses at the end. Oh. And we need to give it an audio file path, which is the audio file that we're going to pass into.

Speaker 1: Yep. Beautiful. Do we need the pathlib?

Speaker 2: Nope.

Speaker 1: Okay.

Speaker 2: And then from there, so I would go to the end and then do a new line. All right. So now that we have a transcription, you said you actually want to view that transcription first, right?

Speaker 1: I wouldn't say I need to do it first, but if it like pops, like if we were able to see it right away, then I know if something happened that I need to rerun it. I don't care that it saves multiple files. If I have to rerun it.

Speaker 2: There's a small, we can, we can, instead of doing that, just go to where it says verbose above and set that to true. Cause that's, that's going to write out true Scott. Yeah. Let's say we're in Python land. We capitalize our T's and our F's. Okay. Cause that's going to do that for us. Then we need to write that transcription to whatever our file path is. And I don't know what you want to call that. That could be something that you pass in as well. So you can say like transcription file name or something like that as a function, or sorry, as a, as an attribute of that function.

Speaker 1: That sounds very scary because if I go back to my channel, like there is no proper naming convention.

Speaker 2: Cool. That means you get to make one up.

Speaker 1: Oh God. Okay.

Speaker 2: All right. So where you have, you see if you have preview and save audio file path, lib path, do a comma, and then just do transcription file name as well.

Speaker 1: Up here. Comma transcription. What?

Speaker 2: Underscore file name.

Speaker 1: File name. File name.

Speaker 2: That's going to be a path, lib path as well. We're using typer though. Okay. We're going to, we're going to get into typer just a little bit. Actually, no, we're not. Cause ultimately you don't care about that part. You just want it to work.

Speaker 1: I don't care about it yet. Maybe someday.

Speaker 2: It's one of those things where like, it would be good if we told it that, hey, this can't exist already. So if you said, hey, transcription, file name, transcript, you know, zero one, if you did transcript zero one, again, it would override it. You could, I mean.

Speaker 1: I'm okay with that. It can override.

Speaker 2: Okay. That's cool. As long as you're cool with it. All right. So then we're going to do trans, we're going to do transcription file name at the bottom of the code here. Yep. Right there. Transcription file name. Dot write text. Transcription.

Speaker 1: Dot transcription.

Speaker 2: Nope. They're going to, that's a function. So write text is a function. So we need to give it the, nope. We got to give it the wavy boy. Yeah.

Speaker 4: Yeah.

Speaker 2: Parentheses or nope. Not those are braces. Those are curly boys. Smooth boys. There we go. Round boys.

Speaker 1: I think is what we call them. Round, round boys. Okay. What did I, what was I supposed to actually put in here?

Speaker 2: Transcription.

Speaker 1: Okay. Thank you.

Speaker 2: Because we write the transcription into the transcription file name. All right. And then go down to, we don't really need to call it preview and save. We just need to call it like save transcription. So just do save transcription here. And no, Change the file. Change the function name. That way the function name is accurate of what it's doing.

Speaker 7: What did you want it to be called?

Speaker 2: Just save transcription. And then what we also want to do is we want to specify that the function above is only used internally. So let's add an underscore in front of transcribe audio file at the, up at the, in the define transcribe audio file. So put a, put an underscore at the front of that. Again, these are things that, no, no, no, no, no, no, no. At the, where it says deaf space, put an underscore in front of it. That underscore doesn't really do anything.

Speaker 1: In front of the deaf or in front of transcribe?

Speaker 2: In front of transcribe. Again, that doesn't do anything, but you notice how it made it a little bit darker. Yeah. That is indicating that, hey, this is the thing that we don't want to publicly expose because it's not getting used outside of this. This is being used by like internally. Again, not, not a big deal. Not a thing that you need to sweat too much about. It's, it's just so that as I'm looking at it, I know that this is a thing being called in internally to the thing. Technically, most of this stuff should be, should leave with an underscore, but hey, I'm trying to set up good practices. So make sure you change the name in a transcription equals transcribe audio file there. Yeah, I was going to say line 91. Yeah, cool. Perfect. All right. And then screw down to the bottom and change that type of run to save whatever. Nope. Save whatever, whatever, whatever our new function is called.

Speaker 1: Save transcription.

Speaker 2: Yep. Cool. All right. So now open up the terminal and then you're going to do Python tools slash transcribe dash dash help. It's transcribe.py. I'm sorry. Yeah. So it'll want the file name. Yeah. Enter.

Speaker 1: No such file directory. Why? Oh, I put tool.

Speaker 2: Well, we should see.

Speaker 1: It's thinking.

Speaker 2: No, it's probably downloading the model for whisper.

Speaker 1: Oh, it's did something.

Speaker 2: Yeah. It did whisper things. All right.

Speaker 5: That didn't help.

Speaker 2: Scroll down. SSL certificate failed. What? Yeah. Whisper load model base checkpoint file. What?

Speaker 3: Hold on.

Speaker 2: I was like, I broke my keyboard, lifting it up. I didn't break it, but like I disconnected.

Speaker 1: I've had a Macklemore and Ryan Lewis thrift shop stuck in my head for the last week. And one line is like, I got a broken keyboard. I gave a broken keyboard.

Speaker 2: Yeah. And. Import SSL. This isn't an answer. It just turns off SSL verification. That's funny. I like that answer. Can I get programmed around it? Fails at SSL verification. I've tried using different things. Okay. What was your error? It wasn't. Was it SSL verification?

Speaker 7: SSL certificate, certificate, certificate verified.

Speaker 2: Yeah, SSL verified. Okay.

Speaker 7: That's an effort to say.

Speaker 2: This is annoying. Well, that's Hobson. I know Hobson. I think that's the Hobson that I know. I don't know. I mean, why would I know a lot of Hobsons? It cannot load the models because of the site they're being referenced from is not trusted. Same issue occurs either in the main OS environment or within Python. This works inside of Python. Do the thing. Okay, well. The models are stored in HomeCache Whisper. You can download them. Whisper load model. Running the following command works. Oh, pip install upgrade certify. Here we go. Here we go. That's interesting. Okay, so I'm gonna private, no, pip install. It's pip install dash dash upgrade. Certify isn't spelled with a Y. It's C-E-R-T-I-F-I. Okay. I know, I know.

Speaker 1: Okay, I'm gonna install pip and upgrade pip because it's-

Speaker 2: Yeah, it's gonna ask you to do that every time you ever do anything with pip. Try it again. I mean, no, not that part. Run the typer thing.

Speaker 1: Oh.

Speaker 2: Python dash, or Python, yeah, that one. No. Still failed. Interesting, interesting. This worked for me running the install.certificates command in the folder. Wait. Do type in which pip? Okay, that's the right pip. Want to make sure. Certify failed. Certify certificate chain. All right. I didn't want to do the other thing, but we can do the other thing. At the very top, you're gonna do, oh, I think I remember this. I think that there's like a, hold on. Python environment variable SSL unverified. Okay. I could have sworn there's an environment variable that you can, five ways to do this. The reason I want you to do it, if we can do it through a environment variable is you use direnv. I know you use direnv because I got you to use direnv. So if you use direnv, then you can set it for just that folder, which I mean, you can also just set it for the code itself, but, yeah, update SSL certificate with pip install dash dash upgrade certify. We did that already. It still didn't work. Okay. Maybe I was wrong. Maybe that didn't do it. Maybe that doesn't do it. Okay. So we can do the create unverified context. Go to the top of that transcript, that transcript.py file. Let's see, can I make this, can I separate this window? So I'm gonna need to be able to read this. Okay. So at the very top, you're gonna import SSL. Not a fan, not a fan of this solution, but okay. And then the next line you're gonna do is, oh, I don't have, I'm gonna copy it to you in private chat just cause I don't wanna mess it up. Oh, why did it, for the love. Okay. Well, I can just. No, cause it's, it wasn't that, the import SSL part. Yeah. Don't separate your imports. So copy that line. Nope. Copy everything from where you got it. Everything from where your cursor is to the right. Nope. Just the SSL create default HTTPS context, that whole thing. Cut that. And then scroll down and paste it above app equals typer stuff. Technically, as long as you paste it before you get to the load model, you should be fine. Yeah. Well, you want some spacing in there, but it's fine. Fine.

Speaker 1: I'll put the imports and froms, but all the imports could be together.

Speaker 2: So in Python, the standard for this is everything that's a standard live application should be together. Then you add a new line and then everything that you actually import directly. We have linters that do this stuff for you. And I'll explain that to you later. Okay. Run it again.

Speaker 1: Yeah. It's doing stuff.

Speaker 2: Yeah. It needed to download the model and it couldn't download the model because the SSL certificate of the model isn't a verified certificate. All right, cool. So now we get what we needed to get. Sweet. So this tells us how we use this command. You see where it says transcribe.py and then audio file, transcription file name. They're both arguments. It's just, we want to make sure that we're doing this in the right order. So we could also just force it to do it that way too, but I'll try to save you time in this. So if you do Python transcribe.py, I guess it's tools slash transcribe.py, but, and then you need to pass in the path to the download file that you downloaded.

Speaker 1: Hold on. Let me put it in the same folder. Yeah. Or at least in this two things.

Speaker 2: Yeah. Just don't, just don't commit it, which I think I might've updated the get ignore to not include those things.

Speaker 1: Then I'll do it in a separate folder, but.

Speaker 2: No, you can do it in a separate, you can do it in the same folder. Just make sure that MP3 files are ignored, which I'm looking at the 84 files that it says have changed. And I'm like, whoa, that's.

Speaker 1: Because we deleted all of the episodes.

Speaker 2: Oh yeah, that's right. That's right. So yeah, go ahead and go ahead and load up the new MP3. Let me go over here on mine. Okay. What were some of the things that I didn't like about this? GHSU create the title is.

Speaker 1: Okay, so I put audio files and then I put the name of the episode.

Speaker 2: Or just whatever, no, the name of the file. So whatever you want to call the file, if you want to call it.

Speaker 1: Oh, well, don't I need to tell it what it's already called or do I need to rename it before I called it?

Speaker 2: No, your audio files is fine. I would wrap that in quotes, it looks like, because it seems like that is not.

Speaker 1: Okay, hold on. Let's do this.

Speaker 2: Yeah, because you have spaces in there. Make sure you, what happened?

Speaker 5: What's going on?

Speaker 1: I'm deleting it because I'm changing something. Hold on.

Speaker 2: Okay.

Speaker 1: We're gonna do this.

Speaker 5: It is. Okay.

Speaker 1: There.

Speaker 2: I mean, that works too.

Speaker 1: All right. So now enter.

Speaker 2: Nope, because you still need to give it the transcription file name.

Speaker 1: So then I would want that in transcripts. And I don't want it to be a .mp3. I want it to be a text, right?

Speaker 2: Mm-hmm.

Speaker 1: There.

Speaker 2: Now hit enter.

Speaker 1: Oh, it did things.

Speaker 2: Okay, so this is fine. This is just, this is talking to us.

Speaker 1: It's pretty.

Speaker 2: It expected a NumPy array?

Speaker 1: I'm trying to go to the beginning of it so we can kind of.

Speaker 2: You're using torch.load. Flip to true. External unpickling. Arbitrary objects will no longer be allowed. In case you don't have control loaded. For any case you don't have control, please open to get a issue. FP16 is not supported. Using FP32 instead. I mean, that's fine. I'm just, I'm just confused in where it's vomiting.

Speaker 5: What's going on, Tom? Oh, keep going.

Speaker 1: That's the end of it.

Speaker 2: Okay, if, sorry, if not torch is tensor. You have to tell it to.

Speaker 1: Can it do it from an MP3 or do we have to do it from a WAV file?

Speaker 2: It's, that's not it. Oh, do you have FFmpeg installed? I'm guessing that you do since you've done this before. Just type in FFmpeg.

Speaker 1: All one word?

Speaker 2: Mm-hmm. Yeah, you do. Okay, so that's not the problem. Let me see, github.com, OpenAI Whisper. We use, we use 399 PyTorch to train and test our model. It's fine. The code base depends on a few packages. OpenAI's TicToken for their fast tokenizer implementation. You can download and install the latest release with OpenAI Whisper. We did that. Don't need to do any of those things. It also requires FFmpeg. We did that. You may need Rust as well in case TicToken does not provide a prebuilt will. You don't need that.

Speaker 1: I'm like, should I brew and install Rust? I can.

Speaker 2: No, no, no, you don't need to brew and install Rust. Um, there is something missing, but I don't know. Internally, the transcribed method reads the entire file and process the audio using a sliding 30-second window, performing auto-regressive sequence-to-sequence predictions on each window. Below is an example of detect language in Whisper and decode. That's also not the problem. Let's go. Well, first and foremost, can I run it on my machine? Let's find out. See, yeah, transcripts. If I do, I don't need to share my screen for this. This is going to be quick. Python.env.

Speaker 5: Okay.

Speaker 2: Yep. Okay, Python. Tools transcribe A3, A2, why not? That download everything. I just want to see if there's like something that I'm missing on mine that has the same problem.

Speaker 1: Makes sense.

Speaker 2: No, mine's running just fine.

Speaker 1: Story of my life.

Speaker 2: Yeah, no, it works on my machine. Okay, so.

Speaker 1: Well, I'm curious. Can I, why don't I push this and then have you try to run this one on your machine? Because I'm wondering if it's something that we changed.

Speaker 2: I mean, if you want to do that, yeah. Expected a NumPy India array and got a POSIX path. All right, so my early, early, early suspicion is that the, some of the things that we changed. None of that matters. None of that matters. Download the file, stream it, get the chunk. Then we just get the name. That's fine. And then from there, transcribe from episode number. Episode number is just the integer metadata URL posts. Output file, path, write text. Okay, so transcription is equal to transcribe from audio file or from audio URL. Transcribe from audio URL does. I'm just trying to make sure that my transcribe from audio URL didn't do some other goofy thing. Audio file, download audio file. That just gives me the path name. Download audio file just returns a path name. Transcribe audio file just runs transcription model transcribe. Yeah. Interesting. Okay, so you pushed your code?

Speaker 1: Yes.

Speaker 2: Okay.

Speaker 1: And then I just sent you the link.

Speaker 2: Did you include your audio file?

Speaker 1: Oh yeah. Because I forgot to get ignore it.

Speaker 5: That's fine.

Speaker 1: But hey, now you have my audio file and I will get ignore stuff. Get ignore.

Speaker 2: New window, cool. And then I need to go back to there. Cool, we're gonna gh-repo-clone. What is yours called?

Speaker 1: TJT-transcripts.

Speaker 2: And you're just ginginod all lowercase? Yeah. On GitHub. TJT, what?

Speaker 1: Transcripts.

Speaker 2: Transcripts.

Speaker 1: Okay.

Speaker 2: This could be something as simple as like, hey, NumPy is supposed to also be downloaded and it's not or something, I don't know.

Speaker 1: Would I put it in the distribution? Just put it at the top. Just put it at the top. And I just do forward slash the folder name, right?

Speaker 2: Yeah. You just do audio files, call it a day. Just do audio files. Just remove the forward slash, just audio files. And then, yeah, add a trailing slash on the end. Okay. You still gotta get remove the other stuff though. All right, cool. But you're gonna wanna do git rm dash dash cache.

Speaker 1: Git remove them when I re-upload?

Speaker 2: Nope. Because, oh. Nope. Once it's added to the git history, it's added to the git history until you tell it it doesn't need to be added. All right, so I am here. I'm gonna share my screen. Okay. Over here.

Speaker 1: Do it.

Speaker 2: Also, I'm sorry for people's eyes. Oh. Sorry.

Speaker 1: I did it. Don't mind me. There you go. It's fixed. Can you zoom in a bit, please?

Speaker 2: Yeah, trying. My keyboard got mad at me again. I cannot zoom in again.

Speaker 1: Okay.

Speaker 2: Oh yeah, I can. There we go.

Speaker 1: Is that better? Yay.

Speaker 2: Yes, thank you. All right. So. There we go. Yeah, you don't need half this stuff. Python. Oh wait. Yeah, Python dash m.

Speaker 5: Okay.

Speaker 1: And I did add link chain to this, so it should.

Speaker 2: Did you add it to the requirements.txt?

Speaker 1: Yeah.

Speaker 3: Okay.

Speaker 5: The suspense.

Speaker 1: I feel like this means that we're probably, since we only have half an hour left, that we'll end up streaming sometime next week or tomorrow to finish this. I'm like, ah. Well, once we get this working,

Speaker 2: the next step is you gotta download, you gotta generate all your transcripts. And I mean, we could do a thing where you like, auto-upload, which is kind of where I'm trying to get to with this as well as where like, again, that's where I want to have my own Terraform file that says, hey, load this thing in, pull down the transcript, chunk it, automatically upload it to the places it needs to be uploaded to. And also do that and just let GitHub do it so I don't have to. That's a good idea. Yeah. Yeah. So I don't have to. That's why I was saying there's a GitHub action in there that is going to fail because I started developing that. And I know it wasn't working when I last tried it. Transcribe. And then you said there's an audio file in there. Yeah. Yeah. I'm going to call this test.txt. Okay. It's downloading things. Huh.

Speaker 1: Okay.

Speaker 2: I wonder if, oh.

Speaker 5: Mm-hmm, mm-hmm, mm-hmm.

Speaker 2: Okay. So let's look at my version that's done, that did the thing. Hey, nope. That's not it either. Okay. There we go. This one. All right. So the thing that I was doing here was I was streaming this audio file in, loading it into bytes. That's weird. That's weird. Okay. I'm going to take my screen down for a second. I'm going to put your screen up just so that I can make sure I don't have anything set on mine that I need to do on yours. Auth tokens, those are fine. Indexes, those are fine. Okay. But separate auth token, are those the same token? Those aren't the same tokens. I don't know where that one's going. That's wild.

Speaker 1: Could it be the, no, nevermind. Oh, with tempfile.name tempfile, that one?

Speaker 2: Yeah, but that's not the problem because your audio is your audio. You don't, you didn't have to do anything with the audio. This is what, this just downloads the audio. I just think that's so wild that like, it was just like, nah, dog. Oh, wait, I'm in mine. Get out of here.

Speaker 1: I mean, yeah, because it's not going to go through because I don't have it from the URL. But I mean, that doesn't matter

Speaker 2: because as long as you've got an MP3, encoder, audio encoder. I'm just wanting to make sure that there's not like some setting I'm just wanting to make sure that there's not like some setting that needs to be updated. Verbose, I mean, the one thing that is different is I have verbose turned off. So let me. Nope, verbose turned off still didn't work.

Speaker 1: But I got a type error, expected MP.nd array.

Speaker 2: Yeah, it's the same error. That's wild that it's, all right, well, let's go to there. Let's see, whisper expected numpy array. Okay, I want to send speech to whisper as a numpy array. Documentation says this is possible but did I get the correct transcriptions? We're doing some py audio stuff. Yeah, this is interesting. Return MP from buffer flattened out as an MP array. The thing that I'm confused about is why does it, it doesn't need to be an MP array. Go to your model. No, no, sorry, go to transcribe.py.

Speaker 4: Okay.

Speaker 2: Your model is base.

Speaker 3: Okay.

Speaker 2: That doesn't make sense that my version works on my machine and your version, but your version doesn't work on my machine. And the only thing that we changed is what the file is saved as. This is not like I'm saving it as an MP array.

Speaker 5: Hold on, let's try that.

Speaker 2: I think that's going to be the same thing. Yeah, you got to change stuff down there too.

Speaker 5: Easy, what did I call that one? Transcribe.

Speaker 2: I'm guessing you'll get the same error. Oh yeah, it doesn't know how to save it because you've taken out the- The save, yeah. Yeah, you've taken out the file. I don't think that's it because the transcription takes time. So if there's a problem, the problem would be in the first part, not the second part. The thing that I'm interested in is, is there something in, let's do some explore, let's do some exploring. Import whisper, whisper. Let's see, dear whisper. Oh, we got to do a model. So model is equal to whisper load model. Yeah, load model. Load model, and we want to do the base model. Okay, base model's done. What does this say? Oh, future warning, you are using torch load with weights only false, the current default value, which uses the default pickle module implicitly. It is possible to construct a malicious pickle in the future release. Default value for weights only will be flipped to true. This limits the functions that could be executed during unpickling. That's fine. Arbitrary subjects will no longer be allowed to load. We recommend you start setting weights only true for any use case where you don't have full control of the loaded file. I don't really care about that. I care more about like, what is the, if we do model, let's see, dear model. Um, because my guess is that it's like a, there's probably something being, that's being set to. It's like, what is the file expected to be? I don't know. I'm trying to see if it's somewhere in here. Device, stems, detect language, decoder, CPU. Okay. Let's look at the code. Look at the code. We go to Whisper. We go to model, model dimensions. Oh, lovely. We just call things X in here. That's beautiful. Gonna vomit.

Speaker 5: QKV.

Speaker 2: Audio encoder. Positional buffering. Text decoder. Okay. Where is your error at? Your error is sitting there at, scroll up a little bit. Yeah. Audio.py 141. Okay. So I'll, I will, I'm gonna share my screen again. All right. Okay. Before all this.

Speaker 1: Okay.

Speaker 2: Yeah, we know your directory. Audio.py. Okay. Okay. Sample rates, frames per second, load audio. Open an audio file and read as a mono waveform. Resampling is necessary. Returns a NumPy array. This launches a sub-process to decode the audio while down-mixing. Requires SFMPEG CLI in the path. Try run command capture output is true, standard output. So part of me wonders if it's your audio file.

Speaker 1: I can download another audio file. Well, no. Or test another audio file. No, it's,

Speaker 2: because I'm downloading from just like a standard, like, downsized MP3 doing things. I don't know how, because, what I'm saying is it's weird that they emulated it's weird that they immediately try to load this as a NumPy array for you. And they weren't able to generate a NumPy array, which is weird.

Speaker 1: Because I didn't- Wait a second. I'll download an actual like podcast podcast from my podcast just to see if it'll make a difference.

Speaker 5: Okay. And it's, it to talk about example. Hmm.

Speaker 2: Hmm. Yeah, this is interesting that it's whisper transcribe.py.

Speaker 1: It did not like a different audio file. That's literally only been an audio file.

Speaker 2: There's a part of me that wants to update. Here, we'll do this. Okay.

Speaker 1: But just to like triple check, I will go back to like my first episodes before I knew that StreamYard was a thing. Just to make sure StreamYard had nothing to do with it.

Speaker 2: I don't think StreamYard had anything to do with it. Okay.

Speaker 3: Hmm.

Speaker 2: Okay. So 2023, 11, 17. Why is that so old? They just like don't care about that anymore? I'm dead serious. Like, has it just not been updated? Is that?

Speaker 1: Maybe? I don't know. We could ask them.

Speaker 2: I keep doing that. I keep going into the wrong window. Here we go. Yeah, it's the same version. Okay, so it's not, it's not a versioning issue. It's not like they changed anything. So it's something that is absolutely different in.

Speaker 1: Which the only thing that I could think of is because we're not going from an audio. So give me a second. What do you mean? Like we're not doing a URL.

Speaker 2: Yeah, but that's not important. The audio, I mean, yes, it could be the reason, but it's how we process, how we download that audio could be a better reason to why it's not doing it than, hey, your MP3 versus my MP3. Why do they look different? Because I don't think they look different. I think they're being processed differently. I need to go back to mine.

Speaker 1: Yes, if I wanted to test a, just to see if the URL worked, I can try the RSS feed from shit to talk about, just to see if it happened to work.

Speaker 2: Okay, let's see here. Transcribe audio file. Audio file is a Pathlib path. We're getting that Pathlib path from here. It's in temporary file.

Speaker 1: You're in yours now, right? Yeah, you are.

Speaker 2: Yeah. For chunk and response, iter by its right chunk. Yeah.

Speaker 1: Where is it pulling the audio file or the audio URL, is this how we put it in there? Okay, yes, yes, yes, yes.

Speaker 5: Hold on.

Speaker 2: Downloading audio file from URL right there. Transcribe audio from audio URL. So download audio file from URL, transcribe audio file. Because what mine does is it just downloads, it saves it to a temporary path and then transcribes based on that temporary path. The metadata and all that other stuff is not important because that's just front matter stuff for the actual file. It's not anything, anything important. Post, front matter post, transcription metadata. Yeah, transcription is the content. This is transcribed from audio URL. Return transcribe audio file, audio file.

Speaker 1: I just tried it with the URL and it didn't work either.

Speaker 2: Same error?

Speaker 1: It's same error.

Speaker 2: Okay, so that tells me it's, it feels like it's a configuration issue, but I don't know, I don't know what in configuration is different in mine than is yours. So I'm gonna jump back to yours, standby. Is there a way I can diff? Well, I can diff the two directories, but that's, LST. Okay.

Speaker 1: Cause you can see that like the path here, I use the URL and it couldn't handle it.

Speaker 2: No, you passed in the RSS feed.

Speaker 1: Wouldn't that be the URL to the episodes?

Speaker 2: Nope. It's gonna be something, something, something.mp3.

Speaker 5: Okay, then I'll try that.

Speaker 1: Wait, then how are you using the RSS feed for the MP3?

Speaker 2: I'm not, I'm scraping the website. I was saying I should be using the RSS feed because the RSS feed is gonna be easier to work with than scraping a website that could technically change over time.

Speaker 1: Then just get me.

Speaker 2: Yeah.

Speaker 1: But I have tried other URLs or other MP3 files.

Speaker 2: Well, there's kind of the other part of that too of, I don't know if the way that I'm downloading it is the same either. But the thing that I'm interested in here is something has to be set differently between the two. And I don't know what is set differently between the two. And that bothers me because you forked my code and it worked. But, or you forked my working code. Like do this, uncomment all the things.

Speaker 1: I did, and it didn't work.

Speaker 2: No, uncomment all the things.

Speaker 1: Okay. Wait, wait, wait, wait.

Speaker 5: And there it is.

Speaker 2: Done. Okay. Now run the Python tools transcribe and just put in transcribe zero. Cause that should in theory transcribe my podcast file. No, no, no, no. Just zero.

Speaker 1: Your file's not there anymore.

Speaker 2: You wanna change instead of save transcription that's gonna transcribe from episode number.

Speaker 1: But I don't have your episodes in there anymore.

Speaker 2: It doesn't matter. The episodes aren't pulled from there. It'll save it there. Episode number. Transcribe from episode number. You probably wanna make sure the stuff doesn't name the same. So scroll up to the top.

Speaker 1: I don't care what it names it.

Speaker 2: Okay. That's fine. Run it. It's, no, I wanna make sure that it's called, everything's being called the same. Yeah. See, it's downloading the Conduit episode. Transcribe audio file got renamed. So you need to go back and remove, remove the underscore. The thing that I did that I was like, yeah, you should do this because technically blah, blah, blah. Yeah. Now I broke everything. So you can blame Jay for, for breaking everything.

Speaker 1: It's doing something. It just works.

Speaker 2: Why does, why does it work that way, but it doesn't work?

Speaker 1: Because it's using your, the URL to the, URL to the episode.

Speaker 2: But the thing is, is it's downloading that URL to a file.

Speaker 1: But it still has a URL to start with.

Speaker 2: But the URL is not important.

Speaker 1: It thinks it is.

Speaker 2: I, and I know that because if you go to, did it, it tells you where the file is saved. Saving audio file to VARS folder, TP, some random string, some random file name. So go do this. Take that random file that it saved. Just copy that whole line. Yeah, that's just audio file. Copy that, copy that VAR folders line there.

Speaker 1: Copy.

Speaker 2: Okay. Now I'm trying to think of the right way to do this because I guess if you do transcribe, you do, the transcribe is the actual file. It doesn't do like an open. So let's just open up, just open up the REPL. So just Python enter. Okay. Import Whisper.

Speaker 1: Enter.

Speaker 2: Import pathlib. Model equals whisper.loadmodel base. You can tab that out, tab. Nope, it's load underscore model. Nope, you're right. I mean, spacing is kind of whatever. Whisper.load underscore model. And then you're going to do base, like you're going to put.

Speaker 1: Am I spelling model right? I always felt like I spelled model wrong.

Speaker 2: Nope, that's right. It's a function. So you're going to pass into it base. As a string, base is a string. That's the name of the model. Enter. Model.transcribe. And then you're going to pass in, you're going to paste in that path. So you're going to do model.transcribe, open parentheses, do, don't, like do pathlib.path. It's capital, it's just capital P, everything else is lowercase, there you go. And then parentheses, quotes, paste. Another quote, yep. Close, close. You've got a space in there, by the way, after pathlib.path. You go back to the very, very-

Speaker 4: Oh, mother trucker.

Speaker 2: Okay. That's fine. Transcription equals, and then all that stuff.

Speaker 1: Equals path-

Speaker 2: Model, model.transcribe.

Speaker 5: Model.transcribe, code pathlib.path. And bracket, A, A, enter?

Speaker 3: Yep.

Speaker 5: No.

Speaker 1: And we got to wrap up too, in like four minutes. So.

Speaker 2: I'm so confused. Because you're right. I mean, it's something to do with how it's being downloaded, or how it's downloaded. What happens if you just pass in as a string, you don't do pathlib.path? So just hit up.

Speaker 1: I am hitting, oh, wait, there we go. Ah, no, go back up.

Speaker 6: Do, do, do, do, do, do, do, do.

Speaker 2: That doesn't feel right, but you're missing an extra one. Just go up and then remove one. Okay, so Holden, yes, you're correct. But here's the sticky wicket. This code works. It just works.

Speaker 1: Only for your stuff.

Speaker 2: Yeah. Yeah. Yeah, we get that it wants a nd array. Somehow I'm creating an nd array, but okay, just transcription. Just enter transcription. Because it seems like it works. That, I mean, it didn't fail. But I'm also confused on like, yeah, that actually worked. So, okay, syntax error. I don't, I don't know why. I'm guessing, and this is just a guess. So when I do download audio file, when I do all that download stuff, the path is being passed in as a string. Yeah, it took a string with the path, but not a path of path. Which is weird because you could just do string path, the path, and that would fix it. Okay, so for you, Jen, what you're gonna wanna do, you still want it to be a path of path because we wanna make sure that in the future we do the thing. Instead of audio file. Nope, scroll down. Right there. Where it has audio file. Nope, keep that as a path of path. Keep that as a path of path.

Speaker 6: Okay.

Speaker 2: Because otherwise, later on, you're gonna try to verify things and it's gonna break. Where it says audio equals audio file. Put audio equals str audio file. Space audio file? Nope. String is a function in this case. So, yeah, audio file. And then, say, go down to the bottom. Transcribe from episode number change. I have to save transcription. Okay, now go, yep, in there. I wanna... Exit out of that. You can just type in exit as a function and it'll exit. See, look, it told you, though. That's what you need to do. Now run your version again. Run your stuff again. Mm-mm, up, keep going up.

Speaker 4: Okay, go up, go up, go up, go up.

Speaker 2: There's no more ups. Why are there no more ups? Oh, I don't know. You don't have history anymore. Okay, so yeah, delete all that and then put in your audio path and then put in your file name.

Speaker 3: That's... That's...

Speaker 2: I don't understand. This is one of those things where, like, a string is okay, but a file name isn't. And the only reason I think that that would even be, like, manageable is, like, maybe you... Yeah, you gotta remove the underscore in the other one. Yeah, right there. The green squiggle, right there. Just remove the underscore.

Speaker 1: Thanks. I was like, sometimes I just start, like, it just starts blurring.

Speaker 2: And now it worked. It's working.

Speaker 1: Well, we got it working. So... All because of, it needs to be a string audio file.

Speaker 2: Okay, so here's the wrap up to this. Where is mypy? This is where mypy and typed Python is great. Oh, this is the comment, right here. This is the comment, right there. If, and I'm not gonna say that they didn't type, they don't have type hints and things like this entered, because maybe they do. Interesting enough, they do. And it can either be a string, a NumPy array, or a Torch TensorFlow, which is really, really interesting. Because the string, like, what do they do? Are they, ugh, ugh, my brain hurts trying to read this code, by the way. I'm not blaming them. I'm just trying to get to where the audio file actually gets called. Like, because the thing that I'm most interested in, in this is, from audio, audio, is if they're checking, if they're allowing you to use a URL, and they're saying, like, this can be, this is a string because it can be a URL, or it could be a file path, and it's just going to assume that, if it's not a URL, it's a file path, then I can understand why making it a string is okay. But, just being a string feels weird. Maybe that's also why I did name. Oh, oh, oh. Okay, scroll up. Just look right there. Notice how, when I download this audio file, and I write it to a chunk, I don't return the actual audio file, I return the name of the audio file. Which gives it the relative path. F.name returns a string, it doesn't return. Otherwise, I would have just returned F. I don't like it. I personally don't like it, not a fan. I'm just saying. Well, we did what we could. No, this is fine, this is perfect. Because this means that now, F being the file handler, yeah.

Speaker 1: We can pick up back on this later. Because we gotta wrap up. Yes. Like four minutes ago.

Speaker 2: Yes. What do you need to do now, now that this works?

Speaker 1: I'm going to upload to GitHub, and try seeing what happens, and come back to it after I step away for a bit, because right now, my mind is mush. Yeah. And I don't wanna break it beforehand, so.

Speaker 2: I was just gonna say, start saving more transcripts. Get your transcripts in there. And whether you choose to download all the audio files themselves, or figure out a way to load them directly from the URL, or whatever. Again, I love the fact that now, you did set this up as a podcast. It's probably gonna take a couple of hours to do the thing. If you do that, you probably would be able to,

Speaker 1: if you didn't.

Speaker 2: If you can find your podcast somewhere, you're more than likely gonna be able to get the stuff done. You're gonna be able to get the, what you call it? What's up, Jen?

Speaker 5: It didn't put the podcast.

Speaker 2: It saved it where you told it to save.

Speaker 3: It's right there.

Speaker 1: Right where?

Speaker 2: It's not in transcripts.

Speaker 1: No, that's your transcript.

Speaker 2: You saved it to test.txt. Where's test.txt?

Speaker 1: That's why. Thank you. I was looking under transcripts, because we tested it, and I forgot we did that. Okay, good. We did it. We got it done.

Speaker 2: Yeah, you've got your other podcasts in here, shit you don't wanna talk about. So you could either, we'll figure it out. I'll do some updates on my end. I'll connect with you again next week, and then.

Speaker 1: I feel like what I need to do next is create a, like this might take a minute, but create a text file that has the URL of each of the episodes, so it knows to pull each URL.

Speaker 2: That exists already. It's your RSS feed.

Speaker 1: I know, but YouTube's RSS feed is not. Then where do I put my RSS feed in here?

Speaker 2: I mean, you could also just download the RSS feed from the internet.

Speaker 1: But then where do I put it in here so it reads the RSS feed?

Speaker 2: You just, that's the part that I'm telling you I need to work on. Because my hope is I need to blow the dust off of an old project that I made four years ago that does that for you. And then we just need to modify it so that it accepts YouTube feeds. And then if we get-

Speaker 1: Sounds like that's what the next stream is gonna be.

Speaker 2: Sure, sounds great.

Speaker 1: It's gonna be fun.

Speaker 2: I like it. All right. All right, this has been fun.

Speaker 1: I'm ending this stream. Thank you, everyone.