How to Run GPT-OSS Locally in LM Studio (Full Transcript)

[00:00:00] Speaker 1: There are a lot of closed models available, but the one that I'm going to talk about today is GPT-OSS 120 billion model, and then we have GPT-OSS 20 billion model, right? This one, these models were developed by OpenAI, the same company that developed Shatcha PT. They have made some of their models publicly available so that you can download onto your computer and use it locally. So in terms of the 120 billion, think about it as data points, right? So you can see that the 120 billion parameters, which is the first model, is larger than the second model, because the second model is only 20 billion parameters. So this means that if you want to use the model, you always have to check. Whether you have enough memory on your computer to use it. So if you don't have enough memory, maybe you have to try the 20 billion model. Because the 120 billion model is, this means that you need a large memory, right, to use it. From this computer, I think I have about 40 gigabytes of memory. So I'm going to try and use the second open source, which is the GPT-OSS 20 billion. You may ask, so how are you going to download this one into your computer? So you can go to these websites, Hugging Face, you can go to Ollama. You can also go to LM Studio, right? You can go to these providers to download this model. One of the providers we're going to use is the LM Studio. It's very easy to download their platform and also download the model that you're using. So if you want to use, right? So there are two things that we have to think about. We have the LLM platform and also the AI model, right? So let me give you an example. Chart GPT, we have GPT-5 is a model, right? But it has to be run on the Chart GPT platform, right? So the same thing for concerning the open model, you have to first download a platform which is when you go to lmstudio.ai, you'll be able to download that platform. And then after downloading, you can also download the open source model that you want and run it on your computer. So these are the things that you have to know. I'm very excited because this one is really going to help a lot of researchers who are concerned about using. Sensitive information or using participant data that you don't want any third party to have access to. So let's see what we're going to get. So when you go to lmstudio.ai, you can first download the LLM platform you're going to use to run the models, right? If you have Mac, it will show Mac here and you choose from, I have Windows. That's why Windows is highlighted here so that I can download. I'll put the link in the description section so that you can get access to it and then click on that. It's free to download, right? So you click on download and downloading on my computer. Okay. So for this, I leave this one alone. I don't, because I'm using it for myself, I just select only for me and I go to nest and you can browse and decide where you want to save that information, but you can save where you want to save it. You want to save this program. So I just leave it this one alone and go to install. So the process is you first have to download and then you can start the installing process. It looks like it's almost done. So it's finished. I click on the finish here. Yes. Now we have downloaded the LLM Studio platform, right? So what do we have to do? We have to look for the open source model and download it here so that we can use, right? So we want to look for it. You can click on discover and then you search for it. So you search for GPT and then the one that is, you know, you can see that this is the one that we are looking for. And then you can, you know, select that. Normally here will be download, right? It will show download here. The reason why I'm not showing that. The reason why I'm not showing download here is that I have already downloaded. So when you already downloaded it, it will not show download here for you to do. Let me show you an example. Since I have not downloaded this, you see how it gives me the chance to download. It also show you the size 11.62 gigabytes, right? So you just have to look for GPT OSS 20 billion. Okay. And then you can click on download. After downloading, if you want to see the model, you can go to this place, right? Or you can click on, you can go to control L. If you control L, it will show them all the models that you have downloaded, right? But you cannot use it without loading it, right? Because let's say you have downloaded about five open source, you can only choose the one that you want to use. I don't know. You can click on that moment, right? So if you want to choose the GPT, this one, the one that we just downloaded, you click on that. When you click on it, it's now loading onto the system so that you'll be able to use it. Right? So let's wait for it to load and then let's see what we're going to get. It looks like that's loaded. So now we can use it, right? But before we use it, you see here, resident airport. This is where you can see. Okay. This is where you can set whether you want the system to use high level of reasoning or low level or medium, right? When it comes to qualitative data analysis, it's not all that complex to the model. So you can choose the lower one because when you choose the higher one, it will take some time for it to work through to give you the answer and you don't need a high one to get rich information. So I'll choose the lower one. Okay. But if you want to do a complex analysis, such as trying to incorporate maybe a theoretical framework, when you're analyzing your data, then you can choose medium or high, but now we're just going to choose the lower one for now. And so before we look into analyzing our data, which is, you know, we have five participants and these are, these are five participants and then the data is not a lot, it's just like one page. For each participant, before we do the analysis, I want to test the system and ask some questions and see how it will respond, right? So for the first one is I want to find out what is the difference between content analysis and thematic analysis. So let me bring this prompt here. So what is the difference between content analysis and thematic analysis? And let's click on enter and see what we're going to get. So it's showing you that it's using the. Okay. The 20 billion parameter model. It's not all that fast. The reason being that you are using it locally. So we don't have a lot of space for the system to run quickly. So you have to, you know, bear with the system, but I think it gives you, it's giving us a very good information. This is awesome. The good thing is that your communication is not being viewed or used by anyone. It's being used by any third party, right? And that's what I like about the open source. So now it's giving you the difference between them. Yes. So yes, you know, content analysis, deductive, thematic analysis, inductive process. Okay. Perfect. Wow. It's giving you a very detailed information. And this is all local, right? It's amazing. Okay. You see that it's not as fast as they're using the closed models. It's also giving information about, you know, um, when to use each of the data analysis strategy. Oh, it's providing even a quick example. That's good. And providing you step-by-step too. Perfect. So you see how the system has given you detailed information. It's giving you information about the difference between them, right? That is good. I really like this one, right? But as I said, it's a little slow compared to the closed models, right? So let's go ahead and ask further questions. So I'm going to ask the system to review the purpose and the research question of my study. And then based on that, decide what I need to do. So the question here is, can you review the study below to determine whether I should use content analysis or thematic analysis? And I provided the purpose of the study and also the research question, right? So let's see. So it's telling me to use thematic analysis and it's now provided me some reason why I should use thematic analysis. Okay. Good. I really like the output there. Yes, it's very straightforward and also giving you tables and you can easily consume that information. So I really like that. I'm going to ask the system a last question. Oh boy, it's really going to detail process. I didn't ask for all these. So it's telling you, you know, the need for data collection. So the data collection, you have to use semi-structured interview, have transcribed, transcribed, web attend. You have to familiarize yourself with the data. You have to develop initial codes, and then you have to set for theme by categorizing the codes. Wow. Giving you detailed steps of how you may conduct the thematic analysis. And then you have to review and refine the themes. Okay. Then you have to define the themes and start writing the findings. Oh boy, it's giving you potential pitfalls and tips. Wow, this is so detailed. So I'm just curious to know the system thinking process, right? Sometimes you have to find out how it came up with the answer. It helps you to make sure that the information is credible. So this is the last question I'm going to ask before we do the qualitative analysis, right? So the last question I'm going to ask is, can you show me your artifacts showing how you came to that conclusion, right? So this one, you know, show me the artifacts like the reasons why you arrived at the conclusion. That thematic analysis is appropriate. So let's see what the system will provide us. So this one, it helps you to know the inner thinking of this model, right? So if you want to know how it arrived at the result, this is the question that you can ask, right? Show me your artifacts. And also that will help me to know how you analyze or you came to that conclusion. If it's taking very long to, you can always stop it, right? Stop generating. So let me stop here. And then we move ahead and talk more about the data analysis. So this is what we're going to use. As you can see here, we have the purpose of the study, the research question. So the purpose of this qualitative study is to explore primary healthcare physician, patient experience about the causes and solutions of burnout. And we have two research questions, one for the causes, one for the solution. And these are the participants, the five participants and their demographics, their age, gender, years of experience, and ethnicity. So that's the information that we have about participants. So we're going to start a new conversation, right? If you want to start a new conversation, you click on chat, you click on this icon, you, and then you're going to, so I'm going to bring the transcript. So I have the five transcripts here, right? And then I'm going to also bring information about the study. Whenever you want the system to analyze your data for you, it's very important to give you a little bit of background information. One of the background information is the purpose of the study, right? And also the research question that you want to address and any other information that will help the system to make informed decisions about your study and also analyze and give you the very good results. So I want the system to go through all the transcripts and identify information that are significant addressing the first research question. So I can say that this is what my study is about. And then I just paste the purpose of the study and I say, can you analyze the attached interview transcript, extracting relevant quotations, addressing the following research question. And then this is a research question. What are the causes of burnout among primary healthcare physicians? Right. You can also say that make sure all quotations are attached. It should take some time because it has to go through all the transcripts and then address the question or to complete the task. So it looks like it's completed processing it. It's giving us the output now. So giving us all the quotations and also the interpretation. It looks like it's doing a very good job in terms of identifying information. Right. That is significant and presented that information to us. And it even went beyond what I'm expecting, providing interpretation based on the information that has been extracted. Right. As we can see, as I said, it takes some time, but the output is good and nobody can assess, right? Nobody can assess your communication with the system. It's localized. You don't have to have access. You don't have to have access to the internet and it's still going to work. So that's a very good one. So when you have this, you can go further and ask the system to develop codes based on the information that is significant. So let's go ahead. Can you develop codes representing quotations you have extracted? So if you go through and come up with the code, right? So what I'm doing right now, we call it prompt chaining. So prompt chaining is where you ask the system one question at a time. And based on the output, you ask for that question. It's just like doing a semi-structured interview, right? So if you want to really get rich information from the system and you want to really review each step, it's good to do a prompt chaining technique, right? You can also try other techniques, maybe contextual prompting, where you provide a lot of context and provide all the information the system needs and the system can go through all the process and to giving you the results, right? One example is that you can say that, can you conduct, use thematic analysis strategies or steps to go through all the transcripts? Right? And then give me the results, right? So the system will take its time and provide you step-by-step. So these are the codes and the system is also provided as the definition of the code. I didn't ask for the definition, but the system has done that. And also the quotations. Wow. This AI tool is very good, right? So as you can see here, it can do a lot of things for you, right? As we are getting to the end of this. Demonstration, please make sure that you share my videos and also you can also subscribe to my channel so that I can provide you more videos that will be useful for your profession or your studies, right? And if you have any questions, please put the question in the comment section and I'll be happy to address them for you. And if you like this video, click on like, it helps me a lot, right? If you do that. So after that, you can even download or you can copy and then paste it on your Word document, right? So this is, you can go on your own and have more conversation, but this is what I have for you. If you want me to go into detail, how to use other open source model, let me know. I can go through and show you and give you more information, but this one is just to let you know that. You have a choice here. You can either use an open source model or closed source model. If you don't feel comfortable using the closed source model, you can use this open source. And the only limitation that it takes some time and sometimes it also depends on your memory capacity. I'm talking about the, your computer, right? So always take these factors into consideration when. Trying to use. To a model locally, and I hope this one was helpful and thank you so much for your time.