Run Free, Private AI Locally With Ollama (Full Transcript)

[00:00:00] Speaker 1: In this video, I'll show you how to use Ollama to run powerful AI models directly on your own computer, completely free. You can chat with AI, analyze documents, and even connect it to other apps for coding and automation. And because everything runs locally, you can use it without an internet connection and your data stays on your computer. I'm Kevin, and let's dive in. First, head to the following website. You'll find a link right here at the bottom of the screen. At the top of the page, let's click on download. Ollama works on Mac, Linux, and Windows. I'll be using Windows 11 for this tutorial, so I'll select Windows. Now, you'll see a command that you could run in PowerShell to install Ollama, but there's an even easier way. Just below, click download manually. This downloads the installer directly to your computer. Once the download finishes, open the installer and follow the steps to install Ollama. Once it finishes installing, you'll see this welcome screen. Click continue to get started. Next, Ollama asks you to create an account. You can sign up to access larger models that run in the cloud, but we're going to run AI directly on our own computer. So, right down below, I'll select no thanks. I'll use Ollama locally. That way, we don't need an account or a subscription. Now, you might see a screen asking you to run a command in Terminal. If you do, you can simply close Ollama and reopen it from the start menu. This brings us to the main Ollama app. By default, we're on the apps page. Here, we can connect Ollama to other applications. As an example, here we have Cloud Code, Codex, and also OpenCode. We'll explore one of these integrations later. For now, in the top left-hand corner, let's open up the sidebar and then right down here, let's click into a new chat. Then, up on top, let's close the sidebar. Here we are in Ollama's chat interface. By default, Ollama has selected a cloud model, but we want to run AI locally. So, let's click the model dropdown. You'll notice several models to choose from. Models with a cloud icon run on remote servers and require an account. You also have models with a download icon, and these run directly on your computer. And for those, we have three different models to choose from. First, we have Quint 3.54b. This is smaller and faster and works on more computers, but it may struggle with complex tasks. We also have larger models like Gemma 4, and here we have Quint 3.8. These can handle more demanding tasks, but they also need more memory and processing power. Let's start with Quint 3.54b. It's developed by Alibaba, and it's a good option for getting started, even if you don't have the most powerful computer. Now, let's prompt Ollama for three creative, low-cost ideas to attract customers to a new cookie shop. Then, right over here, let's send that. Since this is our first time using the model, Ollama needs to download it. Once the download finishes, it'll generate a response. And there we go. You just ran your first AI model directly on your computer, completely free. Now, there's a lot of information here. Let's make it a little easier to digest. So, right down below, I'll ask Ollama to summarize those three ideas as short bullet points with just one sentence per idea. Then, over here, I'll send that. That took about five seconds. Ollama remembers the earlier response, so I don't have to repeat any of the details. You can continue asking follow-up questions to refine the results. Ollama also saves your conversations, so you can come back to them later. To see your chat history, in the top left-hand corner, click on the sidebar icon. And here, you can reopen previous conversations and pick up where you left off. To start a new conversation, right up here, simply click on the new chat icon. You can also use Ollama to analyze documents. I have a PDF with customer feedback for my small business, The Kevin Cookie Company. I'll drag and drop it into Ollama. Now, let's ask Ollama to identify the three most common customer complaints and recommend one improvement for each. Over here, I'll send it. Ollama will now analyze a document using the AI model running locally on my computer. This is especially useful when you're working with files that you don't want to send to a cloud AI service. And there we go. Ollama identified the most common complaints. Here, it references specific customer comments, and it also recommends a way to address each issue. And all of this happened locally on your computer without sending the document to the cloud. In fact, once you've downloaded a model, you could use it even without an internet connection. So far, we've used Ollama to chat with AI and analyze documents, but you could also connect Ollama to other applications and have AI actually create and edit files on your computer. Let's see how this works with OpenCode, an AI coding assistant that can help you build websites and applications. You'll find a link at the bottom of the screen. Right up on top, let's click on Download, and then let's get the OpenCode desktop app. Choose your operating system. I'm using Windows, so over here I'll click on Download. Go ahead and download, install, and then open it. Once you launch OpenCode, you'll land in a new project. Now, right down here, you can click on this to change the project or create a new one. Right over here, you can also choose a specific folder where OpenCode can work with your files. Next, let's click on the model selector. Now, notice that OpenCode automatically detects Ollama in the models that we've already downloaded. I'll select Quen 3.54b. This is the same model that we used earlier. Here's how the two tools work together. Ollama runs the AI model locally, and OpenCode gives it the tools to create and edit files on your computer. Let's put it to the test. Right here in the prompt field, I'll ask OpenCode to create a website for the Kevin Cookie Company, complete with a cookie menu, prices, and a contact section. Now that I've entered the prompt, right over here, let's send that. Now, we can see OpenCode creating the website files directly on my computer using Quen running locally through Ollama. It looks like it finished generating the website. Let's now open up File Explorer and navigate to the project folder to open the website. There's the index.html file. I'll double click on that, and there we go. We have a working website. The cool thing is the AI didn't just generate this code in a chat. It created actual files on my computer, and we didn't have to pay for AI usage at all. And that's how you can get started with Ollama. Pretty cool that you could run AI directly on your computer and have it create and edit files. Thanks for watching, and I'll see you in the next one.