How to run an AI model locally on your laptop — in three commands

It feels like a neural net lives somewhere in the cloud that you knock on over the internet. Here's the surprise: a decent model can run right on your laptop. No internet, no API key, free per request — and your data never leaves. And it's literally three commands. Let's do it step by step.

What you'll need

Not much: an ordinary laptop and Ollama — a free, open-source tool that handles all the dirty work (downloading, compressing the model, running the server). All you do is talk to the model.

The main question is how much RAM you have. That decides which model fits:

| Model size | RAM needed | Runs on | |---|---|---| | 3B (small) | ~4 GB | almost any laptop | | 7–8B (medium) | ~8 GB | a regular laptop, comfortably | | 70B (large) | 48–64 GB | a powerful workstation only |

"B" here is billions of parameters — roughly "brain size." For a beginner on a regular laptop, 3B–8B is ideal — that's what we'll use.

Step by step

1. Install Ollama. Go to ollama.com, download it for your system (macOS, Windows, or Linux), and install it like any app — one click. It quietly starts a background service.

2. Pull a model. Open a terminal and type:

   ollama pull gemma3:4b
   

   This pulls Gemma 3 at 4 billion parameters — a file of about 3.3 GB. It downloads once and sits on your disk. Want a different one? Browse the library at ollama.com/library — hundreds of models (Llama, Qwen, Mistral, and more); same command, just swap the name.

3. Run it and chat. One command:

   ollama run gemma3:4b
   

   A prompt appears — type your question straight into the terminal, like a chat, and the model answers. To exit, type /bye.

4. Check it's really offline. Turn off Wi-Fi and ask again. The model answers as if nothing happened — because it lives entirely on your disk, not on someone else's server.

What you get

On your laptop: your own personal ChatGPT-like chatbot that works with no internet and free per question. Everything you type stays on your computer — handy for personal notes, drafts, and work data you'd rather not hand to the cloud.

Which model to pick

Simple rule: bigger models are smarter — and eat more memory. For a laptop, grab 3B–8B; they're snappy and good for plenty. Large ones (70B and up) are noticeably smarter but need a workstation.

Ollama serves already-compressed (quantized) versions — compression shrinks a model several times over with almost no quality loss, which is why serious models fit on a laptop at all. And now that you get the mechanics, it's worth honestly weighing a local model against a cloud one — each has its place.

Is it free forever?

Yes, running it is free: the models are open-source, there's no per-request charge, no limits. You pay only in electricity and, once, in hardware (you need the RAM). No token bills like cloud services.

Is a local model worse than ChatGPT?

A small one — yes, noticeably weaker than the top cloud models: it's smaller and knows less. But for many tasks (drafts, summarizing, simple code, private notes) it's more than enough. The upsides — privacy, offline, zero per-request cost — often matter more than a few percent of "smarts."