How to Run an AI Model on Your Own Laptop

You don’t need a subscription or an internet connection to use AI. With free tools, you can download an AI model and run it entirely on your own laptop. Your prompts stay on your machine, it works on a plane, and there’s no monthly bill.

The models you can run at home aren’t as capable as the very best paid ones, but they’ve improved quickly and are good enough for summarizing, drafting, brainstorming and simple coding. Here’s how to get started in about 15 minutes.

Why run AI locally?

• Privacy: sensitive documents never leave your computer.
• Offline use: works without internet once the model is downloaded.
• No usage fees: after the download, it’s free to use as much as you want.
• Learning: you’ll understand how AI models work much better.

The trade-offs: smaller models make more mistakes, big models need a powerful computer, and you manage the setup yourself. If you’re unsure whether local suits you, our guide to open-weight vs closed AI models explains the bigger picture.

Check your hardware

Memory matters most. These are rough rules of thumb:

Your computer	What to expect
8 GB RAM	Small models; fine for simple tasks, slower on older machines
16 GB RAM	Many popular mid-size models run comfortably
32 GB+ RAM or a strong GPU	Larger, more capable models

A few points help:

• Apple Silicon Macs (M-series) run local models well, because memory is shared between processor and graphics.
• A dedicated graphics card (GPU) on Windows or Linux speeds things up a lot. Its own memory (VRAM) limits which models fit.
• No GPU? Small models still run on the processor alone, just more slowly.
• Disk space: models are big files, often several gigabytes each.

Option 1: LM Studio (easiest, no commands)

LM Studio is a desktop app with a chat window, much like ChatGPT, but running models on your machine. It’s available for macOS (Apple Silicon), Windows and Linux.

1. Download it from the official site, lmstudio.ai, and install it.
2. Open the app and use its model search to find and download a model. It pulls models from Hugging Face, a large public library of AI models.
3. Pick a small model first, load it, and start chatting.

LM Studio can also run a local server that works with the same API format as OpenAI, so apps and scripts built for online AI can talk to your local model instead.

Option 2: Ollama (simple commands, great for developers)

Ollama runs models from the terminal and in the background, and many other apps plug into it.

Install it:

• macOS: download the app from ollama.com/download.
• Windows: download the installer, or run this in PowerShell:

irm https://ollama.com/install.ps1 | iex

• Linux:

curl -fsSL https://ollama.com/install.sh | sh

Run your first model. Browse model names in the library on ollama.com, then:

ollama run <model-name>

The first time, this downloads the model and then opens a chat right in your terminal. Type your question and press Enter. Type /bye to exit.

A few other useful commands:

ollama pull <model-name>   # download a model without starting a chat
ollama list                # see the models you've downloaded

Use it from other apps. Ollama runs a local server at http://localhost:11434, including an OpenAI-compatible endpoint. Local requests don’t need an API key.

Note that Ollama also offers cloud models that run on its servers. Those don’t use your hardware, and your prompts aren’t purely local. For full privacy, stick to models you’ve downloaded.

Choosing a model

Model names usually include a size in billions of parameters (for example, “4b” or “12b”). Bigger usually means smarter but slower and hungrier for memory.

• Start small. A model in the low single-digit billions of parameters is a good first test on most laptops.
• Move up if it runs well. If replies are fast and you have memory to spare, try a larger version.
• Look for “quantized” versions. These are compressed to use less memory with only a small loss in quality. Most local tools use them by default.
• Check the licence before using a model for commercial work.

Some models use a mixture-of-experts design, where only part of the model is active for each request. Learn more in our explainer on what mixture-of-experts means.

Good first tasks for a local model

Summarize the following notes into 5 bullet points:
[paste notes]

Rewrite this paragraph in simpler English, under 80 words:
[paste text]

Local models do best with clear, specific prompts. Our six-part prompt framework works just as well offline.

Troubleshooting

• Very slow replies: try a smaller model, close other heavy apps, or plug in your laptop (battery saving can slow things down).
• Out of memory errors: the model is too big for your machine; choose a smaller or more compressed version.
• Strange or wrong answers: small models make more errors. Double-check facts, as you would with any AI.

Key takeaways

• Free tools like LM Studio and Ollama let you run AI models offline on your own laptop.
• Memory decides what you can run: 8 GB for small models, 16 GB for many mid-size ones, 32 GB+ for larger.
• Start with a small model, then scale up if your machine handles it.
• Downloaded local models keep your data on your computer; cloud options don’t.

Sources: Ollama download page · Ollama quickstart · LM Studio documentation