How to Run Llama Locally on Windows

If you are completely new to local models, it is worth reading our broader overview of how to run local AI models on Windows first; this guide zooms in on the Llama family specifically.

What Llama is, and why run it locally

Llama is a series of large language models developed by Meta and released with open weights — meaning the trained model files are published for anyone to download, run and build on, under the Llama Community License. That licence is not a fully unrestricted open-source licence: it carries some terms, including an extra permission requirement for the very largest companies — the Llama 4 Community License Agreement spells that out in its Additional Commercial Terms clause. For an individual or a typical business, though, it means you can fetch a Llama model and run it on your own machine without an account or a subscription.

The Llama line has grown into several generations and sizes. Recent releases include compact dense models in the 1B and 3B range aimed at laptops and phones, mid-size models around 8B, larger dense models in the 70B class, and newer mixture-of-experts (MoE) designs — Llama 4 Scout and Maverick — that activate only part of a much larger network per token. For running on a single home PC, the small and mid-size dense models are the realistic targets; the biggest MoE models are designed for far more memory than a typical desktop has.

Why bother running Llama yourself instead of using a hosted chatbot? The reasons are the same ones that make any local model appealing: privacy, because your prompts never leave the device once the file is downloaded; offline use, because no connection is needed after the one-time download; no metered cost, because you are not paying per message; and permanence, because the exact version you keep cannot be deprecated or changed under you. The honest trade-off is that a Llama model small enough to run at home will generally be less capable than the largest frontier cloud services — but for drafting, summarizing, coding help and questions about your own files, a good local Llama is more than enough.

Choosing a size for your hardware

The single biggest factor in what runs well is memory — specifically how much GPU VRAM you have, with system RAM as the fallback. A useful rule of thumb is that a model needs roughly as much memory as its file size on disk, plus some headroom for the conversation context. Quantized versions (more on those below) shrink both. Here is a rough mapping to help you pick:

If a model is larger than your VRAM, both runners can offload part of it to system RAM. It still works — it just runs more slowly, because the CPU-held layers are the bottleneck. The practical advice is the same one we give everywhere: start with the smallest size that does the job, confirm it feels responsive, and only move up when you hit a quality ceiling. For a deeper look at the memory maths, see our guide on how much VRAM you need for local AI.

Quantized GGUF versions explained

When you browse Llama models in a desktop runner you will see file names ending in .gguf and labels such as Q4_K_M or Q5_K_M. Two things are going on there.

GGUF is the file format most local runners use to package a model — the weights plus the metadata needed to load it. It is what Ollama and LM Studio consume under the hood.

Quantization is the compression. A model is originally stored at high precision, which is accurate but large. A quantized build stores those numbers at lower precision — 4-bit, 5-bit, 6-bit and so on — which shrinks the file and the memory it needs, at a modest cost in quality. The Q number is that level: lower is smaller and faster, higher keeps more quality but takes more room. A 4-bit quant such as Q4_K_M is the usual recommended starting point because it is the sweet spot between quality and fitting on ordinary hardware. If you have memory to spare and want a little more polish, a 5- or 6-bit build is a reasonable step up.

Run Llama with Ollama

Ollama is a small native Windows app that runs models from a command line and serves them through a local API. It is the favourite for anyone comfortable in a terminal or wanting to wire a model into other tools. If you have not installed it yet, follow our step-by-step guide to installing Ollama on Windows; the short version is below.

1. Download the Windows installer from the official Ollama site and run it. There are no extra dependencies to add.
2. Open Windows Terminal or PowerShell.
3. Pull and run a Llama model in one command with ollama run, documented in Ollama's CLI reference. Ollama downloads it the first time, then drops you into a chat:

   ollama run llama3.2

   Swap the tag for another size or generation to try something else; the same command re-uses the download after the first time.
4. Type your first prompt and press Enter — something like "Summarize the pros and cons of running AI locally in three bullet points" is a good sanity check. To leave the chat, type /bye.
5. Manage what you have downloaded:

   ollama list
   ollama rm llama3.2

Because Ollama keeps its models on disk, the second time you run the same tag it starts almost instantly. Each model is several gigabytes, so keep an eye on free space and remove ones you no longer use.

Run Llama with LM Studio

Prefer to click rather than type? LM Studio is a desktop application with a built-in model browser and a chat window. It needs no terminal and no Python.

1. Download LM Studio for Windows from its official site and install it.
2. Use the search to look for Llama. LM Studio lists builds from public model hubs and shows a fit hint for each quantization against your machine's memory.
3. Pick a build that fits — a Llama in the 7–8B range at 4-bit is a good first choice — and click Download.
4. Open a new chat, load the model, and start typing. Everything runs locally; nothing you type is uploaded.

Not sure which of the two tools suits you? Our comparison of Ollama vs LM Studio weighs them side by side. Many people keep both: LM Studio for browsing and casual chat, Ollama for scripting and API access.

Using the local API

One of the best reasons to run Llama with Ollama or LM Studio is that both expose a local HTTP API, so your own scripts, editors and desktop apps can talk to the model without any of the conversation leaving your PC.

Ollama runs a local server automatically. By default it listens only on your own machine, and you can send it a prompt with a simple request — Ollama's API documentation gives the default address as http://localhost:11434/api with a /api/generate endpoint that takes model and prompt:

curl http://localhost:11434/api/generate -d "{\"model\":\"llama3.2\",\"prompt\":\"Say hello in one short sentence.\"}"

LM Studio, meanwhile, can start a local server — by default on http://localhost:1234 — that speaks the common OpenAI-style API format (with endpoints such as /v1/chat/completions), which means many existing tools and libraries built for that format can point at your machine instead of a cloud endpoint by changing the base URL — exactly what LM Studio's OpenAI-compatibility endpoint documentation describes. This is how people wire a private Llama into a code editor, a note-taking app or a small automation — the model answers, and the data stays home.

Performance tips

A few practical adjustments make a local Llama noticeably nicer to use:

• Update your GPU drivers. Both runners accelerate on a current NVIDIA GPU automatically, and modern AMD GPUs and some integrated graphics are increasingly supported. Out-of-date drivers are a common reason a model falls back to the slower CPU path.
• Fit the model in VRAM. The single biggest speed win is choosing a size and quantization that sit entirely in your GPU's memory. A smaller model that fits will usually feel faster and more pleasant than a bigger one that spills into system RAM.
• Mind the context length. Longer conversations and larger documents use more memory and slow generation. If replies crawl during a long session, starting a fresh chat frees that context back up.
• Close memory-hungry apps. Games, video editors and dozens of browser tabs compete for the same VRAM and RAM. Freeing them up gives the model more room.
• Prefer the right size over the biggest size. For everyday drafting and coding help, a responsive 8B often beats a sluggish 70B you can barely load. Match the model to the task.

For the wider picture — NPUs, Copilot+ PCs and Microsoft's own developer stack — see our overview of Windows AI, which puts local models in context alongside the built-in features Windows ships with.