Windows AI · Hardware
GPU vs NPU for AI: What's the Difference?
A GPU is a big, flexible parallel processor that can run AI extremely fast but draws a lot of power; an NPU is a small, specialised accelerator that runs AI inference very efficiently at low power. Both do the matrix math behind AI, but they're tuned for opposite priorities — the GPU for raw throughput, the NPU for efficiency. On a modern Windows PC they don't compete so much as divide the labour, with the CPU orchestrating everything.
If you're weighing an "AI PC" or just trying to understand your spec sheet, this guide explains what each chip is good at, where the trade-offs lie, and — importantly — what genuinely runs on the NPU today versus what still leans on the GPU. If you want the standalone primer first, our explainer on what an NPU is covers the basics.
Reach for the GPU when you want maximum AI speed and don't mind the power draw — big local models, training, heavy bursts. Reach for the NPU when you want efficient AI running quietly for hours — camera effects, background features — without draining the battery.
The short version
Both a GPU and an NPU accelerate the same fundamental workload: the enormous piles of multiply-and-add operations that AI models are built from. The difference is design philosophy. A GPU is a general parallel workhorse that happens to be brilliant at AI; an NPU is a purpose-built AI block that does little else. That single distinction ripples out into everything — power draw, heat, flexibility, and which jobs each is actually used for.
What a GPU does
The graphics processing unit was built to render images: shading millions of pixels and vertices in parallel. That same architecture — thousands of small cores working at once — turns out to be exactly what AI needs, because AI math is massively parallel too. A capable GPU can therefore chew through model inference and training at high speed, and discrete graphics cards pair their cores with fast, dedicated VRAM that feeds them at very high bandwidth.
That combination of many cores and high memory bandwidth is why a discrete GPU is still the tool of choice for demanding local AI. The trade-off is power and heat: a high-end card can draw hundreds of watts under load, which is fine in a desktop but hard on a thin laptop's battery. GPUs are also flexible — the same card renders games, encodes video and runs AI — so they're not wasted silicon when you're not doing AI.
What an NPU does
The neural processing unit is a block of silicon designed for one job: running trained AI models efficiently. Microsoft's Copilot+ PC developer guide on Microsoft Learn describes it as hardware built to execute the deep-learning math operations that make up AI models. Because it's specialised, it can be far more power-efficient than a general processor at the same work — it often runs in lower-precision number formats such as INT8, which are good enough for most inference and use dramatically less energy; the same guide notes many NPU devices support only lower-bit integer math for exactly that reason.
What an NPU trades away is generality and, usually, peak speed on the very largest models. It won't run your operating system, render a game, or match a big discrete GPU's raw throughput. Its whole reason to exist is to run sustained AI features — the kind that need to be on for hours — without lighting up the power meter. That efficiency is why NPUs first appeared in phones and now anchor a new class of Windows laptops.
Throughput vs efficiency
The cleanest way to hold the two in your head is a single axis: throughput versus efficiency. A GPU maximises throughput — get the answer as fast as possible — and accepts high power as the cost. An NPU maximises efficiency — do the AI work using as little power as possible — and accepts that it won't win a raw speed race against a big GPU. Neither is "better"; they're optimised for different questions.
You'll see NPUs (and GPUs) rated in TOPS, trillions of operations per second. It's a handy way to sort accelerators into rough classes, but it's a peak theoretical figure. Real performance also hinges on memory bandwidth, drivers, how well software targets the chip, and the model itself. A high TOPS number tells you a machine is in the AI-capable class; it doesn't guarantee any particular app will feel fast. Treat TOPS as a ballpark, not a benchmark.
For running language models in particular, memory bandwidth — how fast weights can be streamed to the processor — frequently matters more than the headline TOPS figure. A discrete GPU's dedicated VRAM has far higher bandwidth than the shared system memory an integrated NPU draws on, which is a big reason GPUs pull ahead on large local models regardless of TOPS.
GPU vs NPU side by side
| Trait | GPU | NPU |
|---|---|---|
| Built for | Graphics and heavy parallel compute; flexible | AI inference only; specialised |
| Priority | Peak throughput (speed) | Efficiency (performance per watt) |
| Power & heat | High under load | Low; suited to all-day tasks |
| Best for | Large local models, training, fast bursts | Background AI features, small on-device models |
| Memory | Dedicated high-bandwidth VRAM (discrete cards) | Shares system memory with CPU and GPU |
When each is used
In practice the division of labour looks like this. Anything that needs maximum AI horsepower — training or fine-tuning a model, running a large local LLM at speed, generating images quickly — goes to the GPU, because raw throughput and memory bandwidth are what those jobs reward. If you're serious about running big models on your own machine, a discrete GPU with plenty of VRAM is the component that matters; our guide on how much VRAM you need for local AI covers the sizing.
Anything that needs to run efficiently in the background for long stretches goes to the NPU: live camera and microphone effects during a video call, on-device translation, and similar features that would drain a laptop if they ran on the CPU or spun up the GPU all day. The NPU's job is to make those feel free — always available, barely touching the battery.
What actually runs on the NPU today
This is where expectations and reality often diverge, so it's worth being precise. On Windows in 2026, the NPU is used mainly by built-in features and purpose-built apps, not by general chat tools. Microsoft routes on-device features such as Windows Studio Effects, live caption translation and other Copilot+ capabilities to the NPU, and delivers small models to developers through Windows AI Foundry and its Foundry Local runtime, which can target the NPU where a chip and model support it. Chip vendors publish their own NPU-optimised model collections and toolkits as well.
What generally does not run on the NPU yet are the popular third-party LLM runtimes. As of mid-2026, tools such as Ollama, llama.cpp and LM Studio execute on the GPU or the CPU and do not route their workloads to the NPU. So if your mental image of "AI on the NPU" is a local ChatGPT-style chatbot, the reality is that today that chatbot most likely runs on your GPU or CPU, while the NPU handles quieter, built-in AI. That gap is closing as software matures, but it's the honest picture right now.
A laptop can clear Microsoft's 40+ TOPS Copilot+ bar — the Windows 11 specifications page sets it as "an NPU capable of performing 40+ TOPS" alongside 16 GB of RAM and 256 GB of storage — and still be slow at running a local LLM, because those chat tools mostly don't use the NPU and because shared memory bandwidth limits large-model speed. The NPU rating tells you the machine can do efficient on-device AI features — not that it's a fast local-LLM box.
The CPU's role
It's tempting to frame this as GPU versus NPU and forget the CPU, but the CPU is the conductor. It runs the operating system and applications, decides what work goes where, and handles the general logic around any AI task — preparing inputs, moving data, and stitching results back into your app. On PCs without a capable GPU or a supported NPU, the CPU can even run AI models directly, just more slowly. Modern processors increasingly ship all three on one chip precisely so Windows can hand each job to whichever unit fits it best — Microsoft puts it plainly in the same Copilot+ PC developer documentation: the NPU works in alignment with the CPU and GPU, and Windows 11 assigns processing tasks to the most appropriate place.
For the bigger picture of how these pieces come together in Microsoft's new hardware category, see our guide to what a Copilot+ PC is, and for hands-on setup, running local AI LLMs on Windows. You can browse everything in the Windows AI hub.
GPU and NPU aren't rivals — they're specialists. The GPU is your muscle for fast, heavy AI; the NPU is your efficient background worker; the CPU coordinates both. Choose based on the job: throughput points to the GPU, all-day efficiency points to the NPU, and most real machines lean on all three.
Frequently asked
Is a GPU or an NPU better for running a local LLM?
For running a local large language model at a usable speed today, a GPU with enough VRAM is generally the better choice. As of 2026 the popular local runtimes such as Ollama, llama.cpp and LM Studio run on the GPU or the CPU and do not route those chat workloads to the NPU. The NPU shines instead at small, sustained on-device tasks and built-in Windows AI features, where its strength is efficiency rather than raw speed for large models.
What is the main difference between a GPU and an NPU?
A GPU is a large, flexible parallel processor built for graphics and heavy compute that can run AI very fast but draws a lot of power. An NPU is a smaller, specialised block tuned only for the low-precision matrix math behind AI inference, so it runs sustained AI tasks at much lower power and heat. In short, the GPU optimises for peak throughput while the NPU optimises for efficiency, and a modern PC uses both alongside the CPU.
Does the NPU replace the GPU?
No. The NPU does not replace the GPU or the CPU; it adds a third specialist to the team. The CPU handles general logic, the GPU handles graphics and heavy parallel bursts including fast AI, and the NPU handles power-sensitive AI that needs to run quietly for long periods. Windows routes each job to whichever processor suits it best, so they complement rather than compete with one another.
What actually runs on the NPU today?
On Windows the NPU is used mainly by built-in features and purpose-built apps rather than general chat tools. Examples include Windows Studio Effects for camera and microphone enhancements, live caption translation, and other on-device features on Copilot+ PCs, along with small models delivered through Microsoft's Windows AI Foundry and vendor toolkits. General third-party LLM runtimes largely do not target the NPU yet, so much of its everyday work is background AI rather than headline chatbots.
Why do TOPS numbers not tell the whole story?
TOPS, trillions of operations per second, is a peak theoretical figure for an accelerator's math throughput and a rough way to sort chips into classes. It does not account for memory bandwidth, drivers, software optimisation or the specific model being run, all of which strongly affect real performance. A high NPU TOPS rating confirms a machine clears the Copilot+ bar, but it does not by itself mean the device will run a local LLM quickly.
Browse all how-to guides for more practical, jargon-free Windows walkthroughs, or dig into the windows-now.com archive of restored community posts. You might also like our explainers on what an NPU is and what a Copilot+ PC is.