How to build llama.cpp on Windows for your Nvidia GPU

You want to get the most out of llama.cpp on an Nvidia GPU? Then don't rely on the prebuilt binaries — build it yourself for your own GPU architecture. This post shows how to build llama.cpp on Windows with CUDA, and how to pick the right CMAKE_CUDA_ARCHITECTURES for your GPU.

What you need

• Git
• Visual Studio with the Desktop development with C++ workload selected (this gives you the MSVC compiler, Windows SDK and CMake integration)
• The CUDA Toolkit (v13) — with Visual Studio integration checked. This registers the CUDA compiler with CMake, which is what makes the build below work

Install Visual Studio first. The Visual Studio integration in the CUDA Toolkit installer is only offered when Visual Studio is already installed on the machine — check the box in the installer or the build will not find the CUDA compiler.

Find your compute capability

-DCMAKE_CUDA_ARCHITECTURES is the flag that tells the CUDA compiler which GPU generation(s) your build targets. The number comes from the GPU's compute capability, listed on the CUDA GPU page.

The three generations you'll most likely run into:

Not sure which one you have? Run this in a terminal:

nvidia-smi

The output on my local machine:

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 610.88                 KMD Version: 610.88        CUDA UMD Version: 13.3     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                  Driver-Model | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 5060 Ti   WDDM  |   00000000:04:00.0  On |                  N/A |
|  0%   42C    P1             24W /  180W |   12808MiB /  16311MiB |      2%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA RTX PRO 4000 Blac...  WDDM  |   00000000:0A:00.0 Off |                  Off |
| 65%   54C    P3             42W /  145W |   23194MiB /  24467MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

The GPU name is in the Name column (here: an RTX 5060 Ti and an RTX PRO 4000 Blackwell) — look that up in the table on the CUDA GPU page.

One build, multiple GPUs

You can pass several architectures, separated by a comma, so a single build works on more than one GPU. If you use a 5060 Ti (Blackwell) at home and a 3060 (Ampere) at the office:

-DCMAKE_CUDA_ARCHITECTURES="86,120"

Cloning llama.cpp

The repository has moved to the ggml-org organization (older posts may still show the ggerganov URL):

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp

For the rest of the steps I use E:\llama.cpp\repos\llama.cpp as the source directory — adjust to wherever you put the clone.

Checking out a version

On the first run you're already on the latest master. If you want to update to the latest version later:

git fetch origin
git status
git pull origin master

Or, if you want a specific version (say v0.6.0):

git fetch --tags
git checkout v0.6.0

Building

I like to do this from the Visual Studio folder itself, because it comes with the right environment:

Open the llama.cpp folder in Visual Studio (devenv E:\llama.cpp\repos\llama.cpp) and start the Developer PowerShell from there.

Then:

# clean build folder
rm build -r

# configure + build, with CUDA enabled for your GPU (120 here = RTX 50 series)
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES="120"
cmake --build build --config Release -j 12

The -j 12 is the number of parallel build workers — bump it up to your core count if your machine can take it.

When it's done, the executables are in:

build\bin\Release\

Keeping a versioned local build folder

I keep my built executables in a separate folder and rotate the old one aside with a date stamp before every new build:

Set current version aside:

Move-Item "E:\llama.cpp\llama-local-build" -Destination "E:\llama.cpp\llama-local-build_$(Get-Date -Format yyyyMMdd)"

Update current version:

Move-Item "E:\llama.cpp\repos\llama.cpp\build\bin\Release" -Destination "E:\llama.cpp\llama-local-build"

That way E:\llama.cpp\llama-local-build always holds the current build, and every previous one is kept under llama-local-build_yyyyMMdd.

Running

Point llama-server at a GGUF model and you're up:

.\llama-server.exe -m C:\models\model.gguf --port 8080

The server exposes the OpenAI-compatible API on http://localhost:8080 — which is what the first blog post in this series builds on.

To check which version you're running, open http://localhost:8080/props in your browser. The build_info property shows the build number and commit:

"build_info": "b11669-620d2225c"

The b11669 part is the release number — you can look it up on the llama.cpp releases page.

That's it

You now have a llama.cpp build tuned for your own GPU architecture, a versioned local build folder, and the knowledge to extend the build to your other GPUs whenever needed.