How to Run an LLM Locally in a Unity Game

Running a language model locally in Unity means the NPC's brain lives on the machine instead of a server: no internet needed, nothing leaves the device, and no per-message cost. This guide shows the fastest way to try it, prototyping with Ollama and the Neocortex Ollama Support package, then explains what shipping a local model to players really involves.

Three ways to run a model locally

Open-source packages such as LLMUnity take the embedded route with llama.cpp, leaving the character logic to you. The rest of this guide covers the Ollama route step by step, then the on-device runtime.

Prototype with Ollama in Unity

Ollama downloads and runs open models on your computer behind a small local server. The Neocortex Ollama Support package connects Unity to it, with a model picker and a ready-made chat sample.

Step 1: install the packages

You need Unity 6 with the Neocortex Unity SDK installed, since the Ollama package uses its chat UI. Then add the Ollama package from git URL in the Package Manager:

https://github.com/neocortex-link/neocortex-ollama-support.git

Step 2: install Ollama and pick a model

Open Tools > Neocortex > Ollama Support. The window walks you through installing Ollama and choosing a model to download. Start small: models of a few billion parameters answer quickly on a typical gaming GPU, while bigger ones need much more memory.

Step 3: run the chat sample

In the Package Manager, open the Ollama Support package's Samples tab and import the Chat Sample. Press Play, pick the model from the dropdown and type. The NPC answers from the model on your own machine.

Step 4: call it from your own code

The sample's OllamaRequest is all you need to wire a local model into your own NPC:

using UnityEngine;
using Neocortex;
using Neocortex.Data;

public class LocalNpc : MonoBehaviour
{
    private OllamaRequest request;

    private void Start()
    {
        request = new OllamaRequest { ModelName = "MODEL_NAME" }; // a model you downloaded with Ollama
        request.AddSystemMessage("You are Brannoch, a gruff village blacksmith. Answer in one or two sentences.");
        request.OnChatResponseReceived += OnReply;
        request.Send("Is the north road safe?");
    }

    private void OnReply(ChatResponse response) => Debug.Log(response.message);
}

The request keeps the conversation history itself, so each Send continues the same chat.

What the Ollama route doesn't cover

It's a prototyping tool, and it's text chat only:

• No voice. Speech recognition and text-to-speech aren't part of it.
• No emotions or actions. You get plain text back.
• Players would need Ollama. Every player would have to install it and download a model, which almost nobody will do.

When the prototype proves the idea, the same Unity SDK talks to Neocortex characters in the cloud, with voice, emotions, actions and a visual editor, and no model on the player's machine. Our Unity AI character tutorial covers that setup.

What shipping a local model really costs

Local inference moves the cost from your server bill to the player's hardware:

• Memory. The model has to fit next to your game in VRAM. A 3B model needs about 3 GB on our runtime, a 2B model under 1 GB.
• Download size. Models add gigabytes to the install.
• Quality. Small models are less capable than large cloud models, so characters need tighter writing and narrower knowledge.
• Platforms. WebGL can't do it, and most phones can't spare the memory.

The Neocortex on-device runtime

For studios that need characters on the player's device, with voice, Neocortex offers an on-device runtime as a Custom plan. On a laptop RTX 3080 it returns its first token in 0.06 seconds and starts speaking in about one second, with a 3B model in about 3 GB of VRAM, and speech recognition and voice running on the CPU. Contact us to plan a deployment, and see local and on-premise AI NPC platforms for how the options compare.

Video tutorial

The video walks through the Ollama Support package in Unity.

Next steps

For the speed side of the decision, read how to get AI NPC voice replies under a second. For the cloud route with no setup on players' machines, see how to make NPCs talk with ChatGPT in Unity, which compares calling an API yourself with the Neocortex SDK. Every tutorial is in the guide to building AI NPCs.

Frequently Asked Questions