How to Add AI NPCs to Unreal Engine 5 with Blueprints

Adding an AI NPC to Unreal Engine 5 usually means a marketplace plugin, a speech service, a text-to-speech service and a lot of Blueprint wiring. This guide does it with one plugin: the Neocortex Unreal SDK. You'll build a character players can type or speak to, who answers in its own voice with emotion, and who can trigger gameplay, all from Blueprints, with the C++ version alongside.

What you need

• Unreal Engine 5.0 or later
• A C++ project. If yours is Blueprint-only, add any C++ class through Tools > New C++ Class to convert it. All Neocortex features still work from Blueprints.
• Git, and a free Neocortex account

Step 1: design the character

In the Neocortex dashboard, create the character: a personality, a knowledge base with your lore, a voice, and Action nodes for anything it should be able to do, such as OPEN_GATE. Talk to it in the browser until it sounds right; in-editor testing is free on every plan. Copy the character's id.

Step 2: install the plugin

In your project's Plugins folder, run:

git clone https://github.com/neocortex-link/neocortex-unreal-sdk Neocortex

Then:

1. Add { "Name": "Neocortex", "Enabled": true } to the Plugins list in your .uproject file.
2. Add "Neocortex" to PublicDependencyModuleNames in your module's Build.cs.
3. Regenerate project files, build, and check Edit > Plugins shows Neocortex enabled.

Step 3: add your API key

Open Edit > Project Settings > Game > Neocortex and paste your API key. The same page has an optional transcription language. The key is saved in your project's Config/DefaultGame.ini, so keep that file out of public repositories.

Step 4: a talking NPC in Blueprints

1. In your NPC's Actor Blueprint, add a Neocortex Smart Agent component and an Audio component. In the Smart Agent's Details, set Project Id to your character's id.
2. When the player speaks to the NPC, call Send Message For Audio on the Smart Agent with their text. For text-only replies, use Send Message.
3. Select the Smart Agent and click + next to On Chat. Break the Chat Response Data to get Response (the reply text), Emotion, Action and Metadata. Show Response in your dialogue UI.
4. Add On Audio: connect its Sound Wave to Set Sound on the Audio component, then Play.
5. Add On Error and print the message while you test.

Press Play and send a line. The NPC answers in text and in its own voice.

The same NPC in C++

The plugin's components work the same way from C++:

// MyNpc.cpp: AMyNpc has a UNeocortexSmartAgent* Agent and a UAudioComponent* Voice
#include "MyNpc.h"
#include "NeocortexSmartAgent.h"
#include "Components/AudioComponent.h"

void AMyNpc::BeginPlay()
{
    Super::BeginPlay();
    Agent->OnChat.AddDynamic(this, &AMyNpc::HandleChat);   // UFUNCTIONs declared in MyNpc.h
    Agent->OnAudio.AddDynamic(this, &AMyNpc::HandleAudio);
    Agent->OnError.AddDynamic(this, &AMyNpc::HandleError);
}

void AMyNpc::Ask(const FString& PlayerLine) { Agent->SendMessageForAudio(PlayerLine); }

void AMyNpc::HandleChat(const FNeocortexChatResponseData& Reply)
{
    UE_LOG(LogTemp, Log, TEXT("%s [action: %s]"), *Reply.Response, *Reply.Action);
}

void AMyNpc::HandleAudio(USoundWave* SoundWave) { Voice->SetSound(SoundWave); Voice->Play(); }

void AMyNpc::HandleError(const FString& Error) { UE_LOG(LogTemp, Warning, TEXT("Neocortex: %s"), *Error); }

Create both components in the constructor with CreateDefaultSubobject, and set Project Id on the agent in the Details panel.

Step 5: let players speak

Add a Neocortex Microphone Recorder component to the NPC actor, next to its Smart Agent, which it finds automatically. If you put it somewhere else, set its Smart Agent reference from Blueprint.

1. On a button press, call Start Recording.
2. On release, call Stop Recording. It transcribes the recording and raises On Transcribed on the Smart Agent with the player's words.
3. Connect On Transcribed to Send Message For Audio, so the transcript goes to the character.

The plugin also includes a ready-made audio chat input widget with a hold-to-record button, which records and sends the audio for you, plus widgets for a chat panel, a text input and a microphone picker. On Android, enable permission support and add the RECORD_AUDIO permission, and on iOS set a microphone usage description: the plugin's README has the exact config lines.

Step 6: emotions and actions

• Emotion: each reply carries an Emotion such as Happy, Curious or Annoyed. Switch on it to pick an animation montage or a facial pose. Our guide to emotional AI NPCs has a mapping you can copy.
• Actions: when the character decides to act, the reply's Action holds the keyword, like OPEN_GATE. Switch on the string and run your own Blueprint, after checking the player is allowed.
• Targets: add a Neocortex Interactable component to objects the character should know about, and turn on Include All Interactables on the Smart Agent (with an optional radius filter). When an action aims at an object, that object comes back in Metadata with Is Subject set.
• World events: call Push Event with a priority and a short sentence, like "The bridge collapsed", and it goes with the next message.

Our guide on how to make AI NPCs act on what they say covers how to design actions players can't abuse.

The sample maps

The plugin's Content/Samples folder has maps for text chat, audio chat, chat history and interactables, with sample Smart Agent Blueprints you can copy from.

Common problems

• The plugin won't compile. The project must be C++, and Neocortex must be in your Build.cs dependencies. Regenerate project files after adding it.
• "Request already in flight." One Smart Agent handles one message at a time. Wait for On Chat, or check Is Request Pending before sending.
• No sound. The plugin hands you the Sound Wave in On Audio; it doesn't play it for you. Make sure Set Sound and Play are connected.
• The microphone doesn't start on Android. Check the permission settings from the README, and that the player granted access.

Next steps

Compare this with wiring a language model yourself in how to use ChatGPT in Unreal Engine Blueprints. For the design side, read how to give NPCs unique personalities and how to keep AI NPCs in character. Every tutorial is in the guide to building AI NPCs, and the Unreal integration page has the requirements in one place.

Frequently Asked Questions