AI Residents for Metaverse Worlds That Feel Lived In

Most social worlds are only as lively as the number of people logged in, so a visitor who arrives at a quiet hour walks through an empty plaza and leaves. Neocortex lets you fill that plaza with AI residents who talk among themselves, notice when someone new turns up, and answer anyone who speaks to them. The world has a pulse even when the server count is low.

What it looks like

Lantern Quay is an original browser world: a harbour plaza at permanent dusk, strung with paper lights, where people meet before heading off to the galleries and game rooms around it. At the edge of the water sits the Drift Café, a kiosk run by Wren, a barista who knows everyone's order and pretends she doesn't. Next door, behind the bar of the Tidewater, works Teodor, dry and unhurried, who thinks coffee after six is a moral failing.

A visitor called Kai steps off the ferry. Nobody talks to him yet, but the world sends an event that a newcomer has arrived, and the next ambient turn picks it up:

Wren: New face on the dock. Ten coins says he orders a flat white. Teodor: Ten coins says he has the sense to come here instead. Wren: Hi there! Don't listen to him, he's been polishing the same glass since the lights came on.

Kai laughs and asks Wren what's good. She answers him directly, and her reply carries a SERVE_DRINK action aimed at Kai's avatar, which the world plays as a cup sliding across the counter. Ten minutes later the ferry docks again and the game adds Captain Henrik, the ferry's skipper, to the cast. Teodor pours him his usual without being asked. Wren wants to know whether the fog is coming in.

When Kai logs out, the conversation keeps its thread. The next visitor finds Wren and Teodor still arguing about the bet.

How Neocortex makes it work

• Group chat puts up to six residents in one conversation (Pro and Team plans), each with its own personality, voice and knowledge.
• Ambient turns are group chat requests with no visitor message. The residents carry on among themselves and the response lists each line, who said it and with which emotion.
• An AI director picks the next speaker. When a beat needs a particular resident, such as the captain announcing departure, force it with speakerCharacterId.
• A dynamic cast follows the list of characters you send. Add Captain Henrik and the others greet him; drop him when the ferry leaves and they mention he's gone.
• Events carry what just happened in the world: a newcomer arrived, the fireworks started, the gallery opened a new show. Residents talk about this world instead of drifting into small talk.
• Actions ride along with lines, so Wren can serve a drink and Teodor can dim the bar lights while they speak.
• Session memory. Keep the returned sessionId for each spot in the world and send it with every turn, so the café's conversation continues across visitors.
• Voices and speech-to-text. Every line streams in the resident's own voice. Visitors talk through the Unity WebGL microphone (over HTTPS) or your own mic capture sent to Neocortex speech-to-text.
• Usage controls. Send a playerId with each visitor's messages, and per-player limits keep a busy world's costs predictable.

Ideas to build

The nightclub with a DJ host. A DJ character introduces each track, banters with the bouncer during the quiet moments, and returns a CHANGE_TRACK action when a visitor shouts a request. Push an event when the dance floor fills up and the DJ hypes the crowd; force the DJ as speaker when the headline set starts.

A fantasy town in a browser MMO. The blacksmith, the herbalist and the town crier share a square. Between player visits, ambient turns let them gossip about the raid that failed last night, fed by real events from your server. A player who lingers learns which dungeon is worth the trip.

The brand world. A sportswear brand builds a rooftop court in the browser. Two residents, a coach and a sneaker collector, chat about the new release in ambient turns and answer questions from a knowledge base of product notes. When the coach mentions a shoe, an action opens its 3D model for the visitor.

The residents' meeting. In a small community world, let the visitor playAs a persona character, the new tenant at a residents' meeting, so the others address them by that role while the group argues about the broken lift.

Building it

The Unity SDK covers WebGL builds and wraps ambient turns in a group director component with a Continue() call. Worlds built on any other web stack call the Web API directly, ideally from their own server.

1. Create the residents in the Neocortex visual editor: personality, backstory, what each knows about the world and the others, a voice, and the actions they may return.
2. Rehearse in the dashboard with the group preview's "Start scene" and "Continue" buttons until the dynamic between them feels right.
3. Give each spot in your world its own session. The café, the bar and the dock keep their own sessionId.
4. Request ambient turns when a visitor enters the area or on a slow timer, with events for anything that just happened.
5. Play the lines and run the actions on every visitor's screen, so everyone nearby hears the same conversation.
6. Route visitor speech into the same session and change the cast as residents come and go.

An ambient turn from the café, with the captain just added:

const res = await fetch("https://api.neocortex.link/v3/chat", {
  method: "POST",
  headers: {
    "x-api-key": apiKey, // from your own server config, never hard-coded
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    characterIds: [wren, teodor, henrik], // Henrik just stepped off the ferry
    sessionId: cafeSession,               // one session per spot in the world
    maxTurns: 2,
    events: [{ priority: 1, content: "A visitor called Kai has just arrived on the dock" }],
    // no "message": the residents talk among themselves
  }),
});
const { sessionId, messages } = await res.json();
broadcastLines(messages); // your own helper: voices and chat bubbles for everyone nearby

Make the request from your server rather than the browser, so your API key never ships to visitors, and pace the turns. Two residents trading a line or two whenever someone walks by reads as life; a nonstop stream reads as a radio nobody can switch off.

Metaverse residents are one entry in our AI NPC, ambient AI and voice command use cases. The game side of the idea lives in ambient NPC conversations, and for organised gatherings read AI hosts for virtual events. Browser specifics are in AI NPCs for WebGL and browser games, and the dynamic characters page and group chat docs cover the rest. Start with two residents at one counter and see who visitors talk to.

Checked October 2026.

Frequently Asked Questions