AI Conversation Partners for Language Learning Apps and Games

Language apps are good at drills and flashcards, then the learner lands in a real café and freezes because nobody has ever answered them back at normal speed. Neocortex gives your app a conversation partner instead: an AI character who stays in a scene, speaks and listens in more than 20 languages, and nudges the learner through a lesson one exchange at a time.

What it looks like

Passaporte is an original web app for travellers learning European Portuguese. Unit six drops the learner into Café Andorinha, a tiled corner café in an imagined Lisbon street, at a counter run by Dona Lurdes. She is patient, a little brisk in the morning rush, and refuses to switch to English.

The learner taps the mic and tries:

Dona Lurdes: Bom dia! O que vai ser? Learner: Queria um... galão? E um pastel de nata, por favor. Dona Lurdes: Um galão e um pastel de nata, muito bem. É para comer aqui ou para levar?

Her reply comes back tagged as happy, so her avatar smiles and the app flashes a small "nicely ordered". Behind the scenes the lesson's Conversation Flow just passed its first checkpoint, and the reply carried the keyword ordered, which the app uses to tick the first card on the lesson strip.

Next the learner mixes up "levar" and "lavar" and asks whether they can wash the pastry. Dona Lurdes looks confused for a beat, laughs, and rephrases the question more slowly. If they stall, she returns a SHOW_HINT action and the app slides in a card with two useful phrases. The lesson ends when the learner asks for the bill and says goodbye, and the app follows up with five quick vocabulary questions from the unit.

The writer defined one character, one scene and four beats. Every exchange is new each time the learner replays it.

How Neocortex makes it work

• AI characters. Dona Lurdes is designed in the Neocortex visual node editor: personality, backstory, the café she runs, how strict she is about English, her voice and her actions. Teachers can try her in the browser dashboard before the app ships.
• More than 20 languages. The same character design can be repeated for each language your course teaches, with a voice to match.
• Conversation Flows. Start, Checkpoint and End nodes define the lesson's beats: greet, order, handle a follow-up question, pay. Each reply returns a flowState keyword your app reacts to. Flows are available on every plan.
• Voices and speech-to-text. Every line streams in the character's voice, and learners answer out loud. In the browser that runs through the Unity WebGL microphone (over HTTPS) or your app's own mic capture sent to Neocortex speech-to-text.
• Emotions. Each line carries an emotion tag. Happy when the learner gets it, confused when a word lands wrong, so the feedback feels like a person reacting rather than a red cross.
• Actions. SHOW_HINT, SHOW_MENU or PLAY_SLOWER come back with replies and your app decides what they look like on screen.
• Knowledge base. Upload the unit's word list and grammar notes as PDF or TXT so the character steers towards vocabulary the learner has just studied. Free includes 1 upload, Starter 5, Pro 20 and Team 100.
• Quiz engine. Question sets and scoring through the API close each scene with a vocabulary check (Team plan).
• Session memory and usage. Keep the sessionId to resume the learner's session later, and use per-player limits to keep practice costs predictable across a whole class.

Ideas to build

The travel phrasebook trip. Chain short scenes along a route: buying a train ticket in Porto, asking a pharmacist about sun cream, checking in at a guesthouse that has lost the booking. Each scene is its own Conversation Flow, so a learner can repeat the pharmacy until it feels easy, then move on.

The job interview. Business learners practise a German interview with a hiring manager who asks follow-up questions about their last role and their salary expectations. The flow checks off the introduction, a question about experience and the learner's own question at the end, and the quiz afterwards covers the formal phrases they used.

A language game for kids. A talking fox lives in a garden and only speaks Spanish. Children help her find things by naming them, and every correct word triggers a SHOW_PICTURE action with the object popping up. Short lines, a warm voice and lots of happy emotion tags keep it playful.

The VR market. On Meta Quest, built in Unity, the learner walks through a busy market and haggles with stallholders face to face. Lip sync from the Unity SDK moves each stallholder's mouth with their speech, and the same flows and quiz from the web version carry over.

Building it

Web apps call the Web API from any framework, Unity apps for desktop, mobile or WebGL use the Unity SDK, and the same characters work in a VR build on Meta Quest.

1. Pick one scene and one level. A café order at beginner level is enough to start.
2. Create the character in the visual editor: who they are, where they work, which language they speak, a voice and a few helpful actions.
3. Design the lesson as a Conversation Flow with a checkpoint for each thing the learner should manage to say.
4. Upload the unit's vocabulary to the knowledge base so the character uses words the learner knows.
5. Wire the mic and the flow states in your app: speech in, voice out, and a progress step ticked for every flowState keyword.
6. Add a quiz at the end of the scene on the Team plan, or your own exercise screen on any plan.

Set the character's patience deliberately. A partner who rephrases once and then waits works better for practice than one who rescues the learner after every pause.

Language partners are one entry in our AI NPC, ambient AI and voice command use cases. For learning in other subjects, read AI historical figures for museums and VR soft skills training with AI. The education and training page, the language support feature page and our post on language support cover the rest. Start with one café and one checkpoint.

Checked October 2026.

Frequently Asked Questions