Voice Control for Game Accessibility: Play More, Press Less

For a player with limited hand mobility, RSI or one working hand, the hardest part of a game is often not the boss: it is the menus, the inventory grid and the camera, all built around quick, precise presses. Neocortex Voice Commands gives those players another way in, matching plain spoken phrases to your game's actions on their own PC, offline and with no audio leaving the device. It is one option among several, next to remapping, toggles and adjustable timing, and it works best when you treat it as a full input method rather than a novelty.

What it looks like

Lanternfall is an original turn-based adventure set in a cliffside fishing town called Gullmouth. Picture a player with RSI in both wrists who plays with a trackball in short sessions. Moving the character is fine. Long trips through nested menus are what hurt.

With voice control switched on in the accessibility settings, a turn sounds like this:

"Open the inventory. Equip the storm lantern. Use a tonic on Bram."

Three commands, and not one trip through the item grid. Bram, the player's companion, is back on his feet. Next: "Rotate the camera left and zoom in." The view swings round to the harbour steps. "Pick up the brass key. Now use it on the chest." The word "it" lands on the key just picked up. Then "end turn", and the tide comes in.

Later, unsure what is still around after a busy fight, the player asks Pell, the old lighthouse keeper who acts as the town's guide. Pell is an AI character:

Player: Pell, what's on screen right now? Pell: You're on the harbour steps. There's a locked chest to your left, the gate straight ahead, and Bram's at half health just behind you. Two gulls. Nothing hostile.

Pell can say this because the game sends the visible objects and characters along with the question. Nothing in that turn replaced the trackball: the player still moves by hand when that is easier, and talks when talking is easier.

How Neocortex makes it work

• Natural phrasing, no command list. "Open my bag", "show the inventory" and "inventory" can all trigger the same action. Players say what they mean instead of studying a cheat sheet first, which helps anyone who tires quickly or plays in short sessions.
• Several actions in one sentence. "Open the inventory and equip the lantern" arrives as two commands, so the player speaks once instead of repeating the talk action for every step.
• Quantities. "Drop three fish" or "buy two ropes" saves the click-click-click of a quantity slider.
• Context. "Pick up the brass key. Now use it on the chest." resolves "it" to the key.
• Push-to-talk or always listening. Always listening suits players who can't comfortably hold a key. Push-to-talk can sit on whatever single input the player already uses well, because the binding is yours to expose in settings.
• On-device, offline and private. Recognition and matching run on the player's PC. No audio leaves the device, nothing depends on a connection, and there is no server lag between speaking and acting.
• Command sets of actors, actions and objects, designed in the Neocortex dashboard with a few example phrases each: menu names, item names, camera moves and common actions.
• Multiple languages. One command set per language, so players can use the language they are most comfortable speaking.

Pell is a Neocortex AI character. Your game sends what is visible as metadata (each entity with an id, a name, a position and properties such as health), and he answers in his own streamed voice. He only describes what your game sends, so the description is exactly as good as that data. Keep it accurate and he is a reliable pair of eyes; leave things out and he will not know about them.

Ideas to build

Menus by voice. "Open settings", "turn subtitles on", "make the text bigger", "load my last save". Menus are where many players get stuck first, and they are the easiest place to start, because every option already has a name your command set can reuse.

Inventory management. "Sell all the fish", "move the potions to the top", "drop two ropes". Grid inventories ask for some of the most precise pointing in a game, and voice turns a minute of dragging into one sentence.

A turn-based game fully playable by voice. In a tactics or card game, nothing is timed, so voice can cover every action: "move the knight to the bridge", "play the frost card in the left lane", "end turn". Build the command set so every action on screen has a spoken equivalent, and the whole game can be finished without a controller.

A guide who describes the screen on request. Take Pell further: on request, a guide character reads out the current objective, summarises what is in the room, or explains what a status icon means, using a knowledge base of your game's help text (one upload on Free, up to 100 on Team). Only on request, so the guide never talks over the game for players who don't need it.

Building it

Neocortex Voice Commands runs inside PC games on the player's machine. The cleanest way to build it is to treat voice as one more input device: each matched command triggers the same input action a button would, so menus, inventory and camera behave the same however the player reaches them.

1. Find the hard parts. Play through a session and note every action that needs precise timing, rapid presses or long menu trips. Those are your first commands.
2. Design a command set in the Neocortex dashboard that mirrors what is on screen: menu and item names as objects, verbs like open, equip, use and drop as actions, and the phrasings players actually use.
3. Add Neocortex Voice Commands to your game and route each matched command into your existing input actions, rather than building a separate voice-only path.
4. Expose the options. Offer both listening modes in your accessibility settings, let players bind push-to-talk to any input, and ask for confirmation before anything destructive, such as overwriting a save.
5. Add a describing guide if it fits your game: a Neocortex AI character that receives the visible entities as metadata and answers "what's on screen?" in its own voice, through the Web API or an engine SDK.
6. Test with the players who will rely on it, and add a command set per language your players speak.

Keep every voice action reachable by other means, and keep remapping in place. Voice helps many players, but not everyone can or wants to speak to play: some have speech differences, some share a room, some just prefer a pad. The strongest accessibility menus offer a stack of options, and voice is a good one to add to the stack, not a replacement for it.

Voice Commands is a licensed SDK: contact us for a Voice Commands license. A guide character like Pell runs on AI characters and the API, which every plan includes, starting with Free.

Accessible voice control is one entry in our AI NPC, ambient AI and voice command use cases. For hands-busy play in a headset read voice commands in VR, and for a character who narrates the game see AI game master and narrator. The voice-controlled gaming page has the overview, and our language support post covers languages. Start with one menu, make it fully speakable, and ask a player who needs it what to cover next.

Checked October 2026.

Frequently Asked Questions