How it works
One editor, five ways in.
OPAL sits between the inputs a disabled creator already uses and the Unreal Editor, translating intent into editor operations. This page is the technical shape of it: what exists, what is designed, and what each layer is responsible for.
Three layers, one of which is already released
-
Input layer: whatever you already drive
Voice, eye gaze, head tracking, switches, sip-and-puff. OPAL is built around hardware and software disabled creators already own and have already learned. Nobody should have to buy a new device just to use one application.
This layer is designed and not built. It isn’t designed from nothing, though. Owen Kent, who leads OPAL, has already solved a version of this problem in shipped software: Nimbus Adaptive Controller, eleven releases since November 2025, takes whatever input someone can drive and turns it into a controller that games and ground-control software accept. The Unreal Editor is the harder case, because a tool demands precision that a game usually forgives, but the shape of the problem is one this team has shipped against before.
-
Translation layer: grammar, pointing and scanning
An open voice command grammar for panels, modes and exact numeric entry, a fixed vocabulary rather than dictation. A pointing model that pairs gaze for coarse jumps with head tracking to refine and a discrete trigger to confirm. A scanning command palette for single-switch and sip-and-puff users. This is the layer the project builds.
-
Editor layer: UEMCP
UEMCP turns an instruction into real editor work through Unreal’s built-in Python remote execution protocol: UDP multicast discovery, a TCP command channel, and structured JSON back out. No plugin to compile, no project files to modify, and an escape hatch for anything the dedicated tools do not cover.
What each input is responsible for
The design rule is that no modality is asked to do work it is bad at. That is the difference between something usable and something that merely works on paper.
| Modality | What it owns | Why | Status |
|---|---|---|---|
| AI intent | Multi-step tasks stated in one sentence | Best channel for work that would otherwise be dozens of precise operations | Released as UEMCP |
| Voice grammar | Panels, tools, modes, exact numeric entry | Deterministic and fast for named targets and numbers, where an open-ended model is the wrong tool | Designed, not built |
| Eye gaze | Coarse pointer jumps across a large viewport | Faster than any other channel at crossing distance, unusable for fine targets | Designed, not built |
| Head tracking | Refinement after a gaze jump | Steadier than gaze at small scale, slower across distance | Designed, not built |
| Switch, sip-and-puff, mouth noise | Confirmation, and full scanning access | One reliable discrete signal, which is exactly what dwell-based selection lacks | Designed, not built |
The rules we will not trade away
- Never gaze aloneDwell selection produces the Midas-touch problem: everything you look at gets clicked. Confirmation is always a separate, discrete act.
- Substitute, do not emulateWhere precision is the enemy, replace the interaction. Do not drive a mouse pointer through a slower channel.
- Nothing to install inside UnrealUEMCP’s premise is zero engine-side install, and the layer above it keeps that promise. No plugin to compile, no engine version matrix to chase.
- Tested by disabled creators from month threeNot validated at the end, when the design is already fixed and the feedback is expensive to act on.
- Open license, twice releasedA mid-project release and a final one, so the community gets working code halfway through, not just at the end.
What you need to try the part that exists
- Unreal Engine 5.0 to 5.7Including Epic Games Launcher builds.
- Python Editor Script PluginEnabled in the editor, with remote execution turned on. About thirty seconds, once.
- An MCP clientClaude Code, Claude Desktop, or any other client that speaks Model Context Protocol.
claude mcp add unreal -- uvx uemcp
Help us build it, or be one of the first to use it
OPAL is being designed and tested with disabled creators from the start, not validated at the end. If you work in Unreal with alternative input, or you want to, we want to hear how you work.