Skip to main content

Elys Listen

Offline speech-to-text for Unreal Engine 5. Elys Listen captures microphone audio and transcribes it locally with a bundled whisper.cpp backend — no cloud, no network, no API keys. It is Blueprint-first and ships as a fully standalone plugin (no other Elys plugin required).

Watch the demo​

See Elys Listen handle voice activity detection, local multilingual transcription, and push-to-talk in Unreal Engine 5.8.

Watch the Elys Listen demo on YouTube.

Why Elys Listen​

  • 100% offline — audio never leaves the machine. Multiplayer-safe: only the transcribed text needs to be replicated, never the audio.
  • Whisper, multilingual — 99+ languages out of the box (MIT-licensed models).
  • Voice activity detection — speech is segmented into utterances automatically (energy-based VAD with hysteresis and pre-roll); push-to-talk workflows are supported too.
  • Never blocks your game — inference runs on a background worker thread.
  • Ships cleanly — models are staged into packaged builds automatically. See Shipping Your Game.
  • Pluggable backend — IERP_STTBackend lets you swap in other engines later.
  • Neutral text output — emits an OnText (FString) event that can feed your own gameplay systems. An optional suite integration with separately installed Elys Mind and Elys Speak plugins is also documented.
  • Useful before you design a HUD — the bundled runtime UI provides a compact microphone control, subtitles, transcript history, diagnostics, and a complete control panel. See Ready-to-use UI.

Installation​

Elys Listen is awaiting Fab approval and is not yet available to download. Check the product page for release status.

After publication, install Elys Listen from the Fab Marketplace via the Epic Games Launcher (Vault → Add to Project), then enable it in Edit → Plugins → Audio.

Download a Whisper model from Window → Rogue Paradigm → Elys Listen Model Manager, or open Project Settings → Plugins → Elys Listen (STT) and select Manage Models..., pick a backend in Project Settings → Plugins → Elys Listen (STT), and you are ready. See the Setup Guide.

A 30-second example​

UERP_STTComponent* STT = Actor->FindComponentByClass<UERP_STTComponent>();
STT->OnSTTResult.AddDynamic(this, &AMyActor::HandleTranscript);
STT->StartListening();

In Blueprint: add an Elys STT Component, bind On STT Result, call Start Listening. That's it.

At a glance​

EngineUE 5.8 (Win64; Mac/Linux planned)
Backendwhisper.cpp (MIT), multilingual
DependencyAudioCapture (engine plugin) — no external Elys plugin
OutputOnSTTResult (FERP_STTResult) and neutral OnText (FString)

Elys Listen includes STT only. LLM and TTS functionality require separately distributed plugins and are not part of this product.