Skip to main content

Optional suite integration

Not included with Elys Listen

Elys Listen provides speech-to-text only. It does not include a language model or text-to-speech backend. The STT → LLM → TTS workflow below requires the separate Elys Mind and Elys Speak plugins, installed and enabled in the same Unreal Engine project.

Elys Listen, Elys Mind (LLM) and Elys Speak (TTS) are separate, independently distributed plugins with no shared dependency. They interoperate through a single neutral type: the humble FString. Each stage exposes a tiny, uniform contract:

StagePluginInput (sink)Output (source)
Speech-to-textElys ListenmicrophoneOnText (FString)
Language modelElys MindPushText (FString)OnText (FString)
Text-to-speechElys SpeakPushText (FString)audio

Because the payload is just text, projects that own the other plugins can wire the stages without introducing shared data types.

Elys Listen exposes OnText on both the STT subsystem (primary) and the optional STT component, so you can wire the chain either way.

Optional voice assistant: STT → LLM → TTS​

In Blueprint (component style):

  1. Elys STT Component → event On Text → call Push Text on the Elys LLM Component.
  2. Elys LLM Component → event On Text → call Push Text on the Elys TTS Component.

In C++ (subsystem style — no actors needed):

// The player speaks → the LLM answers → the answer is spoken aloud.
UERP_STTSubsystem* STT = GetGameInstance()->GetSubsystem<UERP_STTSubsystem>();
UERP_LLMSubsystem* LLM = GetGameInstance()->GetSubsystem<UERP_LLMSubsystem>();
STT->OnText.AddDynamic(LLM, &UERP_LLMSubsystem::PushText);
LLM->OnText.AddDynamic(TTS, &UERP_TTSComponent::PushText);
STT->StartListening();

Voice command (no LLM): STT → TTS​

STT->OnText.AddDynamic(TTS, &UERP_TTSComponent::PushText); // echo what was heard

Notes​

  • OnText fires once per completed utterance (after voice-activity detection ends it), and empty / low-confidence results are filtered out — so the LLM is never spammed with partial or junk text.
  • The same pattern extends to any future text stage (translation, filtering): a node that is both a sink (PushText) and a source (OnText) slots in anywhere.
  • This page is an integration recipe, not a feature bundled with Elys Listen.
  • Each plugin must be acquired, installed, enabled, configured, and packaged separately.