Skip to main content

Chaining speech → text → speech

Elys Listen, Elys Mind (LLM) and Elys Speak (TTS) are independent plugins with no shared dependency. They interoperate through a single neutral type: the humble FString. Each stage exposes a tiny, uniform contract:

StagePluginInput (sink)Output (source)
Speech-to-textElys ListenmicrophoneOnText (FString)
Language modelElys MindPushText (FString)OnText (FString)
Text-to-speechElys SpeakPushText (FString)audio

Because the payload is just text, you wire any combination with a single binding — no shared structs, no glue code.

Elys Listen exposes OnText on both the STT subsystem (primary) and the optional STT component, so you can wire the chain either way.

Voice assistant: STT → LLM → TTS

In Blueprint (component style):

  1. Elys STT Component → event On Text → call Push Text on the Elys LLM Component.
  2. Elys LLM Component → event On Text → call Push Text on the Elys TTS Component.

In C++ (subsystem style — no actors needed):

// The player speaks → the LLM answers → the answer is spoken aloud.
UERP_STTSubsystem* STT = GetGameInstance()->GetSubsystem<UERP_STTSubsystem>();
UERP_LLMSubsystem* LLM = GetGameInstance()->GetSubsystem<UERP_LLMSubsystem>();
STT->OnText.AddDynamic(LLM, &UERP_LLMSubsystem::PushText);
LLM->OnText.AddDynamic(TTS, &UERP_TTSComponent::PushText);
STT->StartListening();

Voice command (no LLM): STT → TTS

STT->OnText.AddDynamic(TTS, &UERP_TTSComponent::PushText); // echo what was heard

Notes

  • OnText fires once per completed utterance (after voice-activity detection ends it), and empty / low-confidence results are filtered out — so the LLM is never spammed with partial or junk text.
  • The same pattern extends to any future text stage (translation, filtering): a node that is both a sink (PushText) and a source (OnText) slots in anywhere.
  • Each plugin is independently sellable and installable; chaining simply requires that the plugins you want to connect are both present in the project.