Skip to main content

Chaining speech → text → speech

Elys Listen, Elys Mind (LLM) and Elys Speak (TTS) are independent plugins with no shared dependency. They interoperate through a single neutral type: the humble FString. Each stage exposes a tiny, uniform contract:

StagePluginInput (sink)Output (source)
Speech-to-textElys ListenmicrophoneOnText (FString)
Language modelElys MindPushText (FString)OnText (FString)
Text-to-speechElys SpeakPushText (FString)audio

Because the payload is just text, you wire any combination with a single binding — no shared structs, no glue code.

Voice assistant: STT → LLM → TTS

In Blueprint:

  1. Elys STT Component → event On Text → call Push Text on the Elys LLM Component.
  2. Elys LLM Component → event On Text → call Push Text on the Elys TTS Component.

In C++:

// The player speaks → the LLM answers → the answer is spoken aloud.
STT->OnText.AddDynamic(LLM, &UERP_LLMComponent::PushText);
LLM->OnText.AddDynamic(TTS, &UERP_TTSComponent::PushText);

Voice command (no LLM): STT → TTS

STT->OnText.AddDynamic(TTS, &UERP_TTSComponent::PushText); // echo what was heard

Notes

  • OnText fires only on final transcription results (not interim), so the LLM is not spammed with partial text.
  • The same pattern extends to any future text stage (translation, filtering): a node that is both a sink (PushText) and a source (OnText) slots in anywhere.
  • Each plugin is independently sellable and installable; chaining simply requires that the plugins you want to connect are both present in the project.