API Reference

All types use the ERP_ / UERP_ / FERP_ prefix and are Blueprint-accessible.

UERP_STTSubsystem — the primary API

Speech-to-text is a machine-global concern (one local mic, one local player), so the primary entry point is a GameInstanceSubsystem you drive directly — no actor or component required. The backend auto-initializes from Project Settings at startup, so in most projects you just bind OnText and call StartListening().

UERP_STTSubsystem* STT = GetGameInstance()->GetSubsystem<UERP_STTSubsystem>();
STT->OnText.AddDynamic(this, &UMyThing::HandleTranscript);
STT->StartListening();

Listening session

Member	Kind	Description
`StartListening()`	function → `bool`	Open the mic and begin transcribing
`StopListening()`	function	Stop (transcribes pending audio first)
`IsListening()`	pure → `bool`	Session active?
`FlushUtterance()`	function	Transcribe accumulated audio now
`SetConfig(FERP_STTConfig)`	function	Language + VAD tuning; effective immediately
`GetConfig()`	pure → `FERP_STTConfig`	Current session config
`OnSTTResult`	event `FERP_STTResult`	Each completed utterance (already filtered)
`OnText`	event `FString`	Transcript text — bind to `LLM.PushText` for chaining

Push-to-talk

Member	Description
`SetPushToTalkMode(ERP_PushToTalkMode)`	AlwaysOn / PushToTalk / PushToSilence
`SetPushToTalkActive(bool)`	Key down (true) / up (false); release also flushes
`SetMuted(bool)`	Mute without stopping the session

Backend & models

Member	Description
`IsBackendInitialized()` → `bool`	Model loaded and ready
`InitializeBackend(Config)` → `bool`	Load a model from an explicit backend config
`InitializeBackendWithModelFile(FileName)` → `bool`	Load a model by filename
`OnBackendReady` / `OnBackendInitFailed`	Backend load success / failure (message)
`GetModelInfo()` / `GetMemoryUsage()`	Loaded model description / footprint
`GetModelsDirectory()` / `GetAvailableModelFiles()`	Models folder / `.bin` files present
`GetSupportedLanguages()` → `TArray<FString>`	Language codes of the active backend
`GetProcessingState()` / `GetLastError()` / `GetLastErrorMessage()`	Diagnostics
`GetPendingTranscriptionCount()` → `int32`	Worker backlog (real-time detection)

UERP_STTComponent — optional wrapper

An optional thin convenience component that forwards to the subsystem and re-broadcasts its events. Use it for drop-in Blueprint setups or to keep the chaining graph symmetric with ElysMind/ElysSpeak. It owns nothing.

Member	Kind	Description
`Config`	property `FERP_STTConfig`	Pushed to the subsystem on `StartListening`
`bAutoStartListening`	property `bool`	Start on BeginPlay, stop on EndPlay
`StartListening()` / `StopListening()` / `IsListening()`		Forward to subsystem
`FlushTranscription()`	function	Forward to `Subsystem::FlushUtterance`
`SetConfig()` / `GetConfig()`		Config, pushed to subsystem
`IsSTTReady()` / `GetProcessingState()` / `GetLastErrorMessage()`	pure	Backend diagnostics
`OnSTTResult`	event `FERP_STTResult`	Re-broadcast from subsystem
`OnText`	event `FString`	Re-broadcast from subsystem

FERP_STTResult

Field	Type	Description
`TranscribedText`	`FString`	The recognized text
`Confidence`	`float`	0.0–1.0 (average token probability)
`bIsFinal`	`bool`	Always true with the Whisper backend (utterance-based)
`ProcessingTimeMs`	`float`	Inference time

FERP_STTConfig

Field	Type	Default	Description
`LanguageCode`	`FString`	`en-US`	Language, or `auto` to detect
`MinConfidenceThreshold`	`float`	0.5	Drop results below this
`bUseVoiceActivityDetection`	`bool`	true	Auto-segment speech into utterances; off = accumulate until `FlushTranscription()`/`StopListening()` (push-to-talk style)
`VADActivationThreshold`	`float`	0.02	Smoothed RMS energy that starts speech
`VADDeactivationThreshold`	`float`	0.01	Energy below which audio counts as silence (hysteresis)
`VADEnergySmoothing`	`float`	0.35	EMA factor on the energy (lower = smoother)
`SilenceTimeoutSeconds`	`float`	1.5	Silence that ends an utterance
`MinUtteranceSeconds`	`float`	0.3	Discard shorter blips
`MaxUtteranceSeconds`	`float`	25	Split longer speech into segments
`PreRollSeconds`	`float`	0.4	Audio kept from before speech onset

UERP_STTSubsystem

GameInstanceSubsystem owning the shared backend and the background transcription worker. Auto-initializes from Project Settings at startup.

Member	Description
`InitializeBackend(Config)` → `bool`	Load a model from an explicit backend config
`InitializeBackendWithModelFile(FileName)` → `bool`	Load a model by filename from the models directory
`ShutdownBackend()`	Unload (joins the worker thread first)
`IsBackendInitialized()` → `bool`	Ready state
`OnBackendReady`	Event: backend finished loading
`OnBackendInitFailed`	Event `FString`: load failed (message for UI)
`GetModelInfo()` → `FString`	Loaded model description
`GetModelsDirectory()` → `FString`	Absolute path of `Content/ElysListen/Models`
`GetAvailableModelFiles()` → `TArray<FString>`	`.bin` files present on disk
`GetSupportedLanguages()` → `TArray<FString>`	Language codes of the active backend
`GetMemoryUsage()` → `int64`	Estimated model memory footprint
`GetPendingTranscriptionCount()` → `int32`	Jobs queued on the worker (backlog detection)
`GetProcessingState()` / `GetLastError()` / `GetLastErrorMessage()`	Backend diagnostics

UERP_AudioCaptureSubsystem

Lower-level microphone capture, fanned out to consumers. The STT subsystem is its main consumer; you normally don't touch this directly (drive push-to-talk through UERP_STTSubsystem instead). Capture uses the device's native format (queried from the hardware).

UERP_STTSettings (Project Settings → Elys Listen (STT))

Setting	Description
`bEnableAudioCapture`	Project-wide microphone kill-switch
`bEnableSTT` / `STTBackend`	Enable and choose the backend (Whisper by default)
`STTModelPath`	Model file; stored project-relative, resolved by `ResolveModelPath()`
`DefaultLanguageCode`	Language used at auto-initialization
`WorkerThreadCount`	Inference threads (0 = auto)
`bEnableGPUAcceleration`	Only with a GPU-enabled whisper.cpp build (bundled lib is CPU-only)
`STTModelCatalog`	The model list shown in the downloader
`bEnableVerboseLogging`	Detailed VAD/transcription diagnostics

IERP_STTBackend

Implement this to add a custom STT engine (e.g. Vosk, Moonshine), then select Custom as the backend in settings. Whisper is the bundled implementation (UERP_WhisperBackend). The primary entry point is TranscribeSamples(Samples16kMono, Config, OutResult) — it must be thread-safe (called from the transcription worker).

UERP_STTSubsystem — the primary API​

Listening session​

Push-to-talk​

Backend & models​

UERP_STTComponent — optional wrapper​

FERP_STTResult​

FERP_STTConfig​

UERP_STTSubsystem​

UERP_AudioCaptureSubsystem​

UERP_STTSettings (Project Settings → Elys Listen (STT))​

IERP_STTBackend​