wifi-densepose/v2/crates/homecore-assist

homecore-assist

Voice-activated intent recognition and execution pipeline for HOMECORE with Ruflo agent bridge (P2).

P1 scaffold: intent recognition via regex patterns, 5 built-in intent handlers (turn on/off, set brightness, cancel), and Ruflo runner trait surface. Real tokio::process subprocess integration (P2) allows orchestration with Ruflo agents for complex multi-step actions.

What this crate does

homecore-assist is the voice/NLU gateway for HOMECORE. It takes natural language utterances, recognizes which intent they represent, and executes the appropriate action. It provides:

• IntentRecognizer trait — abstraction for matching utterances to intents
• RegexIntentRecognizer — P1 built-in; uses regex patterns (HA classic style)
• IntentHandler trait — abstraction for handling recognized intents
• 5 built-in handlers — HassTurnOn, HassTurnOff, HassLightSet, HassNevermind, HassCancelAll (mirrors HA's classic intents)
• RufloRunner trait — abstraction for delegating complex actions to Ruflo agents
• NoopRunner — P1 stub; real tokio::process subprocess integration in P2
• AssistPipeline — wires utterance → recognizer → handler → response

Each component is trait-based so recognizers can be swapped (regex in P1, semantic embeddings in P2) without changing the pipeline.

Features

• Regex pattern recognition — utterance matching via compiled regex (P1)
• 5 built-in intents — Turn On, Turn Off, Set Brightness, Nevermind, Cancel All
• Intent entities + slots — recognized patterns capture entity names and parameters (e.g., "turn on light.kitchen" → entity: light.kitchen)
• Intent responses — structured response with optional text, card (tile data), and conversation context
• Ruflo agent bridge — submit complex intents to Ruflo agents for multi-step workflows (P2 subprocess)
• Trait-based recognizers — pluggable: RegexIntentRecognizer (P1), SemanticIntentRecognizer (P2, ruvector embeddings)
• Trait-based handlers — extensible: built-in HA-mirroring handlers + custom handlers
• No external STT/TTS — this module handles NLU only; STT/TTS via homecore-api or external service

Capabilities

Capability	Type	Method	Notes
Recognize intent	Recognizer	RegexIntentRecognizer::recognize(utterance)	Returns Intent enum or error
Handle intent	Handler	IntentHandler::handle(intent, context) → service call	Execute service, set state, or defer to Ruflo
Call Ruflo agent	Runner	RufloRunner::run(intent, opts) (P2)	Subprocess with JSON request/response
Build response	Response	IntentResponse::new(text, entities, card)	Conversational response + optional card data
Run pipeline	Pipeline	AssistPipeline::process(utterance)	Full utterance → recognizer → handler → response

Comparison to Home Assistant

Aspect	Home Assistant	homecore-assist
Intent framework	HA Assist pipeline (Python)	Rust async trait-based pipeline
Recognizer type	Regex (classic) + ML sentence transformer (2024+)	Regex (P1); semantic embeddings (P2)
Built-in intents	HassTurnOn, HassTurnOff, HassLight*, etc.	5 core intents mirroring HA classic
Custom intents	YAML + Python script integration	Trait + handler registration
Agent orchestration	N/A (HA has no agent framework)	RufloRunner + subprocess bridge (P2)
STT/TTS	Via conversation integration + webhooks	Separate; HOMECORE-ASSIST handles NLU only
Slot extraction	regex groups + sentence-transformers	Regex groups (P1); ruvector embeddings (P2)
Response format	Text + TTS synthesis	Structured IntentResponse with card data

Performance

• Intent recognition latency — < 10 ms per utterance (regex compilation cached)
• Handler execution — < 20 ms per intent (service call latency dominates)
• Ruflo agent subprocess (P2) — ~500 ms per agent call (process spawn + IPC overhead)
• Memory overhead per intent — ~500 bytes (Intent struct + handler state)
• Concurrent utterances — 100+ per second on single machine (tokio task per utterance)
• No per-crate benchmarks yet — a follow-up issue tracks baseline measurements

Usage

Regex intent recognition (P1):

use homecore_assist::{RegexIntentRecognizer, IntentName, IntentRecognizer};

#[tokio::main]
async fn main() {
    let mut recognizer = RegexIntentRecognizer::new();
    
    // Register patterns
    recognizer.register(IntentName::HassTurnOn, r"turn (?:on|up) (?:the )?(\w+)").unwrap();
    
    // Recognize utterance
    let intent = recognizer.recognize("turn on the kitchen light").await.unwrap();
    println!("Intent: {:?}", intent.intent_name);
    println!("Entities: {:?}", intent.entities);
}

Built-in handler (P1):

use homecore_assist::{HassTurnOn, IntentHandler, Intent, IntentResponse};
use homecore::HomeCore;

#[tokio::main]
async fn main() {
    let homecore = HomeCore::new();
    let handler = HassTurnOn::new(homecore);
    
    let intent = Intent {
        intent_name: IntentName::HassTurnOn,
        entities: vec![("entity_id".to_string(), "light.kitchen".to_string())].into_iter().collect(),
        slots: Default::default(),
        ..Default::default()
    };
    
    let response = handler.handle(&intent).await.unwrap();
    println!("Response: {}", response.text.unwrap_or_default());
}

Full pipeline (P1):

use homecore_assist::AssistPipeline;
use homecore::HomeCore;

#[tokio::main]
async fn main() {
    let homecore = HomeCore::new();
    let pipeline = AssistPipeline::new(homecore);
    
    let response = pipeline.process("turn on the kitchen light").await.unwrap();
    println!("Assistant: {}", response.text.unwrap_or_default());
}

Relation to other HOMECORE crates

homecore-assist (intent pipeline + Ruflo bridge)
├─ homecore (state machine; handlers call services)
├─ homecore-api (exposes intent endpoints via REST/WS, P2)
├─ homecore-automation (complex intents can trigger automations)
├─ homecore-server (registers AssistPipeline at startup)
└─ ruflo (Ruflo agent subprocess for multi-step workflows, P2)

References

• ADR-133: HOMECORE Assist — Voice/Intent + Ruflo Bridge
• ADR-126: HOMECORE Home Assistant Port (master)
• Home Assistant Assist Integration
• Ruflo Documentation
• README — wifi-densepose