wifi-densepose/vendor/midstream/plans/04-nanosecond-scheduler-integration.md

Nanosecond-Scheduler Integration Strategy

Executive Summary

This document details the integration of the nanosecond-scheduler crate into the Lean Agentic Learning System. The nanosecond-scheduler provides ultra-low-latency, high-precision task scheduling capabilities essential for real-time AI systems, high-frequency decision-making, and time-critical agent operations.

Research Background

Real-Time Scheduling Theory

Definition: Real-time scheduling involves allocating processor time to tasks with strict timing constraints, ensuring deadlines are met [1].

Key Concepts:

1. Hard Real-Time [1]: Missing a deadline is catastrophic

   • Medical devices
   • Industrial control systems
   • High-frequency trading

2. Soft Real-Time [2]: Missing deadlines degrades performance but isn't catastrophic

   • Video streaming
   • Interactive applications
   • AI inference

3. Scheduling Algorithms [3]:

   • Rate-Monotonic (RM): Priority based on period
   • Earliest Deadline First (EDF): Priority based on deadline
   • Least Laxity First (LLF): Priority based on slack time

4. Jitter and Latency [4]:

   • Jitter: Variation in execution time
   • Latency: Time from trigger to execution
   • Worst-Case Execution Time (WCET)

High-Precision Timing

Modern Hardware Capabilities:

• CPU TSC (Time Stamp Counter): Nanosecond precision
• HPET (High Precision Event Timer): ~10ns resolution
• RDTSC instruction: Direct cycle counting

Operating System Support:

• Linux: CLOCK_MONOTONIC_RAW, SCHED_FIFO
• RT-Linux patches for deterministic scheduling
• CPU isolation and affinity

References

[1] Liu, C. L., & Layland, J. W. (1973). "Scheduling algorithms for multiprogramming in a hard-real-time environment." Journal of the ACM, 20(1), 46-61.

[2] Buttazzo, G. C. (2011). "Hard Real-Time Computing Systems." Springer.

[3] Sha, L., et al. (2004). "Real time scheduling theory: A historical perspective." Real-Time Systems, 28(2-3), 101-155.

[4] Kopetz, H. (2011). "Real-Time Systems: Design Principles for Distributed Embedded Applications." Springer.

[5] Brandenburg, B. B., & Anderson, J. H. (2007). "Feather-trace: A light-weight event tracing toolkit." OSPERT 2007.

Integration Architecture

┌─────────────────────────────────────────────────────────────┐
│         Nanosecond-Scheduler Integration                    │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  ┌────────────────┐        ┌─────────────────┐            │
│  │  High Priority │        │  Deadline       │            │
│  │  Task Queue    │◄──────►│  Manager        │            │
│  │  (ns precision)│        │                 │            │
│  └────────┬───────┘        └─────────┬───────┘            │
│           │                          │                     │
│           │                          ▼                     │
│  ┌────────▼───────┐        ┌─────────────────┐            │
│  │  CPU-Pinned    │        │  Latency        │            │
│  │  Workers       │◄──────►│  Monitor        │            │
│  └────────┬───────┘        └─────────────────┘            │
│           │                                                 │
│  ┌────────▼───────┐        ┌─────────────────┐            │
│  │  Agent         │        │  Real-Time      │            │
│  │  Execution     │◄──────►│  Constraints    │            │
│  └────────────────┘        └─────────────────┘            │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Use Cases

1. High-Frequency Trading Bot

Problem: Execute trades within microsecond time windows.

Solution: Schedule trade decisions with nanosecond precision.

Implementation:

let mut scheduler = NanosecondScheduler::new();

// Schedule high-priority trade execution
scheduler.schedule_with_deadline(
    Task::ExecuteTrade(trade),
    Deadline::from_nanos(5_000), // 5 microseconds
    Priority::Critical,
);

// Ensure execution
scheduler.run_until_idle_with_guarantee();

2. Real-Time Sensor Fusion

Problem: Fuse data from multiple sensors with strict timing requirements.

Solution: Schedule sensor reads and fusion with precise timing.

Implementation:

// Schedule periodic sensor reads
scheduler.schedule_periodic(
    Task::ReadSensor(sensor_id),
    Period::from_micros(100), // 100μs period
    Priority::High,
);

// Schedule fusion with deadline
scheduler.schedule_with_deadline(
    Task::FuseSensorData,
    Deadline::from_micros(150),
    Priority::High,
);

3. Low-Latency Inference

Problem: ML inference must complete within strict latency budget.

Solution: Schedule inference with guaranteed execution time.

Implementation:

// Schedule inference with WCET guarantee
let wcet = estimate_worst_case_execution_time(&model);

scheduler.schedule_with_wcet(
    Task::RunInference(model, input),
    wcet,
    Deadline::from_micros(1000), // 1ms deadline
    Priority::High,
);

Technical Specifications

API Design

pub struct NanosecondScheduler {
    task_queue: PriorityQueue<ScheduledTask>,
    workers: Vec<CpuPinnedWorker>,
    latency_monitor: LatencyMonitor,
    config: SchedulerConfig,
}

pub struct ScheduledTask {
    pub id: TaskId,
    pub task: Task,
    pub priority: Priority,
    pub deadline: Option<Instant>,
    pub period: Option<Duration>,
    pub wcet: Option<Duration>,
}

pub enum Priority {
    Critical,  // RT priority 99
    High,      // RT priority 90
    Normal,    // RT priority 50
    Low,       // SCHED_OTHER
}

pub struct SchedulerConfig {
    pub enable_cpu_pinning: bool,
    pub enable_rt_scheduling: bool,
    pub num_workers: usize,
    pub latency_budget_ns: u64,
}

impl NanosecondScheduler {
    pub fn new(config: SchedulerConfig) -> Result<Self, Error>;

    pub fn schedule(
        &mut self,
        task: Task,
        priority: Priority,
    ) -> TaskHandle;

    pub fn schedule_with_deadline(
        &mut self,
        task: Task,
        deadline: Deadline,
        priority: Priority,
    ) -> TaskHandle;

    pub fn schedule_periodic(
        &mut self,
        task: Task,
        period: Period,
        priority: Priority,
    ) -> TaskHandle;

    pub fn schedule_with_wcet(
        &mut self,
        task: Task,
        wcet: Duration,
        deadline: Deadline,
        priority: Priority,
    ) -> TaskHandle;

    pub fn cancel(&mut self, handle: TaskHandle) -> Result<(), Error>;

    pub fn get_latency_stats(&self) -> LatencyStats;

    pub fn wait_for_completion(&self, handle: TaskHandle) -> Result<TaskResult, Error>;
}

Performance Requirements

Metric	Target	Rationale
Scheduling overhead	<100ns	Minimal impact
Jitter	<1μs	Predictable execution
Deadline miss rate	<0.001%	High reliability
Context switch latency	<2μs	Fast transitions
Wakeup latency	<10μs	Responsive

Integration Points

1. Agent Decision Scheduling

Location: src/lean_agentic/agent.rs

Enhancement:

pub struct RealTimeAgent {
    agent: AgenticLoop,
    scheduler: NanosecondScheduler,
    latency_budget: Duration,
}

impl RealTimeAgent {
    pub async fn make_decision_with_deadline(
        &mut self,
        context: &Context,
        deadline: Deadline,
    ) -> Result<Action, Error> {
        let task = Task::PlanAndAct {
            context: context.clone(),
        };

        let handle = self.scheduler.schedule_with_deadline(
            task,
            deadline,
            Priority::High,
        );

        // Wait for completion
        match self.scheduler.wait_for_completion(handle) {
            Ok(TaskResult::Action(action)) => Ok(action),
            Err(e) => Err(Error::DeadlineMissed(e)),
        }
    }
}

2. Stream Processing with Latency Guarantees

Location: src/lean_agentic/learning.rs

Enhancement:

impl StreamLearner {
    pub fn process_stream_with_latency_guarantee(
        &mut self,
        stream: impl Stream<Item = Message>,
        max_latency: Duration,
    ) -> impl Stream<Item = ProcessingResult> {
        let scheduler = NanosecondScheduler::new(config);

        stream.map(move |message| {
            let deadline = Instant::now() + max_latency;

            let handle = scheduler.schedule_with_deadline(
                Task::ProcessMessage(message),
                deadline,
                Priority::High,
            );

            scheduler.wait_for_completion(handle)
        })
    }
}

3. Knowledge Graph Updates with Priority

Location: src/lean_agentic/knowledge.rs

Enhancement:

impl KnowledgeGraph {
    pub fn update_with_priority(
        &mut self,
        entities: Vec<Entity>,
        priority: Priority,
    ) -> TaskHandle {
        self.scheduler.schedule(
            Task::UpdateKnowledgeGraph { entities },
            priority,
        )
    }

    pub fn critical_update(
        &mut self,
        entity: Entity,
        deadline: Deadline,
    ) -> Result<(), Error> {
        let handle = self.scheduler.schedule_with_deadline(
            Task::UpdateEntity { entity },
            deadline,
            Priority::Critical,
        );

        self.scheduler.wait_for_completion(handle)?;
        Ok(())
    }
}

Implementation Phases

Phase 1: Core Scheduler (Week 1)

• Implement priority queue
• Add CPU pinning support
• Create RT scheduling integration
• Implement basic task execution
• Write unit tests

Phase 2: Deadline Management (Week 2)

• Add deadline tracking
• Implement EDF scheduling
• Create WCET estimation
• Add deadline miss detection
• Write integration tests

Phase 3: Latency Monitoring (Week 3)

• Implement latency tracking
• Add jitter measurement
• Create performance metrics
• Add alerting for violations
• Benchmark performance

Phase 4: Advanced Features (Week 4)

• Add periodic task support
• Implement admission control
• Create task dependencies
• Add load balancing
• Write documentation

Benchmarking Strategy

Benchmark Suite

#[bench]
fn bench_schedule_overhead(b: &mut Bencher) {
    let mut scheduler = NanosecondScheduler::new(default_config());
    let task = Task::Noop;

    b.iter(|| {
        scheduler.schedule(task.clone(), Priority::Normal)
    });
}

#[bench]
fn bench_deadline_scheduling(b: &mut Bencher) {
    let mut scheduler = NanosecondScheduler::new(default_config());
    let deadline = Deadline::from_micros(100);

    b.iter(|| {
        let handle = scheduler.schedule_with_deadline(
            Task::Compute(|_| 42),
            deadline,
            Priority::High,
        );
        scheduler.wait_for_completion(handle)
    });
}

#[bench]
fn bench_periodic_tasks(b: &mut Bencher) {
    let mut scheduler = NanosecondScheduler::new(default_config());

    b.iter(|| {
        scheduler.schedule_periodic(
            Task::Noop,
            Period::from_micros(100),
            Priority::Normal,
        )
    });
}

Latency Measurement

#[test]
fn measure_scheduling_latency() {
    let mut scheduler = NanosecondScheduler::new(config);
    let mut latencies = Vec::new();

    for _ in 0..10000 {
        let start = Instant::now();

        let handle = scheduler.schedule(
            Task::Noop,
            Priority::High,
        );

        scheduler.wait_for_completion(handle).unwrap();

        let latency = start.elapsed();
        latencies.push(latency);
    }

    let stats = LatencyStats::from_samples(&latencies);

    assert!(stats.p99() < Duration::from_micros(10));
    assert!(stats.max() < Duration::from_micros(50));

    println!("Scheduling latency:");
    println!("  p50: {:?}", stats.p50());
    println!("  p99: {:?}", stats.p99());
    println!("  max: {:?}", stats.max());
}

Platform-Specific Optimizations

Linux

#[cfg(target_os = "linux")]
fn configure_rt_scheduling() -> Result<(), Error> {
    use libc::{sched_setscheduler, sched_param, SCHED_FIFO};

    let param = sched_param {
        sched_priority: 99,
    };

    unsafe {
        if sched_setscheduler(0, SCHED_FIFO, &param) != 0 {
            return Err(Error::RtSchedulingFailed);
        }
    }

    // Pin to isolated CPU
    pin_to_cpu(7)?;

    Ok(())
}

fn pin_to_cpu(cpu: usize) -> Result<(), Error> {
    use libc::{cpu_set_t, sched_setaffinity, CPU_SET, CPU_ZERO};

    unsafe {
        let mut cpu_set: cpu_set_t = std::mem::zeroed();
        CPU_ZERO(&mut cpu_set);
        CPU_SET(cpu, &mut cpu_set);

        if sched_setaffinity(0, std::mem::size_of::<cpu_set_t>(), &cpu_set) != 0 {
            return Err(Error::CpuPinningFailed);
        }
    }

    Ok(())
}

Windows

#[cfg(target_os = "windows")]
fn configure_high_priority() -> Result<(), Error> {
    use winapi::um::processthreadsapi::{
        GetCurrentThread, SetThreadPriority
    };
    use winapi::um::winbase::THREAD_PRIORITY_TIME_CRITICAL;

    unsafe {
        let thread = GetCurrentThread();
        if SetThreadPriority(thread, THREAD_PRIORITY_TIME_CRITICAL) == 0 {
            return Err(Error::PrioritySettingFailed);
        }
    }

    Ok(())
}

Success Criteria

• Scheduling overhead < 100ns (p99)
• Jitter < 1μs (p99)
• Deadline miss rate < 0.001%
• Context switch latency < 2μs
• Support for 10,000+ tasks/second
• Zero priority inversions in tests
• Full platform support (Linux, macOS, Windows)

Safety and Error Handling

Deadline Misses

pub enum DeadlineViolation {
    SoftMiss { actual: Duration, expected: Duration },
    HardMiss { actual: Duration, expected: Duration },
}

impl NanosecondScheduler {
    fn handle_deadline_miss(&mut self, task: &ScheduledTask, violation: DeadlineViolation) {
        match violation {
            DeadlineViolation::SoftMiss { actual, expected } => {
                tracing::warn!(
                    task_id = ?task.id,
                    actual_ns = actual.as_nanos(),
                    expected_ns = expected.as_nanos(),
                    "Soft deadline missed"
                );
            }
            DeadlineViolation::HardMiss { actual, expected } => {
                tracing::error!(
                    task_id = ?task.id,
                    actual_ns = actual.as_nanos(),
                    expected_ns = expected.as_nanos(),
                    "Hard deadline missed - critical violation"
                );
                self.trigger_emergency_protocol(task);
            }
        }
    }
}

Monitoring Dashboard

pub struct LatencyMonitor {
    samples: RingBuffer<Duration>,
    violations: Vec<DeadlineViolation>,
    stats: LatencyStats,
}

impl LatencyMonitor {
    pub fn report(&self) -> MonitoringReport {
        MonitoringReport {
            p50_latency: self.stats.p50(),
            p99_latency: self.stats.p99(),
            max_latency: self.stats.max(),
            deadline_miss_rate: self.calculate_miss_rate(),
            jitter: self.calculate_jitter(),
            utilization: self.calculate_utilization(),
        }
    }
}

Future Enhancements

1. GPU Scheduling: Extend to CUDA/OpenCL tasks
2. Distributed Scheduling: Coordinate across machines
3. Energy-Aware: Optimize for power consumption
4. Predictive Scheduling: ML-based WCET prediction
5. Formal Verification: Prove schedulability

References

[1] Liu & Layland (1973). Scheduling algorithms for hard-real-time. [2] Buttazzo (2011). Hard Real-Time Computing Systems. [3] Sha et al. (2004). Real time scheduling theory. [4] Kopetz (2011). Real-Time Systems. [5] Brandenburg & Anderson (2007). Feather-trace.

Appendix A: Example Usage

use midstream::nanosecond_scheduler::*;

// Create scheduler with RT configuration
let config = SchedulerConfig {
    enable_cpu_pinning: true,
    enable_rt_scheduling: true,
    num_workers: 4,
    latency_budget_ns: 1_000, // 1μs
};

let mut scheduler = NanosecondScheduler::new(config)?;

// Schedule high-priority task with deadline
let handle = scheduler.schedule_with_deadline(
    Task::ProcessCriticalEvent(event),
    Deadline::from_micros(100),
    Priority::Critical,
);

// Wait for completion
match scheduler.wait_for_completion(handle) {
    Ok(result) => println!("Completed: {:?}", result),
    Err(Error::DeadlineMissed(..)) => eprintln!("Deadline violated!"),
}

// Get performance statistics
let stats = scheduler.get_latency_stats();
println!("Latency p99: {:?}", stats.p99());