5.9 KiB
5.9 KiB
Benchmark Quick Reference Card
🚀 Run Benchmarks
# All benchmarks
./scripts/run_benchmarks.sh
# Individual crate
cargo bench --bench temporal_bench # Temporal Compare
cargo bench --bench scheduler_bench # Nanosecond Scheduler
cargo bench --bench attractor_bench # Attractor Studio
cargo bench --bench solver_bench # Neural Solver
cargo bench --bench meta_bench # Strange Loop
cargo bench --bench quic_bench # QUIC Multistream
# Specific group
cargo bench --bench temporal_bench dtw
cargo bench --bench scheduler_bench overhead
📊 Performance Targets Cheatsheet
| Benchmark | Key Metric | Target |
|---|---|---|
temporal_bench |
DTW n=100 | <10ms |
| LCS n=100 | <5ms | |
| Edit n=100 | <3ms | |
scheduler_bench |
Schedule | <100ns |
| Execute | <1μs | |
| Stats | <10μs | |
attractor_bench |
Phase space | <20ms |
| Lyapunov | <500ms | |
| Detection | <100ms | |
solver_bench |
Encode | <10ms |
| Verify | <100ms | |
| Parse | <5ms | |
meta_bench |
Learn | <50ms |
| Extract | <20ms | |
| Integrate | <100ms | |
quic_bench |
Stream | <1ms |
| Multiplex | <100μs | |
| Throughput | >1GB/s |
🎯 Common Commands
# Compare branches
./scripts/benchmark_comparison.sh main feature-branch
# Save baseline
cargo bench -- --save-baseline main
# Compare with baseline
cargo bench -- --baseline main
# View HTML reports
open target/criterion/*/report/index.html
# View summary
cat target/criterion/SUMMARY.md
🔍 Profiling
# Flamegraph
cargo flamegraph --bench temporal_bench
# perf
perf record -g cargo bench --bench temporal_bench
perf report
# Valgrind
valgrind --tool=cachegrind target/release/deps/temporal_bench-*
📈 Benchmark Groups
temporal_bench
dtw_benches- DTW performancelcs_benches- LCS algorithmsedit_benches- Edit distancecache_benches- Cache scenariosmemory_benches- Memory patterns
scheduler_bench
overhead_benches- Schedule overheadlatency_benches- Task executionqueue_benches- Priority queuestats_benches- Statisticsthreading_benches- Multi-threading
attractor_bench
embedding_benches- Phase spacelyapunov_benches- Lyapunov calcdetection_benches- Attractor detectiontrajectory_benches- Trajectory analysisdimension_benches- Dimension estimationchaos_benches- Chaos detectionpipeline_benches- Complete analysis
solver_bench
encoding_benches- Formula encodingparsing_benches- Formula parsingverification_benches- Trace verificationstate_benches- State operationsneural_benches- Neural verificationoperator_benches- Temporal operatorspipeline_benches- Complete pipeline
meta_bench
learning_benches- Meta-learningpattern_benches- Pattern extractionhierarchy_benches- Multi-level learningintegration_benches- Cross-craterecursive_benches- Self-referentialpipeline_benches- Complete cycle
quic_bench
stream_benches- Stream operationsmultiplexing_benches- Multiplexingconnection_benches- Connection setupthroughput_benches- Data throughputconcurrent_benches- Concurrent streamserror_benches- Error handling
📂 File Locations
/workspaces/midstream/
├── benches/
│ ├── temporal_bench.rs (~450 lines)
│ ├── scheduler_bench.rs (~520 lines)
│ ├── attractor_bench.rs (~480 lines)
│ ├── solver_bench.rs (~490 lines)
│ ├── meta_bench.rs (~500 lines)
│ ├── quic_bench.rs (~420 lines)
│ ├── README.md (Overview)
│ └── QUICK_REFERENCE.md (This file)
├── scripts/
│ ├── run_benchmarks.sh (Run all)
│ └── benchmark_comparison.sh (Compare)
├── docs/
│ └── BENCHMARK_GUIDE.md (Full guide)
├── BENCHMARKS_SUMMARY.md (Summary)
└── Cargo.toml (Config)
🎨 Output Interpretation
Terminal Colors
- 🟢 Green: Performance improved
- 🟡 Yellow: Within noise threshold
- 🔴 Red: Performance regressed
Key Metrics
- Mean: Average execution time
- Std Dev: Consistency (lower is better)
- Median: Central tendency
- Throughput: Operations/second
⚡ Best Practices
- Close unnecessary apps before benchmarking
- Run multiple times for consistency
- Check std dev - high values indicate noise
- Use baselines for regression detection
- Profile hotspots when optimizing
- Document changes that affect performance
🔧 Troubleshooting
High Variance
# Increase sample size
cargo bench -- --sample-size 200
# Disable frequency scaling
sudo cpupower frequency-set --governor performance
Slow Benchmarks
# Reduce measurement time
cargo bench -- --measurement-time 5
# Run specific tests only
cargo bench --bench temporal_bench dtw_performance/linear/10
Memory Issues
# Run with more memory
cargo bench --release -- --test-threads=1
# Profile memory usage
valgrind --tool=massif cargo bench
📊 Example Output
dtw_performance/linear/100
time: [8.234 ms 8.567 ms 8.912 ms]
thrpt: [11.22 Kelem/s 11.67 Kelem/s 12.14 Kelem/s]
change:
time: [-5.2341% -3.1234% -1.0123%] (p = 0.00 < 0.05)
thrpt: [+1.0226% +3.2345% +5.5234%]
Performance has improved.
🎯 Quick Wins
- First time? Run:
./scripts/run_benchmarks.sh - Optimizing? Profile:
cargo flamegraph --bench <name> - Comparing? Run:
./scripts/benchmark_comparison.sh main feature - Debugging? Check:
target/criterion/*/report/index.html
Total Benchmarks: 6 files | Total Lines: ~2,860 | Groups: 45+ | Scenarios: 150+