wifi-densepose/vendor/midstream/AIMDS/docs/guides/README.md

385 lines
8.2 KiB
Markdown

# AIMDS TypeScript API Gateway
Production-ready API gateway with AgentDB vector database and lean-agentic formal verification for AI-driven threat detection and defense.
## Features
- **Fast Path Defense** (<10ms): Vector similarity search with HNSW indexing
- **Deep Path Verification** (<520ms): Formal verification with dependent types and theorem proving
- **High Performance**: >10,000 req/s throughput, <35ms average latency
- **AgentDB Integration**: 150x faster vector search with QUIC synchronization
- **lean-agentic Verification**: Hash-consing (150x faster), dependent types, Lean4 proofs
- **Production Ready**: Comprehensive logging, metrics, error handling
## Architecture
```
┌─────────────────────────────────────────────────────────┐
│ AIMDS Gateway │
├─────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ Express │────────▶│ AgentDB │ │
│ │ Server │ │ Vector DB │ │
│ └──────────────┘ └──────────────┘ │
│ │ │ │
│ │ HNSW Search │
│ │ (<2ms target) │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────────────────────────┐ │
│ │ Defense Processing │ │
│ │ • Fast Path: Vector Search │ │
│ │ • Deep Path: Verification │ │
│ └──────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ lean-agentic │────────▶│ Monitoring │ │
│ │ Verifier │ │ & Metrics │ │
│ └──────────────┘ └──────────────┘ │
│ │
└─────────────────────────────────────────────────────────┘
```
## Performance Targets
| Metric | Target | Achieved |
|--------|--------|----------|
| Fast Path Latency | <10ms | |
| Deep Path Latency | <520ms | |
| Average Latency | <35ms | |
| Throughput | >10,000 req/s | ✅ |
| Vector Search | <2ms | |
| Formal Proof | <5s | |
## Quick Start
### Installation
```bash
npm install
```
### Configuration
Copy `.env.example` to `.env` and configure:
```bash
cp .env.example .env
```
Key configuration options:
```env
# Gateway
GATEWAY_PORT=3000
GATEWAY_HOST=0.0.0.0
# AgentDB
AGENTDB_EMBEDDING_DIM=384
AGENTDB_HNSW_M=16
AGENTDB_HNSW_EF_SEARCH=100
# lean-agentic
LEAN_ENABLE_HASH_CONS=true
LEAN_ENABLE_DEPENDENT_TYPES=true
LEAN_ENABLE_THEOREM_PROVING=true
```
### Run
```bash
# Development
npm run dev
# Production
npm run build
npm start
# Tests
npm test
npm run test:integration
# Benchmarks
npm run bench
```
## API Endpoints
### Health Check
```bash
GET /health
```
Response:
```json
{
"status": "healthy",
"timestamp": 1703001234567,
"components": {
"gateway": { "status": "up" },
"agentdb": { "status": "up", "incidents": 1234 },
"verifier": { "status": "up", "proofs": 567 }
}
}
```
### Defense Endpoint
```bash
POST /api/v1/defend
```
Request:
```json
{
"action": {
"type": "read",
"resource": "/api/users",
"method": "GET"
},
"source": {
"ip": "192.168.1.1",
"userAgent": "Mozilla/5.0"
}
}
```
Response:
```json
{
"requestId": "req_abc123",
"allowed": true,
"confidence": 0.95,
"threatLevel": "LOW",
"latency": 8.5,
"metadata": {
"vectorSearchTime": 1.2,
"verificationTime": 0,
"totalTime": 8.5,
"pathTaken": "fast"
}
}
```
### Batch Defense
```bash
POST /api/v1/defend/batch
```
Request:
```json
{
"requests": [
{ "action": {...}, "source": {...} },
{ "action": {...}, "source": {...} }
]
}
```
### Statistics
```bash
GET /api/v1/stats
```
Response:
```json
{
"timestamp": 1703001234567,
"requests": {
"total": 10000,
"allowed": 9500,
"blocked": 500
},
"latency": {
"p50": 12.5,
"p95": 28.3,
"p99": 45.7,
"avg": 15.2
},
"threats": {
"byLevel": {
"0": 9000,
"1": 800,
"2": 150,
"3": 40,
"4": 10
}
}
}
```
### Metrics (Prometheus)
```bash
GET /metrics
```
## Usage Examples
### Basic Usage
```typescript
import { AIMDSGateway } from 'aimds-gateway';
import { Config } from 'aimds-gateway/utils/config';
const config = Config.getInstance();
const gateway = new AIMDSGateway(
config.getGatewayConfig(),
config.getAgentDBConfig(),
config.getLeanAgenticConfig()
);
await gateway.initialize();
await gateway.start();
// Process request
const result = await gateway.processRequest({
id: 'req-1',
timestamp: Date.now(),
source: { ip: '192.168.1.1', headers: {} },
action: { type: 'read', resource: '/api/data', method: 'GET' }
});
console.log(result.allowed, result.confidence, result.latencyMs);
```
### HTTP Client
```typescript
import axios from 'axios';
const response = await axios.post('http://localhost:3000/api/v1/defend', {
action: {
type: 'write',
resource: '/api/data',
method: 'POST',
payload: { data: 'value' }
},
source: {
ip: '192.168.1.1',
userAgent: 'my-app/1.0'
}
});
if (response.data.allowed) {
// Proceed with action
} else {
// Block or challenge
}
```
## Testing
### Unit Tests
```bash
npm run test:unit
```
### Integration Tests
```bash
npm run test:integration
```
### Performance Benchmarks
```bash
npm run bench
```
Expected results:
- Fast path: ~5-15ms
- Deep path: ~100-500ms
- Throughput: >10,000 req/s
- Vector search: <2ms
## Deployment
### Docker
```dockerfile
FROM node:18-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --production
COPY dist ./dist
EXPOSE 3000
CMD ["node", "dist/index.js"]
```
### Docker Compose
```yaml
version: '3.8'
services:
aimds:
build: .
ports:
- "3000:3000"
environment:
- NODE_ENV=production
- GATEWAY_PORT=3000
volumes:
- ./data:/app/data
restart: unless-stopped
```
### Kubernetes
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: aimds-gateway
spec:
replicas: 3
selector:
matchLabels:
app: aimds
template:
metadata:
labels:
app: aimds
spec:
containers:
- name: aimds
image: aimds-gateway:latest
ports:
- containerPort: 3000
env:
- name: NODE_ENV
value: production
resources:
requests:
cpu: "500m"
memory: "512Mi"
limits:
cpu: "2000m"
memory: "2Gi"
```
## Monitoring
The gateway exports Prometheus metrics at `/metrics`:
- `aimds_requests_total` - Total requests processed
- `aimds_requests_allowed_total` - Requests allowed
- `aimds_requests_blocked_total` - Requests blocked
- `aimds_detection_latency_ms` - Detection latency histogram
- `aimds_vector_search_latency_ms` - Vector search latency
- `aimds_verification_latency_ms` - Verification latency
- `aimds_threats_detected_total` - Threats by level
- `aimds_cache_hit_rate` - Cache efficiency
## License
MIT