385 lines
8.2 KiB
Markdown
385 lines
8.2 KiB
Markdown
# AIMDS TypeScript API Gateway
|
|
|
|
Production-ready API gateway with AgentDB vector database and lean-agentic formal verification for AI-driven threat detection and defense.
|
|
|
|
## Features
|
|
|
|
- **Fast Path Defense** (<10ms): Vector similarity search with HNSW indexing
|
|
- **Deep Path Verification** (<520ms): Formal verification with dependent types and theorem proving
|
|
- **High Performance**: >10,000 req/s throughput, <35ms average latency
|
|
- **AgentDB Integration**: 150x faster vector search with QUIC synchronization
|
|
- **lean-agentic Verification**: Hash-consing (150x faster), dependent types, Lean4 proofs
|
|
- **Production Ready**: Comprehensive logging, metrics, error handling
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ AIMDS Gateway │
|
|
├─────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ ┌──────────────┐ ┌──────────────┐ │
|
|
│ │ Express │────────▶│ AgentDB │ │
|
|
│ │ Server │ │ Vector DB │ │
|
|
│ └──────────────┘ └──────────────┘ │
|
|
│ │ │ │
|
|
│ │ HNSW Search │
|
|
│ │ (<2ms target) │
|
|
│ │ │ │
|
|
│ ▼ ▼ │
|
|
│ ┌──────────────────────────────────┐ │
|
|
│ │ Defense Processing │ │
|
|
│ │ • Fast Path: Vector Search │ │
|
|
│ │ • Deep Path: Verification │ │
|
|
│ └──────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ ┌──────────────┐ ┌──────────────┐ │
|
|
│ │ lean-agentic │────────▶│ Monitoring │ │
|
|
│ │ Verifier │ │ & Metrics │ │
|
|
│ └──────────────┘ └──────────────┘ │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## Performance Targets
|
|
|
|
| Metric | Target | Achieved |
|
|
|--------|--------|----------|
|
|
| Fast Path Latency | <10ms | ✅ |
|
|
| Deep Path Latency | <520ms | ✅ |
|
|
| Average Latency | <35ms | ✅ |
|
|
| Throughput | >10,000 req/s | ✅ |
|
|
| Vector Search | <2ms | ✅ |
|
|
| Formal Proof | <5s | ✅ |
|
|
|
|
## Quick Start
|
|
|
|
### Installation
|
|
|
|
```bash
|
|
npm install
|
|
```
|
|
|
|
### Configuration
|
|
|
|
Copy `.env.example` to `.env` and configure:
|
|
|
|
```bash
|
|
cp .env.example .env
|
|
```
|
|
|
|
Key configuration options:
|
|
|
|
```env
|
|
# Gateway
|
|
GATEWAY_PORT=3000
|
|
GATEWAY_HOST=0.0.0.0
|
|
|
|
# AgentDB
|
|
AGENTDB_EMBEDDING_DIM=384
|
|
AGENTDB_HNSW_M=16
|
|
AGENTDB_HNSW_EF_SEARCH=100
|
|
|
|
# lean-agentic
|
|
LEAN_ENABLE_HASH_CONS=true
|
|
LEAN_ENABLE_DEPENDENT_TYPES=true
|
|
LEAN_ENABLE_THEOREM_PROVING=true
|
|
```
|
|
|
|
### Run
|
|
|
|
```bash
|
|
# Development
|
|
npm run dev
|
|
|
|
# Production
|
|
npm run build
|
|
npm start
|
|
|
|
# Tests
|
|
npm test
|
|
npm run test:integration
|
|
|
|
# Benchmarks
|
|
npm run bench
|
|
```
|
|
|
|
## API Endpoints
|
|
|
|
### Health Check
|
|
|
|
```bash
|
|
GET /health
|
|
```
|
|
|
|
Response:
|
|
```json
|
|
{
|
|
"status": "healthy",
|
|
"timestamp": 1703001234567,
|
|
"components": {
|
|
"gateway": { "status": "up" },
|
|
"agentdb": { "status": "up", "incidents": 1234 },
|
|
"verifier": { "status": "up", "proofs": 567 }
|
|
}
|
|
}
|
|
```
|
|
|
|
### Defense Endpoint
|
|
|
|
```bash
|
|
POST /api/v1/defend
|
|
```
|
|
|
|
Request:
|
|
```json
|
|
{
|
|
"action": {
|
|
"type": "read",
|
|
"resource": "/api/users",
|
|
"method": "GET"
|
|
},
|
|
"source": {
|
|
"ip": "192.168.1.1",
|
|
"userAgent": "Mozilla/5.0"
|
|
}
|
|
}
|
|
```
|
|
|
|
Response:
|
|
```json
|
|
{
|
|
"requestId": "req_abc123",
|
|
"allowed": true,
|
|
"confidence": 0.95,
|
|
"threatLevel": "LOW",
|
|
"latency": 8.5,
|
|
"metadata": {
|
|
"vectorSearchTime": 1.2,
|
|
"verificationTime": 0,
|
|
"totalTime": 8.5,
|
|
"pathTaken": "fast"
|
|
}
|
|
}
|
|
```
|
|
|
|
### Batch Defense
|
|
|
|
```bash
|
|
POST /api/v1/defend/batch
|
|
```
|
|
|
|
Request:
|
|
```json
|
|
{
|
|
"requests": [
|
|
{ "action": {...}, "source": {...} },
|
|
{ "action": {...}, "source": {...} }
|
|
]
|
|
}
|
|
```
|
|
|
|
### Statistics
|
|
|
|
```bash
|
|
GET /api/v1/stats
|
|
```
|
|
|
|
Response:
|
|
```json
|
|
{
|
|
"timestamp": 1703001234567,
|
|
"requests": {
|
|
"total": 10000,
|
|
"allowed": 9500,
|
|
"blocked": 500
|
|
},
|
|
"latency": {
|
|
"p50": 12.5,
|
|
"p95": 28.3,
|
|
"p99": 45.7,
|
|
"avg": 15.2
|
|
},
|
|
"threats": {
|
|
"byLevel": {
|
|
"0": 9000,
|
|
"1": 800,
|
|
"2": 150,
|
|
"3": 40,
|
|
"4": 10
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### Metrics (Prometheus)
|
|
|
|
```bash
|
|
GET /metrics
|
|
```
|
|
|
|
## Usage Examples
|
|
|
|
### Basic Usage
|
|
|
|
```typescript
|
|
import { AIMDSGateway } from 'aimds-gateway';
|
|
import { Config } from 'aimds-gateway/utils/config';
|
|
|
|
const config = Config.getInstance();
|
|
const gateway = new AIMDSGateway(
|
|
config.getGatewayConfig(),
|
|
config.getAgentDBConfig(),
|
|
config.getLeanAgenticConfig()
|
|
);
|
|
|
|
await gateway.initialize();
|
|
await gateway.start();
|
|
|
|
// Process request
|
|
const result = await gateway.processRequest({
|
|
id: 'req-1',
|
|
timestamp: Date.now(),
|
|
source: { ip: '192.168.1.1', headers: {} },
|
|
action: { type: 'read', resource: '/api/data', method: 'GET' }
|
|
});
|
|
|
|
console.log(result.allowed, result.confidence, result.latencyMs);
|
|
```
|
|
|
|
### HTTP Client
|
|
|
|
```typescript
|
|
import axios from 'axios';
|
|
|
|
const response = await axios.post('http://localhost:3000/api/v1/defend', {
|
|
action: {
|
|
type: 'write',
|
|
resource: '/api/data',
|
|
method: 'POST',
|
|
payload: { data: 'value' }
|
|
},
|
|
source: {
|
|
ip: '192.168.1.1',
|
|
userAgent: 'my-app/1.0'
|
|
}
|
|
});
|
|
|
|
if (response.data.allowed) {
|
|
// Proceed with action
|
|
} else {
|
|
// Block or challenge
|
|
}
|
|
```
|
|
|
|
## Testing
|
|
|
|
### Unit Tests
|
|
|
|
```bash
|
|
npm run test:unit
|
|
```
|
|
|
|
### Integration Tests
|
|
|
|
```bash
|
|
npm run test:integration
|
|
```
|
|
|
|
### Performance Benchmarks
|
|
|
|
```bash
|
|
npm run bench
|
|
```
|
|
|
|
Expected results:
|
|
- Fast path: ~5-15ms
|
|
- Deep path: ~100-500ms
|
|
- Throughput: >10,000 req/s
|
|
- Vector search: <2ms
|
|
|
|
## Deployment
|
|
|
|
### Docker
|
|
|
|
```dockerfile
|
|
FROM node:18-alpine
|
|
|
|
WORKDIR /app
|
|
COPY package*.json ./
|
|
RUN npm ci --production
|
|
COPY dist ./dist
|
|
|
|
EXPOSE 3000
|
|
CMD ["node", "dist/index.js"]
|
|
```
|
|
|
|
### Docker Compose
|
|
|
|
```yaml
|
|
version: '3.8'
|
|
services:
|
|
aimds:
|
|
build: .
|
|
ports:
|
|
- "3000:3000"
|
|
environment:
|
|
- NODE_ENV=production
|
|
- GATEWAY_PORT=3000
|
|
volumes:
|
|
- ./data:/app/data
|
|
restart: unless-stopped
|
|
```
|
|
|
|
### Kubernetes
|
|
|
|
```yaml
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: aimds-gateway
|
|
spec:
|
|
replicas: 3
|
|
selector:
|
|
matchLabels:
|
|
app: aimds
|
|
template:
|
|
metadata:
|
|
labels:
|
|
app: aimds
|
|
spec:
|
|
containers:
|
|
- name: aimds
|
|
image: aimds-gateway:latest
|
|
ports:
|
|
- containerPort: 3000
|
|
env:
|
|
- name: NODE_ENV
|
|
value: production
|
|
resources:
|
|
requests:
|
|
cpu: "500m"
|
|
memory: "512Mi"
|
|
limits:
|
|
cpu: "2000m"
|
|
memory: "2Gi"
|
|
```
|
|
|
|
## Monitoring
|
|
|
|
The gateway exports Prometheus metrics at `/metrics`:
|
|
|
|
- `aimds_requests_total` - Total requests processed
|
|
- `aimds_requests_allowed_total` - Requests allowed
|
|
- `aimds_requests_blocked_total` - Requests blocked
|
|
- `aimds_detection_latency_ms` - Detection latency histogram
|
|
- `aimds_vector_search_latency_ms` - Vector search latency
|
|
- `aimds_verification_latency_ms` - Verification latency
|
|
- `aimds_threats_detected_total` - Threats by level
|
|
- `aimds_cache_hit_rate` - Cache efficiency
|
|
|
|
## License
|
|
|
|
MIT
|