217 lines
6.0 KiB
Markdown
217 lines
6.0 KiB
Markdown
# RvLite - Standalone Vector Database
|
|
|
|
**Status**: Proof of Concept (v0.1.0)
|
|
|
|
RvLite is a lightweight, standalone vector database that runs entirely in WebAssembly. It provides SQL, SPARQL, and Cypher query interfaces, along with graph neural networks and self-learning capabilities.
|
|
|
|
## ๐ฏ Vision
|
|
|
|
A complete vector database that runs anywhere JavaScript runs:
|
|
- โ
Browsers (Chrome, Firefox, Safari, Edge)
|
|
- โ
Node.js
|
|
- โ
Deno
|
|
- โ
Bun
|
|
- โ
Cloudflare Workers
|
|
- โ
Vercel Edge Functions
|
|
|
|
## ๐๏ธ Architecture
|
|
|
|
RvLite is a **thin orchestration layer** over battle-tested WASM crates:
|
|
|
|
```
|
|
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
|
โ RvLite (Orchestration) โ
|
|
โ โโ SQL executor โ
|
|
โ โโ SPARQL executor โ
|
|
โ โโ Storage adapter โ
|
|
โ โโ Unified WASM API โ
|
|
โโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
|
โ depends on (100% reuse)
|
|
โผ
|
|
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
|
โ Existing WASM Crates โ
|
|
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
|
|
โ โข ruvector-core (vectors, SIMD) โ
|
|
โ โข ruvector-wasm (storage, indexing) โ
|
|
โ โข ruvector-graph-wasm (Cypher) โ
|
|
โ โข ruvector-gnn-wasm (GNN layers) โ
|
|
โ โข sona (ReasoningBank learning) โ
|
|
โ โข micro-hnsw-wasm (ultra-fast HNSW) โ
|
|
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
|
```
|
|
|
|
## ๐ Quick Start (Future)
|
|
|
|
```typescript
|
|
import { RvLite } from '@rvlite/wasm';
|
|
|
|
// Create database
|
|
const db = await RvLite.create();
|
|
|
|
// SQL with vector search
|
|
await db.sql(`
|
|
CREATE TABLE docs (
|
|
id SERIAL PRIMARY KEY,
|
|
content TEXT,
|
|
embedding VECTOR(384)
|
|
)
|
|
`);
|
|
|
|
await db.sql(`
|
|
SELECT id, content, embedding <=> $1 AS distance
|
|
FROM docs
|
|
ORDER BY distance
|
|
LIMIT 10
|
|
`, [queryVector]);
|
|
|
|
// Cypher graph queries
|
|
await db.cypher(`
|
|
CREATE (a:Person {name: 'Alice'})-[:KNOWS]->(b:Person {name: 'Bob'})
|
|
`);
|
|
|
|
// SPARQL RDF queries
|
|
await db.sparql(`
|
|
SELECT ?name WHERE {
|
|
?person foaf:name ?name .
|
|
}
|
|
`);
|
|
|
|
// GNN embeddings
|
|
const embeddings = await db.gnn.computeEmbeddings('social_network', [
|
|
db.gnn.createLayer('gcn', { inputDim: 128, outputDim: 64 })
|
|
]);
|
|
|
|
// Self-learning with ReasoningBank
|
|
await db.learning.recordTrajectory({ state: [0.1], action: 2, reward: 1.0 });
|
|
await db.learning.train({ algorithm: 'q-learning', iterations: 1000 });
|
|
```
|
|
|
|
## ๐ฆ Current Status (v0.1.0 - POC)
|
|
|
|
This is a **proof of concept** to validate:
|
|
- โ
Basic WASM compilation with ruvector-core
|
|
- โ
WASM bindings setup (wasm-bindgen)
|
|
- โณ Integration with other WASM crates (pending)
|
|
- โณ Bundle size measurement (pending)
|
|
- โณ Performance benchmarks (pending)
|
|
|
|
## ๐ ๏ธ Development
|
|
|
|
### Build
|
|
|
|
```bash
|
|
# Install wasm-pack
|
|
curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
|
|
|
|
# Build for web
|
|
cd crates/rvlite
|
|
wasm-pack build --target web --release
|
|
|
|
# Build for Node.js
|
|
wasm-pack build --target nodejs --release
|
|
```
|
|
|
|
### Test
|
|
|
|
```bash
|
|
# Run Rust unit tests
|
|
cargo test
|
|
|
|
# Run WASM tests (requires Chrome/Firefox)
|
|
wasm-pack test --headless --chrome
|
|
wasm-pack test --headless --firefox
|
|
```
|
|
|
|
### Size Analysis
|
|
|
|
```bash
|
|
# Build optimized
|
|
wasm-pack build --release
|
|
|
|
# Check size
|
|
ls -lh pkg/*.wasm
|
|
du -sh pkg/
|
|
```
|
|
|
|
## ๐ Documentation
|
|
|
|
See `/crates/rvlite/docs/` for comprehensive documentation:
|
|
- `00_EXISTING_WASM_ANALYSIS.md` - Analysis of existing WASM infrastructure
|
|
- `01_SPECIFICATION.md` - Complete requirements specification
|
|
- `02_API_SPECIFICATION.md` - TypeScript API design
|
|
- `03_IMPLEMENTATION_ROADMAP.md` - Original 5-week timeline
|
|
- `04_REVISED_ARCHITECTURE_MAX_REUSE.md` - Optimized 2-3 week plan
|
|
- `05_ARCHITECTURE_REVIEW_AND_VALIDATION.md` - Architecture validation
|
|
- `SPARC_OVERVIEW.md` - SPARC methodology overview
|
|
|
|
## ๐ฏ Roadmap
|
|
|
|
### Phase 1: Proof of Concept (Current)
|
|
- [x] Create rvlite crate structure
|
|
- [x] Set up WASM bindings
|
|
- [x] Basic compilation test
|
|
- [ ] Measure bundle size
|
|
- [ ] Integration with ruvector-wasm
|
|
- [ ] Integration with ruvector-graph-wasm
|
|
|
|
### Phase 2: Core Integration (Week 1)
|
|
- [ ] Storage adapter implementation
|
|
- [ ] SPARQL extraction from ruvector-postgres
|
|
- [ ] SQL parser integration (sqlparser-rs)
|
|
- [ ] Basic query routing
|
|
|
|
### Phase 3: Full Features (Week 2)
|
|
- [ ] GNN layer integration
|
|
- [ ] ReasoningBank integration
|
|
- [ ] Hyperbolic embeddings
|
|
- [ ] Comprehensive testing
|
|
|
|
### Phase 4: Production Release (Week 3)
|
|
- [ ] Documentation
|
|
- [ ] Examples (browser, Node.js, Deno)
|
|
- [ ] Performance benchmarks
|
|
- [ ] NPM package publication
|
|
|
|
## ๐ Size Budget
|
|
|
|
**Target**: < 3MB gzipped
|
|
|
|
**Expected breakdown**:
|
|
- ruvector-core: ~500KB
|
|
- SQL parser: ~200KB
|
|
- SPARQL executor: ~300KB
|
|
- Cypher (ruvector-graph-wasm): ~600KB
|
|
- GNN layers: ~300KB
|
|
- ReasoningBank (sona): ~300KB
|
|
- Orchestration: ~100KB
|
|
|
|
**Total estimated**: ~2.3MB gzipped โ
|
|
|
|
## ๐ค Contributing
|
|
|
|
This project reuses existing battle-tested WASM crates. Contributions should focus on:
|
|
1. Integration and orchestration
|
|
2. SQL/SPARQL/Cypher query routing
|
|
3. Storage adapter implementation
|
|
4. Testing and benchmarks
|
|
5. Documentation and examples
|
|
|
|
## ๐ License
|
|
|
|
MIT OR Apache-2.0
|
|
|
|
## ๐ Acknowledgments
|
|
|
|
RvLite is built on the shoulders of:
|
|
- `ruvector-core` - Vector operations and SIMD
|
|
- `ruvector-wasm` - WASM vector database
|
|
- `ruvector-graph` - Cypher and graph database
|
|
- `ruvector-gnn` - Graph neural networks
|
|
- `sona` - Self-learning and ReasoningBank
|
|
- `micro-hnsw-wasm` - Ultra-lightweight HNSW
|
|
|
|
---
|
|
|
|
**Status**: Proof of Concept - Architecture Validated โ
|
|
**Next Step**: Build and measure bundle size
|