Core Engine

The brain of StrataRouter — a Rust-powered semantic routing engine with sub-10ms P99 latency.

StrataRouter Core is the routing decision engine: given a query and its embedding, it determines which route — model, agent, or handler — should receive it, in under 10ms.

result = router.route(query, embedding)

print(result.route_id)    # "billing"
print(result.confidence)  # 0.94
print(result.latency_ms)  # 1.2ms

---

Key Features

Blazing Fast

Sub-10ms P99 latency with 10K+ req/s throughput. SIMD-optimized AVX2 vectorization for maximum hardware utilization.

Highly Accurate

95.4% routing accuracy via hybrid semantic, keyword, and rule-based scoring with isotonic calibration (ECE < 0.03).

Memory Efficient

~64MB for 1K routes — 33x less memory than semantic-router (2.1GB). Linear scaling with zero-copy operations.

Production Ready

Thread-safe Rust core with Send + Sync guarantees. 95%+ test coverage and minimal unsafe code.

HNSW Index

Hierarchical Navigable Small World graph for O(log N) approximate nearest-neighbor search with 99.5% recall@10.

Hybrid Scoring

Combines semantic similarity (0.64), BM25 keyword matching (0.29), and rule-based signals (0.07) for robust decisions.

---

Architecture

graph TB
    A[Query + Embedding] --> B[Router Engine]
    B --> C[HNSW Index]
    C --> D[Candidate Routes]
    D --> E[Hybrid Scorer]
    E --> F[Confidence Calibrator]
    F --> G[Route Result]
    H[Route Registry] -.-> B
    I[Scoring Weights] -.-> E
    J[Calibration Model] -.-> F
    style B fill:#4A9EFF
    style C fill:#00C853
    style E fill:#FFC107
    style F fill:#FF5252

Component Breakdown

Router Engine — Orchestrates the routing pipeline. Manages route registration, coordinates all components, and collects telemetry.

HNSW Index — Hierarchical graph enabling O(log N) approximate nearest-neighbor search with 99.5% recall@10 and lock-free concurrent reads.

Hybrid Scorer — Fuses three signals: semantic cosine similarity (64%), BM25 keyword matching (29%), and regex rule patterns (7%).

Confidence Calibrator — Maps raw scores to calibrated probabilities using isotonic regression. Expected Calibration Error: <0.03.

---

Performance

Benchmarks

Metric	StrataRouter	semantic-router	llamaindex	Gain
P99 Latency	8.7ms	178ms	245ms	20–28x faster
Throughput	18K req/s	450 req/s	380 req/s	40–47x higher
Memory	64MB	2.1GB	3.2GB	33–50x less
Accuracy	95.4%	84.7%	82.3%	+10–13 pts

Latency Breakdown (P99)

Component	Time	Share
Index search	~4ms	40%
Hybrid scoring	~3ms	30%
Calibration	~2ms	20%
Overhead	~1ms	10%
Total	~10ms	100%

Memory Scaling

Routes	Memory
1K	64MB
10K	352MB
100K	3.2GB

---

Usage Examples

Basic Routing

from stratarouter import Router, Route

router = Router(dimension=384, threshold=0.5)

router.add_route(Route(
    id="billing",
    description="Billing and payment questions",
    keywords=["invoice", "payment", "refund", "charge"],
    examples=[
        "Where's my invoice?",
        "How do I update my payment method?",
        "I want a refund"
    ]
))

router.add_route(Route(
    id="technical",
    description="Technical support and bug reports",
    keywords=["bug", "error", "crash", "not working"],
    examples=[
        "The app crashes on startup",
        "I'm getting an error message",
        "Feature X is not working"
    ]
))

# Build index once — then route many queries
embeddings = get_embeddings([r.description for r in router.routes])
router.build_index(embeddings)

result = router.route("My payment failed yesterday", embedding)
print(f"Route:      {result.route_id}")       # "billing"
print(f"Confidence: {result.confidence}")     # 0.94
print(f"Latency:    {result.latency_ms}ms")   # ~1.2ms

Advanced Configuration

from stratarouter import RouterConfig

config = RouterConfig(
    dimension=384,
    threshold=0.5,
    max_candidates=10,
    enable_calibration=True,
    semantic_weight=0.64,
    keyword_weight=0.29,
    rule_weight=0.07,
    enable_simd=True,
    num_threads=4,
)

router = Router(config)

# Route with alternatives
result = router.route(query, embedding, k=3)
print(f"Primary:  {result.route_id} ({result.confidence:.2f})")
for alt_id, alt_conf in result.alternatives:
    print(f"Alt:      {alt_id} ({alt_conf:.2f})")

---

Python API Reference

from stratarouter import Router, Route, RouteResult, RouterConfig

# Create and configure router
router = Router(config=RouterConfig(...))

# Manage routes
router.add_route(route)
router.remove_route(route_id)
router.update_route(route)
router.list_routes()

# Build index (required before routing)
router.build_index(embeddings)

# Route queries
result = router.route(query, embedding, k=1)

See Python API Reference for the full documentation.

---

Rust API Reference

use stratarouter_core::{Router, RouterConfig, Route};

let config = RouterConfig::default();
let mut router = Router::new(config);

let route = Route::new("billing")
    .with_description("Billing questions")
    .with_keywords(vec!["invoice", "payment"]);
router.add_route(route)?;

router.build_index(embeddings)?;

let result = router.route(query, &embedding)?;

See Rust API Reference for the full documentation.

---

Next Steps

HNSW Index

Deep dive into the vector search engine powering routing decisions.

→

Scoring Algorithms

Hybrid scoring, BM25 keyword matching, and confidence calibration.

→

Performance Tuning

Optimize for your throughput, latency, and memory requirements.

→

Runtime System

Connect routing decisions to LLM providers with caching and batching.

→