StateSet Sequencer Architecture
This document describes the high-level architecture of the StateSet Sequencer, a Verifiable Event Sync (VES) v1.0 implementation for deterministic event ordering, cryptographic verification, and agent-to-agent payment sequencing.System Overview
Component Relationships
Core Components
1. Ingest Service
Location:src/api/handlers/ingest.rs, src/server.rs
The entry point for all events via HTTP REST and gRPC. Responsible for:
- Authentication: Validates API keys, JWT tokens, or agent Ed25519 signatures
- Schema Validation: Validates payloads against registered JSON Schemas (configurable: disabled, warn, strict)
- Signature Verification: Verifies Ed25519 agent signatures with domain-separated hashing
- Deduplication: Rejects duplicate
event_idandcommand_idvalues - Batching: Groups events for efficient processing with parallel partitioning
- Rate Limiting: Sliding-window per-tenant rate limiting
2. Agent Key Registry
Location:src/auth/agent_keys.rs, src/infra/postgres/agent_key_registry.rs
Manages agent public keys for signature verification:
- Key Registration:
POST /api/v1/agents/keys(REST) andRegisterAgentKey(gRPC) - Key Lookup:
(tenant_id, agent_id, key_id) -> public_keywith LRU caching - Key Types: Ed25519 (signing) and X25519 (encryption)
- Validity Windows: Keys have
valid_fromandvalid_totimestamps - Revocation: Keys can be revoked to invalidate future signatures
- Proof of Possession: Registration requires a signature proving key ownership
3. Sequencer
Location:src/infra/postgres/sequencer.rs, src/infra/postgres/ves_sequencer.rs
Assigns monotonic sequence numbers to events:
- Monotonic Ordering: Each
(tenant_id, store_id)has independent sequence counter - Gap-Free: Sequence numbers are contiguous with no gaps
- Atomic Assignment: Uses a single PostgreSQL transaction with
SELECT ... FOR UPDATEon the per-stream counter - Receipt Generation: Produces signed receipts for each sequenced event (configurable via
VES_SEQUENCER_SIGNING_KEY) - Sequencer Identity: Optional pinned sequencer ID via
VES_SEQUENCER_ID
4. Event Store
Location:src/infra/postgres/event_store.rs
Append-only storage for sequenced events:
- Immutability: Events are never modified or deleted
- Encryption-at-Rest: Optional AES-256-GCM payload encryption (modes: disabled, optional, required)
- Indexing: Efficient queries by sequence, entity, and time
- Range Reads: Fetch events by sequence number range
- Read/Write Splitting: Reads served from replica pool when configured
5. Projector
Location:src/projection/handlers.rs, src/projection/runner.rs
Applies events to domain projections:
- Domain Handlers: Entity-specific projection logic
- Optimistic Concurrency: Version checking prevents conflicts
- Invariant Validation: Rejects events violating business rules
- Checkpoint Tracking: Tracks last processed sequence per store
- Dead Letter Queue: Failed projections are moved to DLQ for retry
- Order:
order.created,order.confirmed,order.shipped, etc. - Inventory:
inventory.initialized,inventory.adjusted,inventory.reserved - Product:
product.created,product.updated,product.deactivated - Customer:
customer.created,customer.updated,customer.address_added - Return:
return.requested,return.approved,return.refunded - x402 Payment:
x402_payment.created,x402_payment.sequenced,x402_payment.settled - x402 Batch:
x402_batch.created,x402_batch.committed,x402_batch.settled
6. Commitment Engine
Location:src/infra/postgres/commitment.rs, src/infra/ves_commitment.rs
Creates Merkle tree commitments over event batches:
- Merkle Roots: SHA-256 trees over event payload hashes
- State Roots: Track state transitions (prev_root -> new_root)
- Inclusion Proofs: Generate proofs for individual events
- Batch Storage: Persist commitments for later verification
- VES Commitments: Separate commitment engine for VES v1.0 events
7. Anchor Service
Location:src/anchor.rs
Submits commitments to Ethereum L2:
- StateSetAnchor Contract: On-chain batch commitment storage
- SetPaymentBatch Contract: On-chain x402 payment batch settlement
- Transaction Building: Constructs and signs anchor transactions using Alloy
- Verification: Confirms anchoring status on-chain
- Gas Management: Handles gas estimation and pricing
- Circuit Breaker Protected: External calls guarded by circuit breaker
8. Compliance Proof Engine
Location:src/domain/ves_compliance.rs, src/infra/ves_compliance.rs
Stores and verifies zero-knowledge compliance proofs generated by stateset-stark:
- Proof Storage: Stores STARK proofs in
ves_compliance_proofstable - Public Input Validation: Ensures canonical public inputs match event data
- Policy Verification: Validates proof matches declared policy
- Idempotency: Deduplicates by
(event_id, proof_type, policy_hash)
9. Validity Proof Registry
Location:src/domain/ves_validity.rs, src/infra/ves_validity.rs
External proof registry for SNARK/ZK proofs attesting to batch properties:
- Proof Submission: External provers submit validity proofs for committed batches
- Proof Storage: Persists proof bytes and public inputs
- Proof Hashing: SHA-256 hash of proof for integrity
- Stream Matching: Trigger enforces proofs reference valid batches
10. x402 Payment Engine
Location:src/domain/x402_payment.rs, src/infra/postgres/x402_repository.rs, src/infra/x402_batch_worker.rs, src/api/handlers/x402.rs
Implements the x402 protocol for agent-to-agent payment sequencing and batched L2 settlement:
- Payment Intent Sequencing: Signed payment intents assigned sequence numbers
- Signature Verification: Ed25519 signatures with
X402_PAYMENT_V1domain separator - Nonce-Based Replay Protection: Per-agent nonce tracking
- Idempotency: Optional idempotency keys for at-most-once delivery
- Batch Assembly: Configurable batch size (default 100, max 1000) and time thresholds
- Merkle Commitments: Merkle root computation over batched payment intents
- Multi-Chain Settlement: Settlement on Set Chain L2 via
SetPaymentBatchcontract - Multi-Asset Support: USDC, USDT, ssUSD, wssUSD, DAI, ETH
Supported Assets:
Payment Intent Lifecycle:
11. Schema Registry
Location:src/domain/schema.rs, src/infra/postgres/schema_store.rs, src/api/handlers/schemas.rs
JSON Schema validation system for event payloads:
- Schema Versioning: Monotonically increasing version per
(tenant_id, event_type) - Compatibility Modes: Forward, Backward, Full, or None
- Validation Modes: Disabled, Optional (warn), Required, Strict
- Status Lifecycle: Active -> Deprecated -> Archived
- LRU Caching: Configurable cache size and TTL for hot schemas
- Detailed Errors: Validation errors include JSON paths and messages
gRPC API (v1 + v2)
Location:src/grpc/, proto/sequencer.proto, proto/sequencer_v2.proto
The sequencer exposes dual gRPC services alongside the REST API:
gRPC v2 Service (Full VES v1.0 Protocol)
Key Management Service (gRPC)
Bidirectional Sync Protocol
TheSyncStream RPC enables full-duplex communication:
Operational Infrastructure
Authentication System
Location:src/auth/
Multi-method authentication with composable validators:
- Rate Limiting: Sliding-window algorithm, configurable per-minute limit
- Permissions Model: Read, Write, Admin scopes per key
- gRPC Auth Interceptor: Shared authenticator for gRPC services
Cache Manager
Location:src/infra/cache.rs
Multi-layer LRU caching with configurable TTL per cache type:
Pool Monitor
Location:src/infra/pool_monitor.rs
Real-time database connection pool health tracking (15-second polling):
- Health States: Healthy (< 50%), Moderate (50-80%), Stressed (80-95%), Critical (> 95%)
- Metrics: Active/idle connections, acquisition latency, slow acquisition tracking
- Integrated: Exposed via
/health/detailedendpoint and Prometheus metrics
Circuit Breaker Registry
Location:src/infra/circuit_breaker.rs
Failure resilience for external service calls (L2 anchoring, chain settlement):
- States: Closed (normal) -> Open (fail-fast) -> HalfOpen (testing recovery)
- Exponential Backoff: Configurable multiplier with jitter
- Slow Call Detection: Configurable threshold for degraded performance
- Per-Service Tracking: Independent breaker per external service
Dead Letter Queue
Location:src/infra/dead_letter.rs
Handles events that fail projection processing:
- Auto-Retry: Exponential backoff (1 min initial, 1 hour max, 10 retries)
- Categorized Reasons: Schema validation, invariant violation, state transition errors
- Non-Retryable: Invariant violations and invalid state transitions skip retry
- Admin Operations: Retry, purge, and inspect via admin CLI
Payload Encryption-at-Rest
Location:src/infra/payload_encryption.rs, src/crypto/encrypt.rs
Automatic event payload encryption in the database:
- Modes: Disabled, Optional, Required
- Algorithm: AES-256-GCM
- HPKE Support: Multi-recipient encryption via X25519-HKDF-SHA256
- Key Rotation: Supports key versioning and rotation
Audit Logging
Location:src/infra/audit.rs
Comprehensive audit trail for administrative operations:
- API key management (create, revoke, update)
- Schema registry changes (register, deprecate, delete)
- Agent key operations (register, rotate, revoke)
- Authentication events (login, failure, token refresh)
- Dead letter queue operations (retry, purge)
Metrics & Telemetry
Location:src/metrics/, src/telemetry/
- Prometheus Export:
/metricsendpoint with 40+ predefined metric names - Component Metrics: Background collection every 15 seconds (pool, circuit breaker stats)
- OpenTelemetry: OTLP export for distributed tracing (
OTEL_EXPORTER_OTLP_ENDPOINT) - Structured Logging: JSON or text format (
LOG_FORMAT) - Counters, Gauges, Histograms: Full metric type support with labels
Graceful Shutdown
Location:src/infra/graceful_shutdown.rs
Coordinated shutdown with request draining:
- Request Tracking: Guard-based in-flight request monitoring
- Shutdown Signals: Coordinated signal propagation to background tasks
- Deadline Enforcement: Configurable drain timeout
stateset-stark (ZK Compliance Proofs)
Repository:stateset-stark
A STARK proving system that enables cryptographic verification of compliance policies on encrypted event payloads without revealing the underlying data.
Purpose
When events contain encrypted payloads (e.g., order amounts), compliance rules (e.g., AML thresholds) need verification without exposing sensitive data.stateset-stark generates zero-knowledge proofs that:
- The prover knows the plaintext payload
- The payload satisfies the compliance policy
- The payload matches the encrypted ciphertext hash
Architecture
Cryptographic Foundation
Supported Policies
Proof Generation Flow
Public Inputs (Canonical JCS Format)
CLI Usage
Integration Points
REST API Reference
Event Ingestion
VES Commitments
VES Proofs & Anchoring
VES Validity Proofs
VES Compliance Proofs
x402 Payment Protocol
Schema Registry
Legacy Events & Commitments
Health & Observability
Data Flow
Event Ingestion Flow
x402 Payment Flow
Commitment Flow
Verification Flow
Compliance Proof Flow
Database Schema
Core Tables
Migrations
Indexes
Cryptographic Design
Signing Hash Construction
Per VES v1.0 Section 8.3:x402 Payment Signing Hash
Merkle Tree Construction
Encryption (HPKE)
Multi-recipient encryption for VES-ENC-1:Offline-First Architecture
Agents operate offline using SQLite:Scalability Considerations
Horizontal Scaling
- Stateless API: Multiple sequencer instances behind load balancer
- Read/Write Pool Splitting: Separate connection pools for reads (replica) and writes (primary)
- Database Pooling: Configurable pool size, acquire timeout, idle timeout, max lifetime
- Sequence Partitioning: Each
(tenant_id, store_id)is independent
Performance Optimizations
- Batch Inserts: Events ingested in batches with parallel partitioning
- Read Replicas: Entity history and read queries served from replica pool
- Multi-Layer Caching: LRU caches for commitments, proofs, schemas, and agent keys
- Proof Memoization: Common proof paths cached with TTL
- Connection Pool Monitoring: Automatic health degradation detection
Resilience
- Circuit Breakers: External service calls (L2 anchoring) protected with exponential backoff
- Dead Letter Queue: Failed projections queued with automatic retry
- Graceful Shutdown: Request draining on SIGTERM
- Rate Limiting: Per-tenant sliding-window rate limiter
Capacity Planning
Security Model
Trust Boundaries
- Agent -> Sequencer: TLS + Ed25519 signatures + API key/JWT auth
- Sequencer -> Database: Network isolation + credentials + session timeouts
- Sequencer -> L2 Chain: Private key for signing + circuit breaker
- Agent -> Agent (payments): Ed25519 signed payment intents + nonce replay protection
Key Management
- Agent private keys: Never leave agent, stored securely
- Sequencer signing key: For receipt signing (
VES_SEQUENCER_SIGNING_KEY) - Sequencer anchor key: For L2 transactions (
SEQUENCER_PRIVATE_KEY) - API keys: SHA-256 hashed, stored in PostgreSQL
- Database credentials: Environment variables, not in code
- Encryption keys: AES-256-GCM for payload encryption-at-rest
Audit Trail
All administrative operations logged via the audit system:- API key lifecycle (create, revoke, update)
- Schema registry changes
- Agent key operations
- Authentication events
- Dead letter queue operations
Configuration
Feature Flags (Cargo Features)
Key Environment Variables
Technology Stack
Binaries
Module Structure
Related Documentation
- Getting Started - Quick start guide
- System Overview - High-level system overview
- VES Specification - Full protocol spec
- API Reference - REST API documentation
- Event Types - Supported event types
- Deployment Guide - Production deployment
- Runbook - Operational runbook
- Security Guide - Security best practices
- ZK Integration Guide - STARK proof integration
- Agent Integration - Agent SDK integration guide
- Anchoring Overview - On-chain anchoring details