Skip to main content

ADR-025: Rover identity uses the ValidationGraph VectorIndex; embeddings are a first-class VG capability

Context​

SPEC-055 (Rover identity & dedup) W1 wires the already-built AsyncIdentityMatcher into a durable, cross-scan candidate index so screen identity survives restarts (SPEC-038 D-B3.2). The matcher's candidate retrieval needs three paths: exact structural-signature_hash, route_activity, and embedding-KNN.

An earlier attempt (PR #433) introduced a Rover-private rover.candidate_index table (a new V003 migration) with its own embedding vector column, queried via raw JDBC + pgvector <=>. That duplicated infrastructure the ValidationGraph already owns: VG's locked plug-in set (SPEC-035 §10, D20/D21/D22) includes a VectorIndex port (upsert / searchSemantic, impl PgvectorVectorIndex over validationgraph.vector_index_entries) that already partitions vectors by tenant and embedding model, and a graph-native GraphStorageEngine.query surface for entity-property lookups. Rover is already a thin consumer of VG (GraphStorageEngine, RP1) and VectorIndex is already bound in Rover's DI (for SemanticSearch). Building a second, parallel embedding store inside Rover contradicts "Rover is a thin module over two existing domains."

Decision​

Embeddings + KNN are a first-class ValidationGraph capability via the existing VectorIndex port, and Rover is VG's first consumer of it — not a duplicator. Rover identity candidate lookup is served by ai.aucert.rover.identity.GraphVectorCandidateIndex, backed entirely by VG ports:

  • Embedding-KNN → VectorIndex.searchSemantic(RawVector), within a dedicated identity EmbeddingModelId partition (rover-identity/…) so identity vectors never collide with SemanticSearch's NL vectors (D21 multi-model coexistence).
  • Screen embedding writes → VectorIndex.upsert(...), keyed by the durable screen entityId + the identity model partition.
  • Exact signature_hash + route_activity → GraphStorageEngine.query(NODE, types={rover:screen}), reconstructing each screen's accumulated signature-hash / route SET from an identity-facet claim (predicate = rover_identity_facet, coexisting with the structural exists facet via VG's per-predicate property union, D3).

There is NO rover.candidate_index table and NO new Rover migration. PR #433's PgCandidateIndex - V003 are rejected and superseded by this design. The whole candidate index is rebuildable from the graph (the VectorIndex is a replica, D20); a screen whose vector is not upserted yet degrades gracefully (still hash/route-retrievable; the vector backfills later; a missing vector is never an error, P11).

The opt-in resolver is flag-gated (ROVER_IDENTITY_RESOLVER=matcher); the default (signature, SignatureIdentityResolver) is unchanged. VG keeps vectors in the companion vector_index_entries index (not an inline entities.embedding column) precisely so an entity can carry multiple embeddings (identity + NL-search) with proper per-model indexing — the dormant entities.embedding inline columns are dead schema whose removal is tracked separately.

Alternatives considered​

OptionProsCons
Chosen: GraphVectorCandidateIndex over VG VectorIndex + GraphStorageEngineNo new table/migration; reuses VG's tenant+model partitioning, RLS, and pgvector adapter; index rebuildable from the graph; Rover stays a thin VG consumerMatcher embedding sub-score needs the KNN path (W2) since the port exposes no raw-vector readback (see Risks); hash/route lookups are bounded in-process scans, not index pushdown
Rover-private rover.candidate_index table + V003 (PR #433)Direct control of columns; <=> KNN + IndexEntry.embedding readback in one tableDuplicates VG's vector store; a second embedding store to maintain/migrate; contradicts RP1; rejected by Vivek 2026-07-15
Inline entities.embedding column on the screen entityOne row per entityVG deliberately avoids it (no per-model indexing; one embedding per entity) — would be a Rover-private embedding store in spirit

Consequences​

What becomes easier​

  • One embedding + KNN infrastructure across the platform (VG's), tenant- and model-partitioned, with RLS and the pgvector adapter already built and tested.
  • Differential-scan identity is durable and rebuildable from the graph — no separate store to back up, migrate, or reconcile.
  • Adding a real semantic embedder (SPEC-055 W2) is a new EmbeddingModelId partition with no schema change (D21).

What becomes harder​

  • The existing AsyncIdentityMatcher computes embedding cosine from IndexEntry.embedding, but the VectorIndex ANN replica exposes no raw-vector readback, so GraphVectorCandidateIndex leaves embedding = null. In W1 only graph-queryable structure + route contribute to the match decision; MatcherBackedIdentityResolver.graphBacked lowers the MATCHED threshold to 0.55 so an exact structure∧route match still MATCHES (keeping 0.65 would re-mint duplicates — the bug SPEC-055 fights). W2 folds the KNN score (searchSemantic) into the composite and restores the calibrated 0.65.
  • Exact signature_hash / route_activity are bounded in-process scans of the scope's rover:screen entities, not index-pushdown — VG's celFilter is itself evaluated in-process after a type-filtered scan, so pushing the predicate into CEL buys no pushdown and would force interpolating a raw hash into a CEL string. Correct and O(scoped-screens) at app-partition corpus sizes; a dedicated indexed lookup is a later optimization but MUST NOT be a Rover-private table.

Risks​

  • Under-merge bias in W1's opt-in matcher path (safe direction, SPEC-038 D-A1.2.1) until W2's embedder
    • KNN fold-in. The default signature resolver is unaffected, so production behavior is unchanged.

References​

  • SPEC-055 (Rover identity & dedup) — §4/§5 describe a rover-owned index; this ADR + the drift-2026-07-15-SPEC-055-vg-vectorindex.md drift note record the VG-VectorIndex-backed deviation.
  • SPEC-035 D20/D21/D22 (the VectorIndex plug-in abstraction), SPEC-038 D-B3.1/D-B3.2.
  • Supersedes PR #433 (PgCandidateIndex + V003 rover.candidate_index).