Requirements: Bedrock Model Spike + Document Versioning
Blue-team requirements contract — 2026-06-28 (Dan & Claude). The locked scope for two parallel pre-Brehob initiatives, blue-teamed together after their red-team passes. Each must-have carries its acceptance bar. Grounds epics/bedrock-model-spike and epics/document-versioning. They run in parallel (independent — no shared code).
A. Bedrock Model Spike
Information-gathering only — measure, bank a scorecard + a go/no-go, change nothing in production.
Must
| # | Requirement | Acceptance |
|---|---|---|
| A-M1 | Cost-vs-quality scorecard instrument — per-arm $/doc co-emitted with recall/MRR in run-experiment.ts; baseline re-freeze | One run emits {recall@k, mrr, $/doc} per arm; baseline re-frozen against the current pipeline |
| A-M2 | Model arms via Bedrock Converse, forced tool-use preserved — Haiku 4.5 (baseline) + Nova Lite + Nova Micro | All three arms run the real grouping prompt + schema against the per-index gold |
| A-M3 | Frontier scored with significance — recall + $/doc per arm; paired McNemar + bootstrap + noise-floor vs the Haiku-4.5 baseline | Each delta carries a significance + noise-floor verdict; no point estimates |
| A-M4 | Bank + recommend — quality-vs-price frontier + a one-paragraph go/no-go | Committed scorecard; recommendation = go (a model is materially cheaper at equal-or-better retrieval) or no-go |
Quality floor (the gate): vector recall@10 ≥ 0.90, FTS recall@5 ≥ 0.85, positive hybrid lift past the noise floor. Winner = cheapest viable arm, never cheapest arm.
Should
- Claude Haiku 3.5 as an extra cheaper rung.
- A Brehob-shaped slice (Slate Trucks) for the headline number.
- A-S1 — Embedding-quality arm: Titan v2 / Cohere Embed vs OpenAI
text-embedding-3-smallon retrieval recall, to bank the all-AWS-parity data point (promoted from optional).
Out
Any conversion / implementation, the embedding move, batch ingestion, the prose-JSON grouping rewrite. The decision to convert is a future, data-gated epic.
B. Document Versioning + Version Detection
Detect re-uploads/re-publishes as versions, preserve history, never destroy it. Editing is Foundry; incremental reuse is deferred.
Must
| # | Requirement | Acceptance |
|---|---|---|
| B-M1 | Unit hashing + source_hash dedup + de-risk measurement | Per-unit content hashes computed; byte-identical re-upload detected as a duplicate (skip + alert); the two-quote (Slate Trucks + a 2nd real quote) unit-Jaccard measured and recorded before the classifier is built |
| B-M2 | Version-detection classifier — candidate (filename/slug) → per-candidate Jaccard → three-way band | An upload is classified new-doc / new-version / duplicate / ambiguous; best-candidate scoring (incidental boilerplate matches ignored) |
| B-M3 | Confirm UX — the gray-zone "new version of <X>?" prompt | Ambiguous-band uploads surface a one-click confirm; high-confidence auto-links; low → new doc |
| B-M4 | Corrective action — undo an accidental version link | A "these aren't versions" action un-supersedes the prior version and splits the two back into separate docs; every link is reversible |
| B-M5 | Soft-supersede + flip-at-finalize | On a confirmed new version, the prior version is soft-superseded (superseded_at + logical_documents lineage) and the current pointer flips at finalize (new version ready), in one transaction; hard-delete removed from publishDocumentInScope + the app path; new version full-reprocesses |
| B-M6 | Version-detection eval set | Labeled link / not-link pairs (key hard-negative: two different template quotes); precision/recall swept across the threshold; sets B-M2's threshold on data, not vibes |
Should
- B-S1 — Stable addressing — find-or-create logical doc per (KB, slug); slug→current-version resolution; resolve the non-unique
name/LIMIT 1ambiguity. - B-S2 — Preserve all versions; partial vector index (current-only) — superseded chunks stay fully preserved but out of the hot retrieval path (needs a denormalized chunk-level
supersededflag + a partialivfflat). The partial index can ship when version volume is real; preserve-all is the default from day one.
Nice / future
- "What changed between versions" diff view — the B1-trust gold-standard use case; named so the preserve-all data model supports it.
- Candidate signals beyond filename (title / structural).
Out
- Editing / editable source of truth → the Foundry epic (where the git-vs-Postgres source-of-truth decision lives).
- Incremental reuse (carry-forward unchanged chunks) + the chunk-ownership-across-versions decision → deferred to re-upload scale.
- Chunk-level versioning.
- Pruning / deleting versions — never. Versions are preserved (the diff use case depends on it).
Both initiatives run in parallel pre-Brehob-kickoff (independent). The Bedrock spike supersedes the prior conversion-epic framing (conversion is now a future, data-gated decision). Versioning is the prerequisite for foundry-integration; editing + the source-of-truth decision live there.