How this is built
An evidence-grounded document-investigation system. A user uploads one or more documents, asks a natural-language question, and ORION either answers with citations verified against the source text, or explicitly declines when the evidence doesn't support an answer.
Reading the live configuration…
Everything below is read from HLD.md and LLD.md in this repository at request time. It is not a summary written for this page.
Intelligence, not authority
An LLM drafts a candidate answer with citations. Plain code — not a second model call — then checks each citation through three ordered stages (§5). A second model cannot be a hallucination guard for the first.
Exactly one component in the request path is probabilistic. Every other component named in this document is ordinary deterministic code.
Pipeline
The verification chain
Three checks run in a fixed order, and the order carries the security property.
- Membership in this run's own evidenceWas this chunk actually retrieved by this investigation? A chunk that exists in the database but was never surfaced for this question is rejected exactly as if it did not exist.
- Citation validityThe cited chunk is re-read and re-validated via
validate_citation(). This runs second, independently — on its own it would happily accept a real chunk this run never retrieved, which is precisely why membership runs first. - Verbatim quote groundingThe claim's quote must occur, normalized for whitespace and case, inside the chunk it cites. String containment against stored text, never a second model call.
Verification is all-or-nothing, not per-claim. If any claim fails any stage, _verify_evidence_briefing returns (False, reason) and the entire run is marked failed. Unverifiable claims are not quietly dropped and no partial answer is returned. This is stricter than a filter, deliberately: a briefing whose claims have been silently pruned still reads as a confident answer, and the reader has no way to know what was removed.
Three outcomes, not two
A refusal and a verification failure are different events with different causes. The system records them separately and analytics distinguishes all three.
Claims produced and every one passed all three stages.
Run status completedAn explicit refusal — retrieval found nothing usable, or the model declined. A successful outcome, not an error.
Run status completedThe model asserted something the verifier could not confirm. The result is discarded rather than shown.
Run status failedData flow, one investigation
- A document is uploaded, parsed, chunked at structural boundaries, classified for content safety, embedded, and indexed for full-text search.
- A question creates a
Run. The per-tenant cap is checked before any work begins. - Hybrid retrieval fuses the lexical and semantic channels by RRF and persists the retrieved citations as the first step's output.
- The model drafts a structured briefing citing specific chunks with verbatim quotes.
- The verifier checks every citation through all three stages against the database.
- If all pass, the run completes; if any fails, the run is marked
failed; if the model declined, the refusal is returned as a completed result. - The run is persisted either way — query, result, citations, timings. That record is what powers analytics later, and what the cost cap counts against.
Key decisions
| Decision | Reasoning | Record |
|---|---|---|
| Deterministic verification over an LLM judge | A second model call cannot be a hallucination guard for the first. | core invariant |
| Stub providers by default, real by explicit config | No unfunded API call happens unless a real provider is deliberately configured in the deployment dashboard. | ADR-0024, ADR-0034 |
| No free local model shipped as default | None reached a production-acceptable bar on the verbatim-quote task. The shipped default was itself measured on 2026-09-08 — see §10 and ADR-0019's addendum. | ADR-0019 |
| Tenant isolation as a hard invariant | Kept even though it makes a pre-seeded shared demo workspace structurally impossible — the correct trade-off for a public deployment, not a gap. | ADR-0024 |
| Analytics keyed by document id, never title | Titles are not unique; keying by them silently merged two different documents into one row. | ADR-0033 |
| Section-aware chunking | Chunks never span a section boundary, and overlap is not carried across one. | ADR-0025 |
| RRF weighting left untouched | No invented tuning without a labelled relevance set to justify it. | deferred |
Testing discipline
A four-tier suite gates every change via make ci (format-check, lint, typecheck, tests). Any generalization to chunking or retrieval is checked for blast radius — confirmed not to silently change behaviour on any already-ingested document — before it ships.
- 395
- Unit Pure logic — parsing, chunking, fusion, classification
- 312
- Integration Real Postgres, through the real router
- 15
- Adversarial Injection and grounding-evasion attempts
- 219
- Frontend Component and interaction tests
Count at fb94686
Known limitations
Stated because they are true, not because they are comfortable.
- The deployed instance runs real providers on both sides
Read the full itemCollapse
llm_providerisanthropic(claude-haiku-4-5) andembedding_providerisvoyage(voyage-4-lite, 256 native dimensions) in production. Both verified live rather than read off a dashboard:scripts/verify_live_providers.pyruns a real investigation against the deployed API, confirms the briefing is notFakeLLMProvider's literal fingerprint, and independently re-checks every returned citation as a verbatim quote inside its cited chunk — client-side, on the wire payload, so the server is never trusted to mark its own homework. Latest run: 4/4 citations verified, andsemantic ranks [1, 2, 3, 4, 10]againstlexical [9, 11, 12, 25, 14]— an ordering a hash embedding cannot produce. The residual constraint is a rate limit, and it is real. The Voyage account has no payment method, so it runs on the free tier's 3 requests per minute and 10K tokens per minute — quoted from the provider's own 429 body, not inferred. Past three embedding calls in a minute, search falls back to keyword-only and says so rather than failing:POST /searchsetsX-Orion-Semantic-Unavailable, and an investigation recordssemantic_unavailablein its step output so the degradation appears in the run trace. Grounding is unaffected — every claim is still verified against a real chunk — but the evidence pool for that query was narrower, and a run whose trace did not say so could not be audited afterwards. 10K TPM is a separate ceiling pacing cannot fix:embed_textsbatches a whole document into one request, so an upload beyond roughly 40KB exceeds the per-minute token budget in a single call. Stated as arithmetic against the provider's number, not as an observed failure. Provider values are dashboard-only (sync: falseinrender.yaml), so this remains a claim to be re-established by running the script after any redeploy or provider change — not read off the API. - Self-hosted embeddings remain impossible; hosted ones solved it
Read the full itemCollapse
ollamais still unusable here —ollama_base_urlis constrained local-only and the deployed container has no Ollama beside it. ADR-0034 concluded from that there was "no paid drop-in", which was never checked and turned out to be false: Voyage is hosted, has a 200M-token free allowance, and supports 256 dimensions natively (ADR-0035). The lesson is the ADR's, not the vendor's — an inference about the world was recorded as a finding without anyone testing it. - The fusion weights have never been tuned against real embeddings
Read the full itemCollapse
semantic_weight = 1.0againstlexical_weight = 0.5was chosen when the semantic channel was hash noise, where weighting it twice as heavily as the genuinely real lexical channel was actively harmful. That channel now carries real signal, so the setting is no longer wrong in the same way — but it was never right on purpose either, and it stays untouched until there is a labelled relevance set to tune against. Guessing at weights and calling the result tuning is the failure this deferral exists to prevent. - RRF scores have no calibrated meaning
Fusion is rank-based, not magnitude-based, so a low top score cannot distinguish "no good answer exists" from "a good answer landed at a bad rank." Deferred pending a labelled relevance dataset that does not yet exist.
- One content-safety false positive is open
The MEDIUM-severity
act_as_personapattern over-matches ordinary prose (3 hits in the current corpus). Tracked, not silently suppressed. - Deleted documents are not marked in history
Citations to since-deleted documents are counted correctly — run history legitimately outlives the documents it cites — but nothing in the interface yet marks such a row as historical (ADR-0033).
- Answer quality is measured; one criterion is unmet and unsound, and one real gap is now localised
Read the full itemCollapse
ADR-0019's bar was measured against the shipped default on 2026-09-08 and re-measured on 2026-09-09 with real embeddings. Failed verification 0/18 (bar ≤1/18, met); median latency 2.754 s (bar <10 s, met); answered 6/18 (bar ≥7/18, not met). On every generation with a deterministic correct outcome the model was right — 9/9, false-refusal and false-answer rates both 0.0% — and ≥7/18 turns out to be unreachable by a correct model on this eval set, since only 6 of its 18 generations have a defined should-answer ground truth. That criterion needs rewriting before the next run, not reinterpreting after this one. Real embeddings changed nothing measurable, and that is a finding about the eval rather than about embeddings. Every scored dimension came back identical.
GOLDEN_CORPUSis seven documents totalling 844 characters whose questions share vocabulary directly with their sources, sotsvectoralone already retrieves the right chunk every time and a semantic channel has nothing to add. This eval is not an instrument capable of measuring retrieval quality; measuring that needs a corpus where lexical search genuinely fails, plus the labelled relevance judgements the fusion deferral is also waiting on. That gap is now closed. Because better retrieval had not moved it, the defect was localised to the prompt — andevidence_briefing.v8changed the one refusal rule responsible, measured on 18 further generations before shipping.incomplete_evidencewent from an empty briefing 3/3 to a cited one 3/3 (citation_completeness0.00 → 1.00), quoting what the document does say and naming what it omits, with no invented figure.hallucination_probeimproved the same way unplanned. Every disqualifier held: 0/18 failed verification, 3/3 on both answered categories, 100% citation validity and quote grounding. Median latency rose 2.754 s → 4.019 s and cost 19%, both stated because the prompt is longer and the model now writes where it stayed silent. Worth knowing when reading any number above: the aggregate scoreboard did not move at all between v4 and v8 — same 6/18 answered, same refusal categories. Three refusals went from useless to useful and every counter stayed still. This eval measures the grounding contract, and measures it well; it does not measure retrieval quality or answer usefulness, and nothing scored against it should be described more widely than that. Full record: ADR-0019's three addenda.
Reference
3. Architecture
Diagram source (Mermaid)
flowchart TB
FE["Next.js frontend<br/>Vercel"]
subgraph API["API service — Render"]
ING["Ingestion & structure-aware chunking"]
SAFE["Content-safety classifier"]
RET["Hybrid retrieval — RRF fusion"]
GEN["Evidence-briefing generation<br/><b>the only LLM call</b>"]
VER["Deterministic verification<br/>3 ordered stages"]
CAP["Per-tenant run cap"]
AN["Investigation analytics"]
end
PG[("Postgres — Neon<br/>pgvector + tsvector")]
EMB["Embedding provider<br/>fake | ollama | voyage"]
LLM["LLM provider<br/>fake | anthropic | ollama"]
FE -->|upload| ING
ING --> SAFE
SAFE --> PG
ING --> EMB
FE -->|ask| CAP
CAP --> RET
RET --> PG
RET --> GEN
GEN --> LLM
GEN --> VER
VER --> PG
VER --> OUT{{"outcome"}}
OUT --> A["answered<br/>run completed"]
OUT --> R["insufficient_evidence<br/>run completed"]
OUT --> F["failed<br/>verification rejected"]
PG --> AN
AN --> FEThree deployed pieces: a Next.js frontend on Vercel, a Python API service on Render, and a Postgres database on Neon (pgvector for the semantic index, tsvector for the lexical one).
Model and embedding access sit behind provider protocols, so the system runs entirely on deterministic stubs for cost-free CI, or on real providers when one is deliberately configured. The deployed values are dashboard-only; see §10.
The pipeline is linear and hand-specified. There is no agent deciding what to run next — domains/execution/service.py executes a fixed WORKFLOW_DEFINITIONS list and moves state only through transitions.py.
4. Components
Every component below is deterministic except where marked.
Ingestion & structure-aware chunking. Parses PDF, Markdown and text, then splits at real structural boundaries rather than fixed character windows: chapter/part/annex dividers, numbered clause anchors, decimal headings, bracket-prefixed footnote citations, lower-roman enumerated items, citation-vocabulary references, ALL-CAPS section headers, entry titles and bullets, and a general fallback for narrative prose with no structural signal at all. A document-level gate decides which ruleset applies, so a memo is not shredded into invented sections. Chunk overlap is boundary-aware: it carries trailing context from a sentence break, and falls back to a word boundary only in documents with no sentence structure anywhere. Generalized and blast-radius-tested across eight document families.
Hybrid retrieval. Combines a lexical channel (Postgres tsvector, always real) with a semantic channel (pgvector cosine similarity, real only when a real embedding provider is configured) by weighted Reciprocal Rank Fusion:
score = semantic_weight / (k + semantic_rank)
+ lexical_weight / (k + lexical_rank)with semantic_weight = 1.0, lexical_weight = 0.5, k = 60.
*Evidence-briefing generation — probabilistic.* The model produces a structured briefing in which every claim carries a verbatim quote asserted to exist in its cited chunk. The output schema holds only briefing, claims and insufficient_evidence — there is no field for model reasoning, so none is persisted. The step is registered with max_retries=2, so one investigation can involve up to three model calls.
Deterministic citation verification. Three ordered checks against the database (§5). This is the component the whole design exists to protect.
Content-safety classification. Pattern-based, severity-tiered scan of ingested content for injection-style attempts, distinguishing directive-form attacks from ordinary prose discussing similar topics in the abstract — regulatory text describing a prohibited technique reads differently from text instructing the system to perform one. A single HIGH-severity hit marks a chunk untrusted; a single MEDIUM hit marks it flagged.
Multi-tenant isolation. ORION_ALWAYS_ISOLATE_TENANTS=true on every public deployment. Registering without naming a workspace creates a fresh tenant-<uuid4hex> organization; naming an existing one returns a clean 409 rather than silently joining it. No cross-tenant or administrative inventory route exists on the deployed API.
Cost & abuse control. ORION_MAX_RUNS_PER_DAY_PER_TENANT caps investigations per workspace over a rolling 24 hours, enforced against the persisted runs table rather than an in-process counter — the API host is ephemeral and spins down when idle, so a counter in memory would reset on every cold start and bound nothing. Checked after the idempotency lookup, so a retry is never charged twice. Default None (unlimited): it fails open, because it is a cost control and not a security boundary. Note the retry multiplier above — a cap of N bounds up to 3N provider calls.
Investigation analytics. Investigation volume, refusal rate, evidence count per answered investigation, and most-cited documents — computed from persisted runs and steps records through domains/analytics' own describe() and time_series_trend(). Documents are counted by identifier, never by title, since two different documents may legitimately share one. Sparse data produces honest empty states rather than a chart.
11. What changed from the pre-verification draft
Checked line by line against the code at 071532d. Five claims did not survive; two sections were added.
- "Currently: Anthropic for the LLM" — removed. The deployed provider variables are dashboard-only with a code default of
fake, and nothing exposes the active provider, so the claim was unverifiable and most likely untrue. Superseded 2026-09-08: the LLM is now genuinely Anthropic in production, established by live verification rather than by assertion (§10). Removing it here was still right at the time — the claim was unverifiable when it was made, and it was only restored once there was evidence for it. - Per-claim filtering — corrected. The draft described unverifiable claims being "dropped," with refusal only when nothing was verifiable. The verifier rejects the entire run on the first failure.
- Two outcomes became three (§6, new). A refusal is a completed run; a verification failure is a failed one. The draft collapsed them, contradicting its own analytics description.
- Verification described as one check — expanded to three (§5, new). Membership in this run's own evidence was omitted entirely, and it is the check that stops a model citing a real document it never retrieved.
- Provider list corrected.
llm_providerisLiteral["fake", "anthropic", "ollama"]— three options, not two. - ADR attribution corrected. The stub-by-default decision belongs to ADR-0024 and ADR-0034; ADR-0019 records a different finding (no free local model reaches the bar).
- Retry multiplier added.
max_retries=2means one investigation can involve up to three model calls, which matters to any statement about bounded cost.
ORION 2.0 — Low-Level Designdocs/architecture/LLD.md
1. Database schema
Every id is varchar(36) holding a UUID4 string (String(36), not a native uuid column). Every timestamp is timestamptz.
1.1 Identity
`organizations` — the tenant boundary.
| Column | Type | Notes |
|---|---|---|
id | varchar(36) | PK |
name | varchar(255) | UNIQUE — this is what makes a taken workspace name a clean 409 |
created_at | timestamptz |
`users`
| Column | Type | Notes |
|---|---|---|
id | varchar(36) | PK |
organization_id | varchar(36) | FK → organizations.id, `ON DELETE RESTRICT` |
email | varchar(320) | UNIQUE (ix_users_email) |
hashed_password | varchar(255) | bcrypt |
is_active | boolean | |
created_at | timestamptz | |
tokens_valid_from | timestamptz | nullable — server-side token revocation cutoff |
users is the one FK in the schema that restricts rather than cascades: an organization holding users cannot be deleted out from under them. Every other tenant-owned chain cascades — documents → document_versions → document_chunks → document_embeddings, and runs → steps — so deleting an organization's data is one delete.
`roles`, `permissions`, `user_roles`, `role_permissions` — a standard RBAC join structure. permissions.code is varchar(150), roles.name is varchar(100); both join tables are composite-key only.
1.2 Knowledge
`documents` — stable logical identity, one row per (organization, source).
| Column | Type | Notes |
|---|---|---|
id | varchar(36) | PK |
organization_id | varchar(36) | FK, cascade |
source | varchar(1000) | caller-supplied stable identifier |
title | varchar(500) | display label — NOT unique |
document_type | varchar(20) | pdf | markdown | text |
created_at | timestamptz |
Constraint: uq_documents_org_source (organization_id, source). Identity is the source; two documents may share a title, which is why analytics keys on documents.id (ADR-0033).
`document_versions` — content-hash idempotency.
| Column | Type | Notes |
|---|---|---|
id | varchar(36) | PK |
document_id | varchar(36) | FK, cascade |
version_number | integer | |
content_hash | varchar(64) | sha256 of the raw bytes |
status | varchar(20) | |
source_metadata | jsonb | |
created_at | timestamptz |
Constraint: uq_document_versions_doc_hash (document_id, content_hash) — re-uploading identical bytes returns the existing version.
`document_chunks`
| Column | Type | Notes |
|---|---|---|
id | varchar(36) | PK |
document_version_id | varchar(36) | FK, cascade; indexed |
chunk_index | integer | |
text | varchar (unbounded) | normalized chunk text |
section | varchar(500) | nullable — hierarchical label, e.g. CHAPTER I > 3. Definitions > iv. |
page | integer | nullable |
char_start, char_end, char_count | integer | offsets into the original parsed text |
content_hash | varchar(64) | |
text_search | tsvector | generated; GIN index ix_document_chunks_text_search_gin |
trust_label | varchar(20) | trusted | flagged | untrusted |
matched_patterns | jsonb | which safety patterns fired |
created_at | timestamptz |
section is capped at 500 because a raw clause anchor can be the whole paragraph; uncapped it is a failed INSERT, not an ugly string (ADR-0026).
`document_embeddings`
| Column | Type | Notes |
|---|---|---|
id | varchar(36) | PK |
chunk_id | varchar(36) | FK, cascade |
provider_name, model_name, model_version | varchar | provenance of the vector |
dimension | integer | |
vector | vector(256) | pgvector; HNSW index, vector_cosine_ops, m=16, ef_construction=64 |
created_at | timestamptz |
Constraint: uq_document_embeddings_chunk_model (chunk_id, provider_name, model_name, model_version) — one chunk may hold vectors from several providers, but not duplicates from one. The column dimension is fixed at migration time, so a provider whose native dimension differs must truncate — the Ollama provider lops nomic-embed-text from 768 to 256. The Voyage provider does not: 256 is a natively supported output_dimension, so the model returns exactly the right width and no tail is discarded (ADR-0035).
1.3 Execution
`runs`
| Column | Type | Notes |
|---|---|---|
id | varchar(36) | PK |
organization_id | varchar(36) | FK, cascade |
workflow_type | varchar(100) | |
idempotency_key | varchar(255) | |
status | varchar(20) | pending | running | verifying | completed | failed | cancelled |
input | jsonb | |
result | jsonb | nullable — holds briefing, claims, insufficient_evidence |
failure_reason | varchar(500) | nullable |
max_steps, steps_executed | integer | hard loop ceiling |
created_at, started_at, completed_at | timestamptz | last two nullable |
Constraint: uq_runs_org_idempotency_key (organization_id, idempotency_key). Its leading column is what makes the per-tenant run-cap COUNT indexed without a new index.
`steps`
| Column | Type | Notes |
|---|---|---|
id | varchar(36) | PK |
run_id | varchar(36) | FK, cascade |
step_index | integer | |
name | varchar(100) | hybrid_retrieval | llm_briefing |
status | varchar(20) | |
input | jsonb | |
output | jsonb | nullable — retrieval citations live here |
retry_count, max_retries | integer | |
failure_reason | varchar(500) | nullable |
created_at, started_at, completed_at | timestamptz |
Constraint: uq_steps_run_index (run_id, step_index).
`execution_events` — append-only trace: run_id, nullable step_id, event_type varchar(50), payload jsonb, created_at.
1.4 There is no citations table
Citations are not a table. Retrieved citations are persisted inside steps.output->'citations' as a JSONB array; the claims that cite them live in runs.result->'claims'. Each citation object carries:
chunk_id, document_id, document_version_id, document_title, source_type, page, section, content, retrieval_score, retrieval_rank, retrieval_method
A claim's citation_ids are chunk ids, resolved back to documents through this run's own steps.output — never through a global lookup. That is the same rule the verifier enforces, and it is why analytics aggregates in Python over one query rather than joining a citations table.
2. API surface
Mounted in apps/api/main.py: auth at /api/v1/auth, admin at /api/v1/admin, knowledge at /api/v1/knowledge, execution at /api/v1, health unprefixed.
2.1 Ingestion — apps/api/routers/knowledge.py
`POST /api/v1/knowledge/documents` → 201 IngestResponse Multipart: file (≤ 10 MB, enforced by reading _MAX_UPLOAD_BYTES + 1), plus form field title.
Two size limits, answering different questions. The 10 MB byte cap guards server memory. The one that binds on the hosted tier is the embedding budget: after parsing and chunking, a document whose estimated tokens exceed what the provider can embed within max_ingestion_embedding_seconds is refused with 413 before any embedding call (DocumentTooLargeToEmbedError), stating the real numbers — about 32,000 tokens, ~125 KB of text, at Voyage free-tier defaults. Providers with no rate limit (fake, local Ollama) have no such budget. § ADR-0035's 2026-09-11 addendum.
IngestResponse: document_id, document_version_id,
version_number, status, chunk_count`GET /api/v1/knowledge/documents` → list[DocumentPublic]
DocumentPublic: id, source, title, document_type, created_at,
last_indexed_at, usage: DocumentUsagePublic | None
DocumentUsagePublic: times_retrieved, times_cited, times_in_refusedUsage is looked up by `document.id`, from one org-wide document_usage() call for the whole list.
`GET /api/v1/knowledge/documents/{document_id}` → DocumentDetail (as above plus versions: list). `DELETE /api/v1/knowledge/documents/{document_id}` → 204.
`POST /api/v1/knowledge/search` → list[SearchResultPublic]
SearchRequest: query_text, method, top_k, score_threshold, document_id
SearchResultPublic: chunk_id, document_id, document_version_id,
document_title, source_type, page, section, content,
score, rank, retrieval_method, trust_label,
semantic_rank, lexical_rank,
semantic_score, lexical_scoreThe per-channel rank/score fields are returned so a caller can see why a chunk ranked where it did, rather than only the fused score.
2.2 Investigation — apps/api/routers/execution.py
`POST /api/v1/runs` → 201 RunPublic
RunCreateRequest: workflow_type (Literal), input: dict, idempotency_key
RunPublic: id, workflow_type, status, input, steps_executed, max_steps,
failure_reason, created_at, started_at, completed_at,
steps: list, insufficient_evidenceError mapping: UnknownWorkflowError → 400; RunQuotaExceededError → 429 with a real Retry-After header and error code rate_limited; WorkflowInputValidationError → 422 through the central handler in apps/api/errors.py.
`GET /api/v1/runs` (limit ≤ 50), `GET /api/v1/runs/{run_id}`, `POST /api/v1/runs/{run_id}/cancel`, `GET /api/v1/runs/{run_id}/events`, `GET /api/v1/runs/{run_id}/summary`.
`GET /api/v1/runs/{run_id}/result` → RunResultPublic
RunResultPublic: run_id, status, result: dict,
citations: list, completed_at, refusal2.3 Analytics — same router
`GET /api/v1/investigations/analytics` → InvestigationAnalyticsPublic
total, answered, insufficient_evidence, failed, cancelled, completed, refusal_rate, # None when nothing completed evidence_mean, evidence_median, evidence_min, evidence_max, most_cited_documents: list[CitedDocumentPublic] CitedDocumentPublic: document_id, document_title, times_cited
`GET /api/v1/investigations/trends` → InvestigationTrendPublic
bucket, points, total_investigations, completed_investigations, enough_for_volume_trend, enough_for_refusal_trend, reason
The enough_for_* booleans and reason exist so the UI renders an honest empty state instead of a chart drawn over two data points.
2.4 Auth and health
POST /api/v1/auth/register → 201 UserPublic; POST /api/v1/auth/login → Token (rate-limited); POST /api/v1/auth/logout → 204 (revokes via tokens_valid_from); GET /api/v1/auth/me → UserPublic; GET /api/v1/admin/ping (requires role admin); GET /health (liveness, no DB), GET /health/ready (opens a connection).
/health deliberately does not touch the database: Neon's free compute sleeps, and a readiness-based platform check would report the service unhealthy because the database was idle (ADR-0024).
3. Module map
| Concern | File | Key names |
|---|---|---|
| Parsing | domains/knowledge/parsers.py | select_parser, PDFParser, MarkdownParser, TextParser, ParsedBlock, ParsedDocument |
| Chunking | domains/knowledge/chunking.py | chunk_blocks, ChunkDraft, _boundary_aware_overlap, _split_oversized_block |
| Normalization | domains/knowledge/normalization.py | normalize_text |
| Ingestion | domains/knowledge/ingestion.py | ingest_document, get_or_create_document, find_version_by_hash |
| Retrieval | domains/knowledge/retrieval.py | retrieve_hybrid, retrieve_semantic, retrieve_lexical, deduplicate_results, RetrievalRequest, RetrievalResult |
| Citations | domains/knowledge/citations.py | build_citation, validate_citation, Citation, CitationValidationError |
| Content safety | domains/knowledge/security.py | classify_content, ContentClassification |
| Embeddings | domains/knowledge/embeddings.py | EmbeddingProvider, FakeEmbeddingProvider, OllamaEmbeddingProvider, get_configured_embedding_provider, validate_dimension |
| Workflows | domains/execution/workflows.py | WORKFLOW_DEFINITIONS, WORKFLOW_INPUT_SCHEMAS, WORKFLOW_VERIFIERS, _hybrid_retrieval_step, _llm_briefing_step, _verify_evidence_briefing, _get_llm_provider, _get_embedding_provider |
| Orchestration | domains/execution/service.py | create_run, start_run, resume_run, cancel_run, _run_step, _verify_and_complete, _fail_run, _enforce_run_quota, RunQuotaExceededError |
| State machine | domains/execution/transitions.py | validate_run_transition, validate_step_transition, is_run_terminal |
| Grounding contract | domains/execution/schemas.py | quote_is_grounded, briefing_contract_violation, BriefingOutput, Claim |
| LLM providers | domains/execution/llm.py | LLMProvider, FakeLLMProvider, AnthropicLLMProvider, OllamaLLMProvider, LLMProviderError + subclasses |
| Analytics | domains/execution/analytics.py | investigation_analytics, investigation_trends, document_usage |
| Statistics | domains/analytics/ | describe, time_series_trend |
| Identity | domains/identity/service.py | create_user, authenticate_user, assign_role, revoke_tokens, _generated_tenant_name |
| Auth deps | domains/identity/dependencies.py | get_current_user, require_role |
| Tokens | domains/identity/security.py | hash_password, verify_password, create_access_token, decode_access_token |
| Login throttle | apps/api/rate_limit.py | FixedWindowRateLimiter, login_rate_limiter |
| Error contract | apps/api/errors.py | register_exception_handlers, _STATUS_CODES |
| Settings | packages/config/settings.py | Settings, get_settings |
Two distinct rate-limiting mechanisms, deliberately: rate_limit.py is an in-process fixed window guarding login against password guessing, and _enforce_run_quota is a database-backed count guarding spend. The second cannot use the first, because the host spins down when idle and an in-memory counter would reset on every cold start.
4. Single-investigation sequence
Real function names at every step.
Diagram source (Mermaid)
sequenceDiagram
autonumber
participant C as Client
participant R as routers/execution.py
participant S as execution/service.py
participant W as execution/workflows.py
participant K as knowledge/retrieval.py
participant L as LLMProvider
participant DB as Postgres
C->>R: POST /api/v1/runs
R->>S: create_run(...)
S->>DB: _find_run_by_idempotency_key()
S->>S: _enforce_run_quota(db, organization_id)
Note over S: 429 RunQuotaExceededError if over cap
S->>DB: INSERT runs + steps (pending)
R->>S: start_run(run)
S->>W: _hybrid_retrieval_step(db, run, step)
W->>W: _get_embedding_provider()
W->>K: retrieve_hybrid(db, request, provider)
K->>DB: retrieve_semantic() — pgvector cosine, HNSW
K->>DB: retrieve_lexical() — tsvector, GIN
K->>K: weighted RRF + deduplicate_results()
K-->>W: list[RetrievalResult]
W->>W: build_citation() per result
W->>DB: steps.output = {citations, count}
S->>W: _llm_briefing_step(db, run, step)
W->>W: build_evidence_briefing_prompt(query, citations)
W->>W: _get_llm_provider()
W->>L: generate_structured(system, user, BriefingOutput)
Note over S,L: failure retries via StepDefinition.max_retries=2<br/>up to 3 total calls
L-->>W: BriefingOutput
W->>DB: steps.output = briefing.model_dump()
S->>S: _claim_run_for_verification(run)
S->>W: _verify_evidence_briefing(db, run, steps)
W->>W: 1. membership in THIS run's citations
W->>DB: 2. validate_citation(db, citation)
W->>W: 3. quote_is_grounded(quote, chunk.text)
alt every claim passes
S->>DB: runs.status = completed
R-->>C: 201 RunPublic (answered)
else model declined
S->>DB: runs.result.insufficient_evidence = true
R-->>C: 201 RunPublic (refusal, completed)
else any claim fails verification
S->>S: _fail_run(run, "verification failed: ...")
S->>DB: runs.status = failed
R-->>C: 201 RunPublic (failed)
end5. Provider abstraction
Both providers are structural typing.Protocols — no base class, no registration, no inheritance. A conforming object is a provider.
5.1 EmbeddingProvider — domains/knowledge/embeddings.py
class EmbeddingProvider(Protocol):
provider_name: str
model_name: str
model_version: str
dimension: int
def embed_texts(self, texts: list[str]) -> list[list[float]]: ...
def embed_query(self, text: str) -> list[float]: ...embed_query is separate from embed_texts because real providers often embed queries and documents asymmetrically. embed_texts takes a list and implementations are expected to batch — OllamaEmbeddingProvider sends all chunks of a document in one request, so a 352-chunk document is one call, not 352.
Implementations: FakeEmbeddingProvider (sha256-seeded, L2-normalized, deterministic; explicitly not semantic — nearby texts are not nearby vectors), OllamaEmbeddingProvider (nomic-embed-text, truncated to 256 dims).
5.2 LLMProvider — domains/execution/llm.py
class LLMProvider(Protocol):
provider_name: str
model_name: str
def generate_structured(
self, *, system: str, user: str, schema: type[BaseModel]
) -> LLMGenerationResult: ...Contract: returns a response already validated against schema, or raises an LLMProviderError subclass — LLMTimeoutError, LLMProviderUnavailableError, LLMRefusalError, context-limit and malformed-output variants. Never a partially-valid result.
"Retry provider failures, never retry deterministic validation failures" holds structurally, not by convention: a provider failure is an exception raised inside a step handler, which _run_step catches and retries up to max_retries. A verification failure is not an exception — _verify_evidence_briefing returns (False, reason), and that happens in a separate phase (_verify_and_complete) that runs only once every step has already succeeded. There is no retry path reachable from it.
Stale reference, noted not fixed:
llm.py:40names adomains.execution.workflows.GroundingValidationErrorin prose. No such class exists anywhere in the codebase — the tuple return above is the real mechanism. Left alone here because this was a documentation pass; worth deleting from that docstring in a code change.
Implementations: FakeLLMProvider, AnthropicLLMProvider, OllamaLLMProvider.
5.3 Adding a provider
Selection happens in exactly one place per provider kind — workflows._get_llm_provider() and embeddings.get_configured_embedding_provider(). Nothing else names a provider or a model.
To add one (e.g. a Gemini embedding provider, which is the live candidate since ollama cannot be hosted on the current deployment):
- Write a class satisfying the Protocol in the same module as the existing implementations. Truncate or pad to
EMBEDDING_DIMENSION(256) —validate_dimension()will reject a mismatch loudly rather than let pgvector fail opaquely. - Extend the
Literalinpackages/config/settings.py(embedding_providerorllm_provider) and add any credential field, with amodel_validatorenforcing that the credential is present when that provider is selected — the pattern_anthropic_provider_requires_api_keyalready establishes. - Add a branch in the single factory function.
- Unit-test against a mocked transport, plus a live integration test that auto-skips when the service is unreachable — the pattern both
test_execution_ollama_live.pyandtest_knowledge_ollama_embeddings_live.pyuse. - Declare the credential as
sync: falseinrender.yaml. A real key never enters the repository.
Changing the embedding provider on an existing corpus requires re-ingestion: hash vectors and semantic vectors share one vector(256) column and are not comparable.
6. Configuration
packages/config/settings.py, ORION_-prefixed, validated at startup.
| Setting | Type | Default |
|---|---|---|
environment | local|test|ci|production | local |
database_url | str | required |
jwt_secret | str | required, ≥ 32 chars |
jwt_algorithm | HS256|HS384|HS512 | HS256 |
jwt_expire_minutes | int | 60 (0 < n ≤ 1440) |
cors_origins | str | http://localhost:3000 |
always_isolate_tenants | bool | False — `true` on every public deployment |
max_runs_per_day_per_tenant | int | None | None (unlimited), > 0 |
llm_provider | fake|anthropic|ollama | fake |
llm_model | str | claude-haiku-4-5 |
anthropic_api_key | str | None | None |
ollama_base_url | str | http://localhost:11434 — local-only by constraint |
voyage_api_key | str | None | None — required when provider is voyage |
voyage_embedding_model | str | voyage-4-lite (256 native dims) |
voyage_min_interval_seconds | float | 1.0 — pacing, see ADR-0035 |
max_documents_per_day_per_tenant | int | None | None (unlimited), > 0 |
embedding_provider | fake|ollama|voyage | fake |
Two flags default in opposite directions on purpose. always_isolate_tenants fails safe — it protects a security boundary, so a deployment sets it explicitly and the code never guesses. max_runs_per_day_per_tenant fails open — it is a cost control, and a global default would break every local checkout and CI run. Both are set in render.yaml so neither depends on being remembered.