WA Health Summary

Washington Health Summary (WAHS) · Architectural & Functional Audit

WAHS engine architectural and functional audit.

A code-level architectural and functional report on wahs-engine as it operates today (19 November 2026): how the standalone middleware connects patient-mediated WAHS frontends, zero-privilege HAPI FHIR proxies, and open-weight GPU inference; what is verified on main (85fb799) versus open full-stack pull requests (PR #10 at 193cf1a); and the concrete clinical, privacy, concurrency, and contract findings surfaced by a full module-by-module audit.

Audit date: 19 November 2026 · Audited revisions: wahs-engine@85fb799 (main) & PR #10 (193cf1a) · Test verification: 81 passed on main, 86 passed on PR #10

Executive assessment

  1. Clean decoupled foundation on main (85fb799): wahs-engine cleanly separates a pure-Python stdlib clinical library (wahs_core, 10 modules + 5 static clinical KB packs) from a 6-dependency FastAPI OpenAI /v1 service (wahs_engine, 8 modules) running non-root (65532:65532) with offline Gemma 4 and BPE tokenizers.
  2. Strong empirical clinical & safety performance: Across two 900-cell blinded two-judge H200 benchmarks, Gemma 4 26B + 4-Layer KB at a 16k chart budget achieved 85.1/100 (84.1 safety-adjusted, 0/75 confirmed unsafe answers, 3.42s median p50 latency, 100% budget compliance, 99.78% inline [N] citation UUID resolution, and 0/900 third-party privacy leaks).
  3. Active cross-repo frontier & targeted remediation queue: Full-stack 12-persona demos (shl-ips#2, wahs-mobile-frontend#2, and local 10-service Compose) currently target open wahs-engine#10. Before merging PR #10, three runtime regressions introduced on that branch (request-local HTTP client closure under single-flight coalescing, unbounded @<budget> GPU num_ctx allocation, and refresh=True dataset cache writeback) plus high-priority wahs_core status and sensitivity edge cases should be resolved.

24.1k LOC

18 Python modules, 5 clinical KB packs, and 8.5k+ lines of unit and live HAPI/jwt-proxy tests on main.

85.1 / 100

Top 75-scenario H200 score (Gemma 4 26B + KB @ 16k): 0/75 confirmed harms, 3.42s p50 wait.

99.78%

Inline [N] citation UUID resolution (5,191 / 5,203 cited indices) with 900/900 budget compliance.

27 Findings

16 in wahs_core clinical/privacy logic and 11 in wahs_engine runtime, streaming, and caller contracts.

On this page
  1. Architecture diagram
  2. Foundational pillars
  3. Repository snapshot
  4. wahs_core 4-layer audit
  5. FHIR R4 coverage
  6. Condenser & entity graph
  7. wahs_engine runtime audit
  8. Caller & repo alignment
  9. 900-cell H200 benchmarks
  10. Code audit findings
  11. Remediation roadmap

High-level system architecture

End-to-end dataflow, trust boundaries, and 4-layer clinical pipeline

The diagram below maps how wahs-engine sits between patient-controlled client BFFs, the zero-privilege fhir-auth HAPI FHIR R4 boundary, and stateless OpenAI-compatible model hosts. Every chat turn splits its prompt into a query-invariant chart prefix (messages[0], cached in GPU VRAM across follow-up turns) and a turn-local enrichment suffix (messages[-1], carrying precomputed temporal horizons, clinical entity graph edges, matched guidelines, and optional note passages).

WAHS Engine High-Level Architecture and 4-Layer Clinical Pipeline Three-column architecture diagram showing Client Surfaces and Zero-Privilege FHIR Boundary on the left, the standalone wahs-engine FastAPI middleware and embedded pure-stdlib wahs_core 4-layer clinical pipeline in the center, and Stateless Southbound OpenAI /v1 GPU Inference plus Northbound SSE and x_wahs AI-Provenance return envelopes on the right. CLIENT APPS & OIDC IDENTITY WAHS Web (:4000) & Mobile (:4001) shl-ips SvelteKit + Mobile Node BFFs Scope pills · /api/chat · Citation drawer Keycloak 26 (:8088) & SHL (:8092) RS256 OIDC JWT (jwt.sub tenant key) SMART Health Links (A256GCM JWE IPS) ZERO-PRIVILEGE FHIR BOUNDARY fhir-auth (:8091 · uwcirg/jwt-proxy) Verifies RS256 JWKS & meta.security Enforces meta.security.code == jwt.sub Blocks direct fhir-internal bypass HAPI FHIR R4 v8.0.0 (fhir-internal) MasterPatient.link[] star-graph topology Per-clinic + My Story / Goals datasets Patient/{id}/$everything + meta.lastUpdated EVALUATION & SHARED CORE Validation Harness (harness wahs) 4-pathway benchmark (chart/kb/rag/tools) Mode A inline bundle · 2-judge blind eval Med Agent Hub & WAHS Explorer Imports pure-stdlib wahs_core @ 85fb799 Interactive 256k-to-1k sandbox workbench wahs-engine Standalone Middleware (FastAPI :8080 · USER 65532) Zero HAPI admin credentials · Offline Gemma 4 & tiktoken tokenizers · Stateless HTTP wahs_engine Ingress, Star-Graph Assembler & Sizing app.py · jwt_claims.py · assembler.py · tokenizers_registry.py • Binds jwt.sub; probes MasterPatient + Patient/{ds} meta.lastUpdated • Bounded TTL=1800s LRU: _dataset_cache (256) + _prefix_cache (128) • Single-patient scope guard + HTTP 409 partial-fetch share block • derive_effective_chart_budget (window reserves, 16k default ceiling) Embedded Pure-Stdlib wahs_core (Zero External Dependencies) Layer 0 · Normalizers, Coded KB & Sensitivity Classifier normalizers.py + kb_lookup.py · RxNorm, SNOMED, LOINC, CVX/ACIP packs DS4P meta.security + 42 CFR Part 2 / WA MHMDA sensitivity labels Layer 1 · FHIR R4 Extractor (extractors.py · 16 Resource Types) Emits ClinicalItem[] (23 sections) + NumericPoint[] + SHA-256 note dedup 5-step Provenance & clinic Organization attribution; strips billing noise Layer 2 · 2-Stage Monotonic Condenser (condenser.py) Stage A: RLE folding + LOINC trajectories + [1] Patient + [2] Manifest Stage B: Hold-clause note excerpts + pass_tier=0 quotas + binary search Monotonic coverage: covered_uuids(B1) <= covered_uuids(B2) across 512..131k Layer 3 · Pre-Generation Clinical Enrichment & Tools ClinicalEntityGraph (11 node types, 7 edges, UCUM safety rules) TemporalDigest (trajectories & horizons) + SQLite FTS5 KB + BM25 RAG wahs_tools.py: search_chart_entries · get_lab_history · read_clinical_note Two-Region Prompt Assembly & Tool Loop (upstream.py) Region 1: messages[0] Query-invariant prefix: • WAHS system rules • [1]..[N] condensed chart Byte-identical across turns Region 2: messages[-1] Query-conditioned suffix: • Trimmed chat history • Digest + Graph + KB/RAG Sheds gracefully if tight SOUTHBOUND OPENAI /v1 UW CIRG cobalt H200 (141 GB) LiteLLM / Ollama southbound Gemma 4 26B / 12B / E4B num_ctx + reasoning_effort="none" Local & Agency Gateways Mac llama-router (:8077 GGUF) vLLM or OpenAI-compatible hosts enable_thinking=false (llamacpp) Stateless VRAM KV-Cache Reuses unchanged messages[0] < 0.3s follow-up prefill Zero session DB · No user JWT NORTHBOUND SSE & x_wahs Sanitized SSE Stream Strips <think> / thought channels Caps parallel tools <= 4 / round PR #10: 96-char live lookahead x_wahs.citation_map Binds [1]..[N] & [P1]..[PM] tags to FHIR ResourceType, UUID, clinical date, & source clinic HL7 AI-Provenance & Audit recorded_model · clamp_reason suspected_upstream_truncation human_review_required: true 1 Bearer JWT 2 $everything 3 Mode A / Core 4 No user JWT Tokens/Tools 5 SSE + x_wahs
wahs-engine middleware & x_wahs wire envelope Embedded pure-stdlib wahs_core (Layers 0–3) Zero-privilege fhir-auth boundary & stateless southbound model host Offline benchmark & shared-core R&D consumers
Trace summary: (1) Client BFFs forward the signed-in patient's Keycloak RS256 Bearer JWT and optional wahs.chart_scope to wahs-engine. (2) wahs-engine probes MasterPatient and linked Patient/{datasetId} metadata via fhir-auth before reading in-memory caches, fetching paginated $everything bundles only for new or modified datasets. (3) Validation Harness and Med Agent Hub exercise the identical wahs_core pipeline in offline/benchmark runs. (4) upstream.py sends the fixed messages[0] chart prefix and turn-local messages[-1] enrichment suffix to the model server with zero user credentials. (5) Sanitized SSE chunks and the final x_wahs provenance map return to the caller for clickable [N] source verification.

Foundational aspects & standards links

What we are building on: clinical standards, privacy law, and shared OpenClinAI primitives

Every design choice inside wahs-engine traces back to four foundational commitments: standards-based patient-mediated health data exchange, non-destructive deterministic context engineering, zero-privilege authorization with selective disclosure, and blinded empirical safety verification. Use the links below to inspect each specification, repository, and benchmark artifact directly.

Pillar 2 · Context & Clinical Rules

Deterministic 2-stage condensation and pre-generation clinical graphs

Open-weight 4B–26B models fail silently when multi-megabyte raw FHIR JSON overflows context windows or buries cross-clinic medication holds inside thousands of routine entries. Deterministic Python code normalizes, condenses, and links clinical entities before prompt assembly.

Pillar 3 · Zero-Privilege Auth & Privacy

Per-user security labels and selective external-share redaction

wahs-engine holds zero administrative credentials to HAPI FHIR and stores zero PHI transcripts on the model server. Privacy postures separate full patient self-review from sensitive-record exclusion during external sharing.

Pillar 4 · Empirical Validation & Reproducibility

Blinded multi-judge clinical benchmarks and interactive chart inspection

Every engine milestone is gated by deterministic token/citation invariants, live HAPI + jwt-proxy integration tests, and blinded multi-judge evaluations over synthetic multi-clinic and de-identified PhysioNet MIMIC-IV-on-FHIR cohorts.

Current repository snapshot (19 November 2026)

Module inventory and branch state: main (85fb799) vs. open pull requests

Between 13 and 15 November 2026, seven foundational pull requests (#1, #2, #3, #5, #6, #7, and #8) merged to pmanko/wahs-engine main, establishing the clean 4-layer architecture, offline tokenizers, typed clinical entity graph, and deterministic temporal digest. Two open pull requests (PR #10 and PR #11) hold the unmerged 12-persona full-stack seeder, live tool-lookahead SSE streaming, and profile-suffix routing.

Package / Subsystem Key Modules & Assets Size on main Runtime Role & Frontier Delta (PR #10)
src/wahs_core/
Pure Python stdlib
extractors.py (2,218L), clinical_graph.py (2,265L), condenser.py (1,784L), temporal_digest.py (771L), normalizers.py (492L), wahs_tools.py (463L), kb_lookup.py (383L), kb.py (304L), chart_serializer.py (106L) 8,893 LOC Zero third-party imports (enforced in CI via python -S). 0 lines changed between main (85fb799) and PR #10 (193cf1a).
src/wahs_core/kb_data/
Offline JSON / JSONL
clinical_exclusivity_and_privacy.json (753L), clinical_graph_rules.json (713L), rxnorm_lookup.json (500L), acip_rules.json (139L), wahs_corpus.jsonl (10 rows) 2,115 lines (~104 KB) Externalized RxNorm classes, cross-reactivity, KDIGO/ADA/AHA safety & monitoring edges, ACIP schedules, and DS4P/42 CFR Part 2 sensitivity rules.
src/wahs_engine/
FastAPI + 5 deps
upstream.py (1,473L), assembler.py (1,265L), seeder.py (838L), config.py (232L), app.py (198L), tokenizers_registry.py (128L), jwt_claims.py (123L), schemas.py (108L) + gemma4_tokenizer.json.gz (5.1 MB) 4,392 LOC
(+1,060 in PR #10)
OpenAI /v1 HTTP/SSE service, fhir-auth star-graph assembler, exact offline tokenizers. PR #10 expands seeder.py to 1,610L (12 personas + Keycloak 26 sync), adds parse_wahs_profile_spec, x_wahs_stream, and single-flight fetch coalescing.
tests/ & CI
81 / 86 passed
11 test suites including test_clean_layers_and_temporal_digest.py (1,942L), test_clean_layer_core_graph.py (1,458L), test_low_level_condensation_and_sizing_units.py (990L), test_live_hapi_jwt_proxy_integration.py (603L) 8,528 LOC
(+700 in PR #10)
3-job GitHub Actions matrix: stdlib purity + unit/contract tests, air-gapped --network none non-root Docker smoke, and live hapiproject/hapi:v8.0.0 + uwcirg/jwt-proxy RS256 integration.

Subsystem audit 1 · src/wahs_core/

Functional audit of the 4-layer clinical core

wahs_core transforms raw FHIR R4 bundles into numbered, token-budgeted clinical ledgers and structured pre-generation clinical context without mutating source records or calling external networks.

Layers 0 & 1: Normalization, provenance attribution, and FHIR R4 resource coverage

extract_bundle() (extractors.py ↗) runs a two-pass index over bundle entries, resolves source clinic organizations through a five-step fallback chain (direct serviceProvider/recorder/asserter/requester → performer[] → Encounter.serviceProvider → inverted Provenance.target[] → wahealthsummary-source-name meta tag), deduplicates base64 clinical notes by SHA-256 across DocumentReference and DiagnosticReport.presentedForm, and emits typed ClinicalItem (across 23 clinical sections) and NumericPoint records.

FHIR R4 Resource Category Extracted Types / Sections Current Behavior & Code-Audit Coverage Notes
IPS Mandatory Safety Core Patient, AllergyIntolerance, active Condition, active Medication*, active Goal, implant Device Pinned with mandatory=True across all budgets. Audit finding: only active Condition filters refuted/entered-in-error; erroneous or refuted AllergyIntolerance records currently retain mandatory=True, and MedicationStatement.dosage (a FHIR JSON list rather than dict) is silently skipped at extractors.py:1022.
Labs, Vitals, Notes & Imaging Observation, DiagnosticReport, DocumentReference, ImagingStudy Extracts numeric LOINC points (suppressing entered-in-error/cancelled points), comparator strings (<, >=), and base64 UTF-8 notes. Audit finding: DiagnosticReport.conclusion is never read, reports lacking both result[] and inline base64 presentedForm.data are dropped, and DocumentReference ignores docStatus="entered-in-error" and URL-only attachments.
Procedures, Immunizations, CarePlans & Visits Procedure, Immunization, CarePlan, Encounter, Appointment Separates high-acuity hospitalizations from routine visits, classifies terminal vs. planned vs. in-progress dates, and evaluates ACIP vaccine intervals. Audit finding: Procedure omits status filtering and display (rendering not-done like completed procedures), and cancelled/noshow Appointments receive date_kind="planned".
Counted Admin vs. Unhandled US Core Types 9 billing types + 6 lookup types counted in excluded_resource_counts Explicitly strips and counts Claim, ExplanationOfBenefit, Coverage, etc. Coverage gap: US Core / IPS types ServiceRequest, QuestionnaireResponse, FamilyMemberHistory, CareTeam, Composition, Consent, Flag, and AdverseEvent are neither extracted nor tallied in excluded_resource_counts.

Layers 2 & 3: Monotonic 2-stage condensation, 7-edge entity graph, and temporal digest

Layer 3a · clinical_graph.py

11 node categories & 7 typed clinical edges

  • High-priority safety edges (always eligible): CROSS_REACTS_WITH (RxNorm allergy cross-reactivity), CONTRAINDICATED_OR_DOSE_ADJUST_IN (UCUM-normalized eGFR, creatinine, directional potassium, INR, hemoglobin, glucose), CONFLICTS_WITH (duplicate therapy & stage conflicts), and CONTRADICTED_OR_MODIFIED_BY_NOTE (chronological hold-vs-resume reconciliation).
  • Query-gated & provenance edges: MONITORED_BY and TREATS fire only when matched to query concepts and cap cleanly when high-priority alerts are dense; REPORTED_BY_ORG tracks multi-clinic provenance.
  • Tier-0 overflow & FTS5 KB: Recovers omitted items on tight budgets with sequential [N+1..M] citations and queries in-memory SQLite FTS5 over wahs_corpus.jsonl.
Layer 3b · temporal_digest.py

Deterministic temporal horizons & privacy

  • Realized reference anchor: Computes reference_date strictly from completed, non-invalidated, non-administrative, non-redacted clinical events so future care-plan end dates or device manufacture dates never skew relative time windows.
  • Tie-safe lab trends: Refuses UUID tie-breaking on ambiguous same-day boundary values, suppresses single-point trends (n=1), and emits 6-month/12-month horizons plus upcoming vs. fulfilled appointments.
  • external_share isolation: Filters SDOH/PRIVACY-SENSITIVE items prior to Stage-A condensation, graph construction, KB code extraction, temporal digest calculation, and BM25 note chunking.

Subsystem audit 2 · src/wahs_engine/

Runtime service, caching, offline token sizing, and SSE orchestration

wahs_engine wraps wahs_core in a stateless FastAPI middleware exposing GET /healthz, GET /v1/models, POST /v1/wahs/context/preview, and POST /v1/chat/completions. Note that callers must use dedicated /v1/wahs/context/preview for context warm-up; WahsRequestOptions sets extra="forbid" and rejects unknown fields with HTTP 422.

Runtime Mechanism Verified Implementation (main @ 85fb799) Operational Caveats & PR #10 Changes
JWT Binding & fhir-auth Enforcement
(jwt_claims.py, assembler.py)
Extracts jwt.sub and enforces live HTTP probes against fhir_auth_base_url (MasterPatient + all linked Patient/{ds_id} stubs) before consulting in-memory caches. Rewrites HAPI _getpages links back onto fhir-auth and blocks external_share on truncated fetches with HTTP 409. decode_and_bind_jwt() does not verify RS256 signatures locally; security depends on fhir_auth_base_url pointing to uwcirg/jwt-proxy (currently guarded only by checking "fhir-internal" not in url). Also, HAPI does not bump Patient.meta.lastUpdated on in-place child resource edits without wahs.refresh=True.
In-Memory LRU Caches & Multi-Worker State
(assembler.py)
_dataset_cache (256 entries, 1800s TTL) keyed on (sub, ds_id, lastUpdated) and _prefix_cache (128 entries, 1800s TTL) keyed on (sub, datasets_fp, scope_hash). Clones citation_map per turn to prevent cross-request bleed. Caches are per-process and unbounded by byte size (large decoded clinical notes across multiple budgets can consume several GBs). Running multi-worker uvicorn --workers N splits cache hits and refresh=True invalidations across workers.
Offline Tokenizers & Window Reserves
(tokenizers_registry.py, assembler.py)
Bundles gemma4_tokenizer.json.gz and pre-cached tiktoken (o200k_base/cl100k_base) for zero-network exact sizing (<0.01% delta vs. Ollama). Deducts system framing, 12.5% enrichment/output reserves, and 15% conversation reserve, clamping to a 16,384-token default chart ceiling; _trim_history_to_fit pops oldest turns first. PR #10 adds @<budget> model suffixes (e.g. wahs-26b-kb@32k) that auto-lift chart_budget_ceiling and context_window, but currently lacks an upper cap on @<budget> (see Finding E2).
SSE Streaming, Tool Loop & <think> Stripping
(upstream.py)
Supports litellm/ollama (num_ctx, reasoning_effort="none"), llamacpp (enable_thinking=False), and openai flavors. Caps parallel tool calls at 4 per round (pairing skipped calls with error tool messages) and flags suspected_upstream_truncation. On main, stream=True buffers Round 1 whenever overflow tools are attached. PR #10 adds 96-char live lookahead streaming + x_wahs_stream events (context_ready, tool_round/reset_content), though reset_content requires WAHS-aware SSE parsers.

Cross-repository integration audit

Full-stack 10-service topology and contract alignment across all 4 callers

In the full WAHS stack (configured locally in hq/wahs/docker-compose.yml and staged for cloud deployment in openclinai.org#48 ↗), ten Docker services coordinate over an isolated network: Postgres 14.4 (wahs-db), Keycloak 26.1 (:8088), HAPI FHIR v8.0.0 (:8090 / fhir-internal), uwcirg/jwt-proxy (fhir-auth-proxy), NGINX CORS gateway (:8091 / fhir-auth), SMART Health Links server (:8092), one-shot wahs-seeder, wahs-engine (:8095), SvelteKit web app shl-ips (:4000), and Node companion wahs-mobile-frontend (:4001), backed southbound by macOS llama-router (:8077) or UW CIRG cobalt H200.

Downstream Caller Current Revisions Contract Alignment with wahs-engine (main vs. PR #10)
1. WAHS Web App BFF
pmanko/shl-ips ↗
main (63d183d)
Open PR #2 (17e4fc9)
main cleanly isolates chat history by (patientId, sharingPosture, chartScope) and defers default sizing to wahs-engine, but its /api/chat route drops retrieval_strategy/enable_kb and types completeness_manifest as a string instead of a dict. Open PR #2 forwards all strategy flags and adds 20 pathway options + @<budget>, requiring wahs-engine#10 merged first.
2. Mobile Companion BFF
pmanko/wahs-mobile-frontend ↗
main (f536318)
Open PR #2 (d70a827)
Active mismatch on main: UI dropdown exposes wahs-*-raw ("Condensed FHIR Only") and qwen3 IDs, but BFF strips retrieval_strategy and enable_kb, causing HTTP 400/404 southbound against wahs-engine main. Open PR #2 fixes strategy forwarding and normalizes child-proxy caregiver_proxy to patient_self, also requiring wahs-engine#10.
3. Shared R&D Engine
med-agent-hub →
main (532913d) pyproject.toml pins wahs-engine at exact main SHA 85fb799 and delegates extractors, normalizers, kb_lookup, wahs_tools, and tiered condensation to wahs_core. Remaining drift: server/fhir/clinical_graph.py (2,923L) is still a diverged local copy, temporal_digest is not yet wired, and Hub uses heuristic token estimation instead of tokenizers_registry.
4. Evaluation Runner
validation-harness →
main (59ad40a)
Open PR #219 (ee01bca)
WahsHubClient strips wahs-engine-{kb,rag,tools,chart}: and @<budget> client-side and sends explicit wahs JSON options, working cleanly with wahs-engine main up to 16k. Subtle trap: stripping @32k client-side bypasses PR #10's ceiling auto-lift unless WAHS_CHART_BUDGET_CEILING=none is set on the engine container; also _preview_wahs_engine_context omits model.

Empirical performance & clinical safety readout

Latest 900-cell NVIDIA H200 benchmarks across pathways, budgets, and Gemma 4 models

Following the Stage-B horizon, privacy-scoping, clean-layer graph, and deterministic TemporalDigest merges (PRs #5–#8), standalone wahs-engine was evaluated in two 900-cell blinded two-judge studies on UW CIRG's NVIDIA H200 (141 GB HBM3e): the 14 November 16k vs. 32k Calibration ↗ (confirming 16,384 tokens beats 32,768 tokens on overall accuracy, safety, and latency while eliminating pre-fix 32k truncation failures) and the 16 November 4-Pathway 75-Scenario Sheet Benchmark (evaluating exact 85fb799 wahs_core across 45 clinical Q01–Q15 and 30 privacy/sharing P01–P10 cases on Marcus Vance, MIMIC-IV 10014354, and Maurice Auer, staged in openclinai.org#49 ↗).

WAHS Engine Pathway & Model Arm Benchmark Safety-Adj. Confirmed Harms Median p50 [N] UUIDs Resolved
Gemma 4 26B · Condensation + 4-Layer KB (@16k) #1 Overall 85.1 / 100 84.1 0 / 75 (0.0%) 3.42 s 230 / 230 (100%)
Gemma 4 26B · Condensation + Note-RAG & KB (@16k) #1 Clinical Q01–15 83.4 / 100 82.2 1 / 75 (1.3%) 4.88 s 280 / 280 (100%)
Gemma 4 12B · Condensation + Note-RAG & KB (@16k) 82.8 / 100 81.6 0 / 75 (0.0%) 6.45 s 262 / 262 (100%)
Gemma 4 12B · Condensation + 4-Layer KB (@16k) 82.0 / 100 80.4 2 / 75 (2.7%) 5.65 s 264 / 264 (100%)
Gemma 4 12B · Condensed Chart Only (@16k) 79.1 / 100 77.4 1 / 75 (1.3%) 3.29 s 267 / 267 (100%)
Gemma 4 26B · Condensed Chart Only (@16k) 79.1 / 100 77.3 2 / 75 (2.7%) 2.23 s 245 / 245 (100%)
Gemma 4 E4B (4B eff.) · Condensed Chart Only (@16k) 76.9 / 100 74.8 2 / 75 (2.7%) 1.77 s 231 / 231 (100%)
Gemma 4 E4B · Condensation + 4-Layer KB / Note-RAG (@16k) 73.3 / 72.3 70.3 / 68.9 3 / 75 & 2 / 75 3.00 s / 8.55 s 667 / 667 (100%)
Agentic Tool-Calling Over 2k Overview (26B / 12B / E4B @2k) 69.4 / 66.7 / 65.8 64.4 / 61.8 / 60.4 7 / 7 / 10 of 75 2.57 s / 4.44 s / 2.08 s 432 / 436 (99.1%)
Key empirical takeaway: Pre-generation 16k structural condensation + deterministic TemporalDigest + 4-Layer Clinical KB lifted Gemma 4 26B by +6.0 points (to 85.1/100 with zero confirmed harms) and Gemma 4 12B by +2.9 to +3.7 points (to 82.0–82.8/100) over unenriched charts, whereas forcing models to discover clinical context iteratively via 2k tool calls concentrated 64.9% (24/37) of all confirmed unsafe answers and 93.3% (28/30) of all over-abstentions when tool queries missed cross-clinic hold notes or sensitive history.

Deep code-audit register

Concrete defects, edge cases, and technical debt identified in code review

Every module in src/wahs_core/ and src/wahs_engine/ was audited line-by-line across both main (85fb799) and open PR #10 (193cf1a). The tables below organize all significant findings by severity, exact file/line location, and failure mechanism.

Part A: wahs_engine runtime, streaming, security & PR #10 findings

ID & Severity Location & Revision Defect Mechanism & Clinical / Operational Impact
E1
P0 Critical
assembler.py:737, 894–933, 1034
PR #10 only
Single-flight coalescing closes request-local httpx.AsyncClient under shielded task: When fhir_http_client=None (production default), Request 1's request-local client is captured inside asyncio.shield(task) and shared via _inflight_datasets with concurrent Request 2. If Request 1 disconnects or times out first, its finally runs await client.aclose() while Request 2 is still awaiting page 2 of $everything, crashing Request 2 with RuntimeError: Cannot send a request, as the client has been closed.
E2
P0 High
config.py:72–79, 238–251, upstream.py:462
PR #10 only
Unbounded @<budget> suffix enables GPU VRAM OOM: _BUDGET_SUFFIX_RE accepts arbitrarily large integers (e.g. wahs-12b-kb@500000k), lifting context_window = req_b * 2 and forwarding "num_ctx": 1024000000 directly to Ollama/LiteLLM, bypassing configured WAHS_CONTEXT_WINDOW and WAHS_CHART_BUDGET_CEILING guardrails.
E3
P0 High
assembler.py:920–944
PR #10 only
refresh=True fails to overwrite _dataset_cache: Moving _dataset_cache.put() inside _run_dataset_fetch omitted writeback on the effective_refresh=True path. After an in-place HAPI edit (where Patient.meta.lastUpdated does not change), a refresh=True call updates _prefix_cache for one budget while leaving stale resources in _dataset_cache, resurrecting stale data on subsequent turns with a different budget or scope.
E4
P0 High
jwt_claims.py:81–102, config.py:81–87, assembler.py:738
main & PR #10
Zero local RS256 verification + substring-only proxy check: decode_and_bind_jwt() only supports optional HS256 secrets (incompatible with Keycloak RS256) and otherwise skips signature verification, relying entirely on upstream fhir-auth. config.py only checks "fhir-internal" not in url; misconfiguring WAHS_FHIR_AUTH_BASE_URL to an unproxied HAPI host would permit forged alg:"none" JWTs, and explicit master_patient_id skips checking MasterPatient identifier ownership locally.
E5
P0 High
upstream.py:149–155
main & PR #10
clean_model_text() leaks unclosed <think> blocks on length truncation: Requires closing </think> or <channel|> before stripping reasoning. When finish_reason == "length" truncates mid-thought in non-streaming complete() (or dynamic_fhir_tools streaming), raw chain-of-thought (potentially quoting sensitive items under Rule 5) is returned verbatim.
E6
P1 Medium
app.py:168–177, upstream.py:1281–1362
main & PR #10
stream=True commits HTTP 200 headers before prompt sizing & upstream connect: HTTPException(400/422) inside prepare_upstream_messages() or upstream httpx.HTTPStatusError occurs after Starlette starts the 200 SSE response, dropping the connection mid-stream instead of returning structured HTTP 400/422/502 errors.
E7–E11
P1 / P2
upstream.py, assembler.py, config.py, docs/ E7: PR #10 reset_content: true after 96-char preamble duplicates draft text in vanilla OpenAI SDKs, while dynamic_fhir_tools skips x_wahs_stream.
E8: Unchecked non-dict JSON tool arguments ("null", "[]") or non-numeric limit raises 500.
E9: Synchronous CPU condensation/BM25 runs on the main asyncio loop thread without asyncio.to_thread(), and per-request httpx.AsyncClient skips connection pooling.
E10: PR #10 resolve_model_spec misses cross_key, omits cfg.default_model suffix inference when req.model is None, and assigns gemma4 tokenizer to wahs-qwen3-*.
E11: top_p/stop/user and stream_options.include_usage are dropped southbound; Mode-A chart_scope ignores category filters; README.md and docs/architecture.md omit tokenizers/tiktoken and PR #10 contracts.

Part B: wahs_core clinical extraction, privacy & normalization findings

ID & Severity Location (src/wahs_core/) Defect Mechanism & Clinical / Privacy Impact
C1
P0 High
extractors.py:1062, 1857, 1913, 1958 + wahs_tools.py:30–34 Sensitivity bypass in wahs_tools via unclassified reasons/source_org/notes: extractors.py classifies sensitivity on Medication*, Procedure, CarePlan, and Goal using only (code, display), omitting reasonReference (e.g. OUD/HIV), source_org, and Goal.note interpolated into .text. While condenser.py and clinical_graph.py re-scan .text, wahs_tools._is_privacy_redacted() checks only item.sensitivity, allowing search_chart_entries to return those items under external_share.
C2
P0 High
wahs_tools.py:30, temporal_digest.py:419, clinical_graph.py:877 + kb_data/* Fail-open posture checks & uncoded MOUD/ARV lexical gaps: Direct calls to wahs_tools, build_temporal_digest, or ClinicalEntityGraph test sharing_posture == "external_share" without validating allowed literals (unlike condenser.py's fail-closed check), and no caregiver tier exists yet. When EHR exports lack DS4P meta.security tags, lexical fallback omits MOUD drugs (buprenorphine, suboxone, naltrexone), HIV antiretrovirals (biktarvy, tenofovir, dolutegravir), and WA MHMDA reproductive medications.
C3
P0 High
extractors.py:823, 929, 1147, 1227, 1627, 1845, 2189 Invalidated or unperformed FHIR statuses rendered as clinical events: entered-in-error/refuted AllergyIntolerance stays mandatory=True; Procedure renders not-done/entered-in-error identically to completed surgeries; DiagnosticReport appends invalid child Observation text into child_parts (L1627); DocumentReference ignores entered-in-error/superseded; not-done Immunizations enter series counts; and cancelled/noshow Appointments render as [planned].
C4
P0 High
extractors.py:304, 1022, 1262, 1461 Dropped DiagnosticReport.conclusion & MedicationStatement.dosage: DiagnosticReport.conclusion is never extracted (dropping radiology/pathology impressions when presentedForm is absent), and isinstance(res.get("dosage"), Mapping) fails on MedicationStatement.dosage[] lists and MedicationAdministration.dosage.dose.
C5
P1 Medium
normalizers.py:32, 177–188 _DOSE_NOISE salt stripping corrupts electrolyte supplements: Stripping sodium, potassium, and calcium turns Potassium Citrate into "citrate" (missing hyperkalemia hazard rules) and collapses Sodium Bicarbonate and Potassium Bicarbonate into "bicarbonate" (false duplicate-therapy alert).
C6
P1 Medium
clinical_graph_rules.json:439, 661–690, clinical_graph.py:176, 1584, condenser.py:796 Note-directive punctuation & HbA1c hypoglycemia false positives: Trailing spaces in note_resume_or_continue_cues ("resumed ") miss sentence-final "Lisinopril was resumed."; "outside" in hold cues triggers false alerts on "outside pharmacy"; _HOLD_CLAUSE_RE misses passive "Metformin was held."; and high HbA1c (hyperglycemia) triggers a false Hypoglycemia risk edge via is_abn fallback at clinical_graph.py:1584.
C7–C10
P1 / P2
clinical_graph.py, temporal_digest.py, extractors.py, kb.py C7: clinical_graph.py:1030 groups trajectories without unit and allows same-day comparator observations to overwrite exact values; temporal_digest.py:194 invalidates multi-year trends when any point lacks a UTC suffix, omits comparators, and counts "on-hold" meds as stopped.
C8: Dead KB schema keys in normalizers.py (irule["label"], unimplemented heart_failure_stage, hardcoded 14-day vs. 28-day vaccine duplicate window).
C9: _coding_pair() inspects only coding[0], missing SNOMED/LOINC/RxNorm codes when ICD-10-CM or local codes appear first; extract_bundle computes reference_date before external_share redaction.
C10: kb.py shares an unlocked sqlite3.Connection(check_same_thread=False) across threads, and src/wahs_core/__pycache__/ carries stale .pyc files from retired modules.

Prioritized engineering remediation roadmap

  1. Unblock and harden wahs-engine PR #10 before full-stack merge Immediate · P0

    Move Mode-B HTTP calls onto a shared lifespan httpx.AsyncClient pool (fixing E1), enforce an upper cap on @<budget> suffixes against a max operator ceiling (fixing E2), restore _dataset_cache.put() on effective_refresh=True (fixing E3), and fix resolve_model_spec cross_key/Qwen tokenizer mapping (E10) so wahs-engine#10, shl-ips#2, and wahs-mobile-frontend#2 can merge cleanly.

  2. Close wahs_core privacy, FHIR status, and clinical-rule precision gaps Next · P0 / P1

    Classify full ClinicalItem.text at extraction time and make wahs_tools/temporal_digest/clinical_graph fail-closed on unknown sharing_posture values while expanding coded + lexical MOUD/HIV/MHMDA sensitivity dictionaries (C1, C2); filter entered-in-error/refuted/not-done statuses across allergies, procedures, child panel observations, notes, and appointments (C3); extract DiagnosticReport.conclusion and list-valued MedicationStatement.dosage (C4); and fix electrolyte salt normalization, word-boundary resume cues, and HbA1c directionality (C5, C6).

  3. Harden wahs_engine defense-in-depth auth, SSE pre-validation, and async scaling Runtime · P0 / P1

    Strip unclosed <think> blocks on finish_reason="length" in clean_model_text() (E5), run prepare_upstream_messages() before returning StreamingResponse so sizing errors return HTTP 400/422 cleanly (E6), add optional Keycloak JWKS RS256 verification alongside fhir-auth enforcement (E4), guard malformed LLM tool arguments (E8), and offload CPU-heavy condensation/BM25 calls via asyncio.to_thread() with byte-bounded cache eviction (E9).

  4. Converge downstream callers and documentation Ecosystem · P2

    Replaces med-agent-hub's diverged server/fhir/clinical_graph.py with wahs_core.clinical_graph + temporal_digest, pass model on Validation Harness context-preview calls and align @32k ceiling handling, and synchronize wahs-engine/docs/architecture.md and /wahs/engine-reference/ with the 7-edge graph and x_wahs wire contract.

Back to top