On-prem RAG and OCR
Clearing a 7,000-case review backlog without moving regulated documents off-prem
A local document-review system combining OCR, retrieval, small language models, and a reviewer surface inside the data boundary.
- Problem
- A 7,000-case review backlog was trapped behind slow manual search, while regulated documents had to remain inside the controlled environment.
- Decision
- Keep inference local and require page-addressable evidence, OCR quality signals, and explicit fallback states for reviewers.
- Business / proof change
- Cleared the 7K-case backlog and reduced review time by about 90% in the measured workflow.
- Ownership
- Technical lead for applied ML, local deployment, evaluation, and workflow integration
- Scale / team
- Payer modernization program with engineering, operations, and review stakeholders
- Evidence
- 2 source notes · 3 explicit limits
Cleared a 7K-case backlog and reduced review time by about 90% in the measured workflow.
The answer is only as useful as its source trail.
Inspect a page, the extracted field, and the reason a local workflow routes the case to review.
Context and stakes
A growing review backlog was trapped between document volume, slow manual search, and a hard data boundary. Sending source material to a hosted model was not an acceptable trade.
The useful question was not whether a local model could answer questions. It was whether a local system could make reviewers faster without concealing OCR, retrieval, or citation failures.
My role
I led the applied ML and deployment work: document parsing, chunking, retrieval, local inference, containerization, reviewer experience, evaluation, and the operational handoff needed to keep the system maintainable.
Architecture
Documents remained inside the controlled environment. OCR produced page-addressable text and quality signals. Chunking preserved section and page provenance. Retrieval returned evidence spans before the model produced a structured answer.
The reviewer surface placed extracted fields beside citations and uncertainty reasons. Low-quality OCR, weak retrieval, conflicting dates, and missing evidence routed to explicit fallback states.
Why local inference
The local boundary reduced data movement and increased deployment control. It also imposed real costs: model selection was narrower, GPU capacity had to be planned, upgrades needed regression tests, and operations owned more of the serving stack.
What failed or was unreliable
Long documents and repeated boilerplate degraded naive chunking. Tables and low-resolution scans caused OCR defects. Retrieval similarity alone over-ranked plausible but irrelevant sections.
Section-aware chunks, page citations, document-family evaluation, and reviewer feedback loops performed better than adding prompt complexity.
Evaluation and human review
The acceptance set measured OCR coverage, retrieval recall, structured-field accuracy, citation correctness, and reviewer time. A response without an evidence span could not be treated as complete.
Reviewers remained the authority for conflicts, low-quality source pages, and policy interpretation. The system reduced search and transcription work; it did not remove accountability.
Outcome
The system helped clear a 7,000-case backlog and reduced review time by approximately 90% in the measured workflow. Both outcomes remain attached to that workstream.
What I would improve next
I would add ongoing OCR-quality sampling, retrieval drift reporting by document family, and a formal reviewer-disagreement queue that feeds the next evaluation release.
Source notes
What you can inspect—and what remains private.
Career record
Backlog and review-time outcomes are supported by the current career record.
Related public pattern
A separate public local-document-AI experiment illustrates adjacent techniques; it is not the professional system.
Open source ↗Limits
- No regulated source data is available publicly.
- Public repositories are related experiments, not replicas.
- Local serving details vary with approved infrastructure.
No regulated documents, member information, screenshots, prompts, or private retrieval content are published.
Illustrative reviewer route
review_case:
source: synthetic-policy-17.pdf
pages: [4, 5]
retrieval_score: 0.84
extraction_state: qualified
route: reviewer_confirm
reason: conflicting_effective_dates