Clearing a 7,000-case review backlog without moving regulated documents off-prem

Context and stakes

A growing review backlog was trapped between document volume, slow manual search, and a hard data boundary. Sending source material to a hosted model was not an acceptable trade.

The useful question was not whether a local model could answer questions. It was whether a local system could make reviewers faster without concealing OCR, retrieval, or citation failures.

My role

I led the applied ML and deployment work: document parsing, chunking, retrieval, local inference, containerization, reviewer experience, evaluation, and the operational handoff needed to keep the system maintainable.

Architecture

Documents remained inside the controlled environment. OCR produced page-addressable text and quality signals. Chunking preserved section and page provenance. Retrieval returned evidence spans before the model produced a structured answer.

The reviewer surface placed extracted fields beside citations and uncertainty reasons. Low-quality OCR, weak retrieval, conflicting dates, and missing evidence routed to explicit fallback states.

Why local inference

The local boundary reduced data movement and increased deployment control. It also imposed real costs: model selection was narrower, GPU capacity had to be planned, upgrades needed regression tests, and operations owned more of the serving stack.

What failed or was unreliable

Long documents and repeated boilerplate degraded naive chunking. Tables and low-resolution scans caused OCR defects. Retrieval similarity alone over-ranked plausible but irrelevant sections.

Section-aware chunks, page citations, document-family evaluation, and reviewer feedback loops performed better than adding prompt complexity.

Evaluation and human review

The acceptance set measured OCR coverage, retrieval recall, structured-field accuracy, citation correctness, and reviewer time. A response without an evidence span could not be treated as complete.

Reviewers remained the authority for conflicts, low-quality source pages, and policy interpretation. The system reduced search and transcription work; it did not remove accountability.

Outcome

The system helped clear a 7,000-case backlog and reduced review time by approximately 90% in the measured workflow. Both claims are résumé-supported and intentionally scoped.

What I would improve next

I would add ongoing OCR-quality sampling, retrieval drift reporting by document family, and a formal reviewer-disagreement queue that feeds the next evaluation release.