Retrieval-augmented generation had become a common way to deploy large language models for legal analysis, clinical documentation, and regulatory auditing. Existing automated evaluations judged the pipeline as one black box. They mixed retrieval errors with generation errors and did not check whether a cited page was the page that contained the evidence. This study built a dual-stage ground-truth cross-verification framework. Stage 1 scored retrieval against document and page anchors before any answer was written, using document hit rate, page hit rate, mean reciprocal provenance rank, and span containment. Stage 2 scored semantic correctness and citation fidelity, then placed each question in one of four diagnostic quadrants. The test set contained 1,200 audit questions over 8,520 pages of pharmaceutical validation standards, including United States Food and Drug Administration Title 21 Code of Federal Regulations Part 11 and European Union Annex 11. On the strongest retriever, document hit rate at five was 95.6 percent and page hit rate at five was 85.3 percent, a gap of 10.3 percentage points, and mean span containment was 0.88. With that retriever, a reranker, and GPT-4o, 80.1 percent of answers were fully grounded and 8.1 percent were correct in meaning but cited the wrong page. The same generator with sparse retrieval produced fabricated citations on 19.6 percent of questions. When the true passage sat in the middle of the prompt, extraction success fell by 19.1 percentage points for Llama-3.3-70B-Instruct. Separating provenance from generation therefore gave an auditable account of where these systems fail.
Retrieval-augmented generation had become a common way to deploy large language models for legal analysis, clinical documentation, and regulatory auditing. Existing automated evaluations judged the pipeline as one black box. They mixed retrieval errors with generation errors and did not check whether a cited page was the page that contained the evidence. This study built a dual-stage ground-truth cross-verification framework. Stage 1 scored retrieval against document and page anchors before any answer was written, using document hit rate, page hit rate, mean reciprocal provenance rank, and span containment. Stage 2 scored semantic correctness and citation fidelity, then placed each question in one of four diagnostic quadrants. The test set contained 1,200 audit questions over 8,520 pages of pharmaceutical validation standards, including United States Food and Drug Administration Title 21 Code of Federal Regulations Part 11 and European Union Annex 11. On the strongest retriever, document hit rate at five was 95.6 percent and page hit rate at five was 85.3 percent, a gap of 10.3 percentage points, and mean span containment was 0.88. With that retriever, a reranker, and GPT-4o, 80.1 percent of answers were fully grounded and 8.1 percent were correct in meaning but cited the wrong page. The same generator with sparse retrieval produced fabricated citations on 19.6 percent of questions. When the true passage sat in the middle of the prompt, extraction success fell by 19.1 percentage points for Llama-3.3-70B-Instruct. Separating provenance from generation therefore gave an auditable account of where these systems fail.