RAG inside a regulated institution: useful, if you respect the boundaries.

Inside every large institution there is a quiet tax on everything: the time it takes to find the authoritative answer. What does the policy say about this exception? Which version of the procedure is current? What are the documentation requirements for this product? The answers exist, in hundreds of documents, and a small set of veterans serve as the human search engine.

Retrieval-augmented generation, RAG, addresses exactly this. Instead of asking a model to answer from its general training, the system first retrieves the relevant passages from your own document corpus and instructs the model to answer from those passages, with citations. Done properly, the assistant answers from your rulebook, not from the internet's.

The boundaries that make it safe

• Citations or nothing. Every answer must show the source passages. An answer the employee cannot verify is a liability, not a productivity gain. In our deployments, the citation is the feature; the prose is just packaging.
• Access control carries through. The assistant must respect document permissions. If an analyst cannot open the credit committee minutes, the assistant must not quote them to her. This is an architecture decision made on day one, not a patch.
• Your environment, your data. For regulated workloads, retrieval and generation can run inside your cloud tenancy or on-premise. Sending internal policy documents to a third-party service is a decision for your risk function, not a default.
• A curated corpus. RAG over an unmanaged file share retrieves obsolete drafts with confidence. The corpus needs an owner, version control, and a process for retiring superseded documents.
• An evaluation set. Before launch, a few hundred real questions with known correct answers, scored for accuracy and for refusal behavior. The assistant must say "I do not find this in the corpus" rather than improvise.

What good looks like

The strong deployments are unglamorous: a branch operations team that gets policy answers in seconds with paragraph references, a compliance team that asks "what changed between version 6 and 7" and gets a sourced summary, a contact center where new hires perform like second-year staff because the rulebook sits in the workflow. Measured properly, the wins show up as fewer escalations to the expert team and faster cycle times on routine work.

The failure modes, named

Three failures account for most RAG disappointments in regulated settings, and all three are preventable:

• The confident answer from the obsolete draft. Retrieval does not know that version 7 superseded version 6 unless the corpus does. Prevention: one owner per document family, version control, and a retirement process. Corpus hygiene is the unglamorous half of RAG.
• The permission leak. The assistant retrieves a document the asking employee cannot open and paraphrases it helpfully. Prevention: retrieval filtered by the user's actual entitlements, tested with accounts at each permission tier before launch.
• The stitched-together answer. The model combines two passages into a policy that exists in neither. Prevention: instruction to answer only from retrieved text, refusal when retrieval is weak, and an evaluation set that scores refusal behavior as strictly as accuracy.

What a credible evaluation looks like

Before launch, assemble two to three hundred real questions from the teams who will use the assistant, each with the correct answer and its source passage agreed by the document owner. Include questions the corpus cannot answer; the correct behavior there is a clean refusal. Score accuracy, citation correctness, and refusal discipline separately. A practical bar for go-live in a regulated setting: above 90 percent on answerable questions, near-zero fabricated citations, and refusals on at least 95 percent of out-of-corpus questions. Re-run the same set after every corpus or model change. The evaluation set is a permanent asset; treat it like one.

Build or buy, decided honestly

The market now offers credible off-the-shelf assistants, and for some institutions they are the right answer. The decision turns on four questions. Where may the documents go? If policy or regulation keeps them inside your tenancy, the vendor list shortens fast, and an in-environment build moves up. How specific is your access model? Off-the-shelf products handle simple permission tiers well and complex, role-by-department entitlements poorly. Who maintains the corpus? A product does not curate your documents; that obligation lands on you either way, and it is the larger half of the work. And can you evaluate it? If a vendor cannot run your evaluation set and show the scores, you are buying the demo, not the system. Institutions that answer these four questions before the procurement usually find the choice makes itself; the expensive failures come from choosing the tool first and discovering the constraints after.

The practical first step

Choose one document family with high question volume and a clear owner: HR policy, product manuals, or operational procedures. Build the corpus, the access model, and the evaluation set, then pilot with one team. Resist the temptation to index everything on day one. A trustworthy assistant over one corpus beats a plausible one over ten.