Accounting technology / Glossary / Retrieval-augmented generation (RAG)

What is retrieval-augmented generation (RAG)?

Definition

Retrieval-augmented generation (RAG) is a way of building AI tools in which the system first searches a defined set of documents, such as accounting standards, contracts or a company's own policies, and then has a language model write its answer from the passages it found. It lets the answer cite its sources and reduces made-up content.

Also calledRAGgrounded AIretrieval augmented generation

Compare the softwareBest AI accounting software →All 22 bookkeeping and accounting vendors →

Step one: Search a defined document set
Step two: Generate an answer from the results
Benefit: Answers cite their sources

How retrieval-augmented generation works

Documents are split into passages and indexed so that the system can find the ones most relevant to a question. When a user asks something, such as how a lease modification should be treated, the tool retrieves the matching paragraphs of the standard, the lease and any internal memos, and passes them to the language model with the question. The model answers using that material and points to the paragraphs it relied on, so a reviewer can check them. The quality depends on what is in the document set and how well retrieval finds it: an out-of-date standard or a missed clause leads to a wrong answer that still looks well sourced. In accounting, RAG sits behind research and technical accounting tools; Quillon traces each statement in its memos back to the source paragraph.

Common questions

Why does RAG matter for accounting?+

Accounting answers depend on specific text: a paragraph of a standard, a contract clause or a policy. RAG makes the AI work from that text and show it, so the answer can be checked.

Does RAG stop AI hallucinations?+

It reduces them, because the model works from retrieved passages, but does not remove them. The model can still misread a passage, and retrieval can miss the relevant one, so a person should review the cited sources.

What is the difference between RAG and training a model on your data?+

RAG looks documents up at the time of each question, so new or changed documents are used straight away. Training changes the model itself, which is slower and does not show where an answer came from.