"What was the torque value in the assembly instructions for this product?", "What does the dealer agreement say about the return period?", "Where does a new employee submit a leave request?" The answers to these questions sit in a folder, a PDF or an email somewhere in your company. The problem is finding them. An AI that answers from company documents, technically called RAG (Retrieval-Augmented Generation, meaning it "retrieves, then generates"), aims to answer such questions directly from your documents. In this post we explain how it works, what to watch for during setup and what to realistically expect.
How does RAG work?
The system works in two stages:
- Retrieval. When a question comes in, the system searches a prepared document archive for the passages most relevant to it. The search is based not only on matching words but also on similarity of meaning; ask about the "return period" and it can also find a paragraph about the "right of withdrawal."
- Generation. The passages it found are handed to the language model together with the question. The model writes its answer based on those passages and shows which document it came from.
This design has two big advantages: instead of making up what it doesn't know, the model looks at your documents, and when documents change you don't need to retrain the model, you just update the archive. We cover accuracy checking more broadly in our post on hallucinations and accuracy checks.
The real work of setup: preparing the documents
In RAG projects, most of the time goes into the documents, not the AI. The system is only as good as the documents you give it.
- Currency. If three versions of the same procedure sit in the archive, the system may find the outdated one. Obsolete documents should be removed or clearly marked.
- Readability. Scanned PDFs without a text layer, complex tables and handwritten notes cause trouble. They need text recognition (OCR) and review.
- Structure. Documents with clean headings and clear sections give much better results. Long documents are split into meaningful chunks before going into the archive.
- Ownership. Each group of documents should have someone responsible for keeping it up to date.
Access rights: not everyone should see every document
This is the most frequently overlooked issue. If HR files, salary tables, board meeting notes and customer contracts all go into the same archive with no permission checks, the AI can summarize that information for anyone who asks.
In a proper setup, the system knows which documents the person asking is allowed to see and searches only within those. Where possible, these permissions are mapped to the ones already in your file server or document management system. Documents containing personal data also need a separate review under KVKK (Türkiye's Personal Data Protection Law); we cover this in our post on giving company data to AI tools.
Where will it run?
A RAG system has three parts: the document archive, the search layer and the language model. The archive and search layer can live on your own server or in a cloud environment of your choice. For the language model there are two routes: a cloud-based AI service or an open-source model running on your own server. Cloud services usually deliver stronger answer quality; local models keep the data entirely in-house but need hardware and maintenance. The choice depends on how sensitive your documents are and on your budget. To keep the system current and secure, a hosting and maintenance plan should also be considered from the start.
Common setup mistakes
| Mistake | Result | Fix |
|---|---|---|
| Uploading every folder as is | Outdated and contradictory answers | Clean up documents first |
| Skipping permission checks | Confidential information leaks | Search limited per user |
| Not citing sources | Nobody trusts the answers | A link to the document and section in every answer |
| No test questions | Quality can't be measured | Regular tests with questions whose answers are known |
| One-off setup | The archive goes stale | A document update workflow |
The list of test questions is especially important. Ask your team for 30-50 questions whose answers they know for certain, and run the system against them after every change. That way you see whether quality is improving by measurement, not by gut feeling.
Realistic expectations
An AI that answers from documents is very good at cutting the time it takes to find information. But it cannot produce information that isn't in the documents, it cannot decide which of two contradicting documents is right, and it can make mistakes with complex tables. The healthiest way to use it is for the system to give the answer together with its source, and for users to check the source with one click on important matters.
How we do it at Globya
We start projects like this with a small group of documents, for example only technical data sheets or only internal procedures. We clean up the documents with you, set up the permission structure, prepare the test questions and expand the scope once results have been measured. Connecting the system to your existing file server, intranet or CRM is part of our integration and custom software work; from setup to maintenance, we are your single point of contact.
Frequently asked questions
Do we need to train the model on our own data for RAG?
No. The advantage of RAG is that it gives the model your documents at the moment a question is asked, without any training. When documents change, updating the archive is enough.
Which file types can be used?
Word files, PDFs, text files, web pages and emails are usually processed without trouble. Scanned documents and complex tables need extra preparation.
Does the system handle questions in Turkish well?
Current language models understand and write Turkish well, and English too. What really determines answer quality is whether your documents are clean and up to date.
How is access removed when an employee leaves?
If permissions are tied to the company's user management, closing the account also closes access to the AI. That is why keeping a separate user list is not recommended.
The Globya assistant is online 24/7; it answers right away and passes your question to the team if needed.