The Request Every Company Makes

"We want a chat that knows everything about our company." It is the most common AI request I hear, and it is a reasonable one: the knowledge exists — in a wiki, a shared drive, five years of support tickets and a few thousand PDFs — it is just unfindable. The technique that makes this work is retrieval-augmented generation, or RAG. The idea is unglamorous: before answering, find the relevant fragments of your data and hand them to the model along with the question.

How It Works, Without the Jargon

• Your documents are split into small, meaningful pieces
• Each piece is indexed so it can be found by meaning, not just by keyword
• A question triggers a search that returns the handful of most relevant pieces
• The model answers using only those pieces, and cites where each fact came from

The model is not "trained on your data" — nothing is memorised. That is a feature: you can revoke a document, update a price list or remove an employee's access and the answers change immediately.

Why First Attempts Usually Fail

The demo always works. The pilot with real users is where it falls apart, and almost always for reasons that have nothing to do with the model.

• The source data is contradictory — three versions of the same policy, none marked current
• Retrieval is the weak link: if search returns the wrong five paragraphs, no model can save the answer
• Access rights are ignored, so the assistant cheerfully quotes salary data to an intern
• There is no evaluation set, so nobody can tell whether a change made things better or worse
• Nobody owns the content, so the index rots within a quarter

What a Serious Implementation Includes

• Permission-aware retrieval — the assistant sees exactly what the asking user is allowed to see
• Citations on every answer, linking back to the source document
• A refusal path: "I don't have that" beats a confident invention
• A test set of real questions with known correct answers, run on every change
• Logging of questions, so you learn what people actually need
• A clear owner for the content itself, not just for the software

Where It Pays Off Fastest

The best first project is narrow, high-volume and low-risk. Customer support answers, internal onboarding questions, product specification lookups, tender and contract search — anything where a person currently spends fifteen minutes finding a document they already know exists. Start there, measure the time saved, and expand only after the first case is genuinely trusted.

Conclusion

Connecting AI to company knowledge is less an AI project than a data hygiene project with an AI interface. The companies getting real value are not the ones with the biggest model — they are the ones who cleaned up their documents, respected their access rules and measured the answers. The technology is ready. The question is whether your knowledge is.