1. Why an AI knowledge base needs a structured approach
A company knowledge base is where the organisation gathers what it knows: operating procedures, standard contracts, product manuals, internal regulations, minutes and work instructions. Connected to an artificial intelligence model, it lets you ask a question in natural language and receive an answer built on the documents, instead of searching by hand through several folders. The benefit is real only if the answers are correct and if whoever reads them can trace them back to the source document, with its version and the exact passage.
The most widespread technique is called RAG, short for Retrieval-Augmented Generation. The system first searches for the relevant passages in the documents, then the model writes the answer based on those passages. It is not an installation to be completed in a day: it requires decisions about content, structure, permissions and testing. Every error made at one of these stages is repeated, identically, in every answer the base produces, which is why it pays to build the process in order.
2. Prerequisites: documents and responsibilities before systems
Before choosing any technology you need an inventory of documents. Not everything that exists in shared folders deserves to be in the base. Drafts, duplicate copies, superseded versions and documents that contradict each other produce contradictory answers even when the system works well from a technical standpoint. Ask each area manager which documents are official, who approves them, who updates them and how often. The answers to these questions are worth more than any demonstration.
It is advisable to fix three rules before the first line of configuration. The first is one owner for each collection, responsible for approving content and deciding which version is in force. The second is a minimum classification of documents, at least between internal use, confidential and containing personal data, because permissions depend on this distinction. The third is a written decision on where the data resides and who can access it, to be shared with the supplier before any file is uploaded.
3. The build steps in order
The process follows a logical sequence. Skipping a step does not make it disappear: it moves the problem into wrong answers, which surface once users are already working and nobody has time to check them. Each stage produces a verifiable result, which also helps justify the choice to management and to whoever oversees security. Follow the sequence below and do not move to the next stage until the previous one has been approved by the content owner, with a written note recording the decision.
- Define the use cases: which questions the base must be able to answer, and for which groups of people.
- Select the official sources and discard superseded copies, keeping a single valid version for each topic.
- Convert the documents into readable text, including reading scanned pages where needed.
- Divide the texts into coherent sections and attach metadata to each: title, date, version, owner and confidentiality level.
- Build the search index by combining search by meaning with search for exact words.
- Link the permissions of the source documents to the answers, so that nobody receives passages they are not allowed to read.
- Generate answers that cite the document, the section and the version used.
- Evaluate the system with real questions before opening access, and repeat the evaluation after every significant change.
4. The technical choices that emerge along the way
During construction four technical decisions come up that deserve separate study. You do not need to read them all at once: start with the one you meet first and test it against your real documents, not against generic examples or the cases the supplier has already solved. The choices influence one another, so it is wise to make them in the same period and track the linked decisions, so you do not have to revisit them one at a time when something stops working.
The practical advice is not to choose a technology because it is fashionable. Each decision should be tested with a small set of real documents and questions, and confirmed only if it improves the answers compared with the simplest alternative. If the simplest solution meets the use case, it is almost always the one to prefer, because it is easier to check, update and explain to whoever signs off the project. Complexity must be justified by a measured result, not by a presentation.
| Decision | Question to ask | Topic to explore |
|---|---|---|
| How to split the texts | Which parts must stay together to make sense | Document chunking and splitting strategies |
| Which embedding model to use | How to measure quality on Italian-language texts | Choosing an embedding model for Italian documents |
| Where to search for passages | Is a vector store needed, or is traditional search enough | Vector database or traditional search |
| How to read scans | Can the text be extracted well, or is visual recognition needed | Traditional OCR or multimodal models |
5. Permissions: each person sees only what they are allowed to read
A knowledge base does not isolate data by default. If all documents end up in a single searchable archive open to everyone, a collaborator may receive in an answer passages from a confidential file, even without ever having opened it. Access control must be applied to the source documents and enforced at the moment the answer is generated, not only in the interface that shows the list of results. A filter forgotten at this point goes unnoticed until someone discovers it by chance.
Verify three things: that the search takes into account the groups each user belongs to, that revocations propagate in a predictable way, and that the logs record who queried which documents. This topic deserves a dedicated study, because an error here is a breach of personal or confidential data, with consequences well beyond a wrong answer. Ask the supplier to demonstrate the filter with a test, not simply to confirm that it exists.
6. Testing and updating: how to know it works
Before opening the system, prepare a set of real questions collected from the colleagues who would use the base, each with the correct answer and the document that contains it. This set is your yardstick. Without it every answer seems plausible and nobody can say whether it is right, because fluent text does not prove correctness. Include questions the base should not answer, to check that the system says it does not know rather than inventing an answer.
Measure two aspects separately. If the system retrieves the right document, the problem lies in search. If it retrieves the right document but the answer is not faithful to the text, the problem lies in generation. After every change to the documents, the model or the permissions, the evaluation must be repeated, otherwise the base will degrade without anyone noticing. A dashboard with a single score is not enough, because it hides exactly the difference needed to correct the system.
7. Common mistakes to avoid
The first mistake is starting from the model rather than the documents. A brilliant technical choice does not compensate for a collection full of conflicting versions. The second is opening the base to the entire company without a content owner, so wrong answers keep circulating and nobody corrects them. The third is neglecting updates: a document changed after indexing continues to produce old answers, with the risk that someone acts on outdated information and still considers it valid.
The fourth mistake is using the base for decisions that affect people, such as staff selection or assessments, without a preliminary review. In these cases it may be worth asking an adviser whether the project falls under the obligations of the European AI Regulation. In the interface, always state that the answer is generated and must be checked against the original document, especially when it concerns deadlines, amounts or obligations.
8. How to start this week
You do not need to install anything to begin. Choose an area with few official documents and frequent questions, appoint the content owner and gather a set of real questions with the expected answers. This work takes a few days and decides more than any technical choice, because it immediately shows which documents are missing, which contradict each other and which permissions must be defined before anything else.
Then ask the supplier for a trial on one of your documents, using the same questions, and verify in writing where the data is processed. Only at that point does it make sense to compare solutions and approve a plan with owners and timelines. A project built on these foundations is easier to check and correct over time, and does not depend on the memory of a single person.
Frequently Asked Questions
Do we need an AI model trained on our documents?
As a rule, no. In most projects the general model stays unchanged and, when a question is asked, receives the passages retrieved from the base. Training a model on your documents is a different project, more costly and harder to update, and it is rarely the starting choice. Begin with passage retrieval and measure quality before considering anything else.
How many documents do we need to start?
There is no valid threshold for everyone. Quality matters more than quantity: a small set of official, up-to-date documents produces better answers than a huge, disorganised archive. Choose a work area with frequent questions and clear documents, and only widen the base once the evaluation is satisfactory.
Does the knowledge base replace the head of a department?
No. The base speeds up finding information, but decisions remain with people. A manager approves the content, decides what is official and corrects wrong answers. Without this role the base degrades over time and loses the trust of those who use it.
Does document data end up with third-party services?
It depends on the architecture chosen. Two questions should be kept separate: where the data is stored, and who can lawfully access it, including under legal obligations. Ask the supplier in writing where the components that process the texts are located, which data leaves your perimeter, and how keys and logs are managed.
How long does the project take?
It varies with the scope of the documents, the quality of the content and the number of systems to be connected. A pilot area with well-organised documents is much quicker than a complete company-wide base. Be wary of fixed timeframes proposed before the document inventory has been done.
How do we stop the AI from inventing an answer?
It cannot be eliminated completely, but it can be reduced. The system should answer only with the retrieved passages, state when it cannot find information and cite its source. An evaluation with real questions shows where the system makes mistakes. Answers concerning obligations or deadlines should always be checked against the original document.
Where should we start if we already have a document management system?
Start from the existing document system, without duplicating it. The goal is to connect search to the documents already classified and the permissions already in use. Verify that the supplier can read the source permissions and respect them, otherwise the base becomes a channel that bypasses existing controls.