AI Knowledge Base: Build One From Your Own Documents

A colleague asks you for the refund rules. Again. You know they are written down somewhere, in a shared folder, an old email or a document nobody has opened since spring. So you search for ten minutes, or you just answer from memory and hope it is still right.

The short answer: an AI knowledge base is a set of your own documents that an AI assistant answers from, instead of from the open internet. You do not need to build software for this. Put your documents into a Claude project or a Gemini Gem, tell the assistant to answer only from them, and test it with ten real questions. That is the whole method, and the steps are below.

What an AI knowledge base actually is

Strip away the jargon and it is three things: your documents, an assistant that reads them, and a rule that says “stay inside these documents”. The rule matters most. Without it, the assistant fills gaps with a confident guess. With it, you get an answer from your own text, or an honest “not found”.

This is different from asking a chatbot a general question. If you want to see how one document gets turned into a clear summary first, our guide on summarizing long documents with ChatGPT covers that step.

Step 1: Pick ONE area, not the whole company

The mistake that kills most attempts is starting with everything. Pick one area where the same questions come back every week. Good first choices:

• Onboarding questions from new colleagues
• Pricing, refunds and standard replies for customers
• How you run a recurring process, such as a client kickoff

Ten to fifteen documents is plenty for a first version. If you do not have written procedures yet, start there: how to use AI to write SOPs walks through it, and AI for employee onboarding shows what a good onboarding set looks like.

Step 2: Clean the documents before you upload them

The assistant is only as good as what you give it. Before uploading, do four quick things:

1. Delete or fix anything outdated. Two versions of the refund policy will produce two different answers.
2. Give each file a clear name, like “Refund policy, updated March” instead of “final_v3”.
3. Remove what should not be there. Salaries, passwords, personal data and confidential client details do not belong in an AI tool unless your company policy says so. If you are unsure, ask first.
4. Prefer text over screenshots. A scanned page or a photo of a whiteboard is hard for the assistant to read.

You can ask the AI to help with the cleanup too. Paste a document and use this:

Read the document below. Write a one-paragraph summary, then list the 5 questions a new colleague is most likely to ask about it, each with a short answer taken only from the text. Mark anything that looks outdated or unclear.

[paste document]

The “mark anything outdated” line is the useful part. It often points at the contradictions you stopped seeing.

Step 3: Choose where the knowledge base lives

You have two easy options that cost nothing to try.

Claude Projects

Anthropic describes projects as self-contained workspaces with their own chat histories and knowledge bases. You upload documents, add instructions, and every chat inside the project can use them. According to that help page, projects are available to all users, including free accounts, though free accounts have a limit on how many you can create. For a first test that is enough. I would start here if your documents are long, such as handbooks or contracts. Our guide to using Claude for work shows the rest of the setup.

Gemini Gems

Google calls its version Gems: custom versions of Gemini that you set up with your own instructions and files. If your team already lives in Google Docs and Drive, this keeps everything in one place. Which features you get depends on your plan, so check Google’s page for your account. More on this in how to use Gemini for work.

ChatGPT has a similar workspace feature called Projects. Availability and file limits depend on your plan, so look at your own account before you count on it.

Step 4: Write the instruction that keeps it honest

This is the part that turns a chatbot into a knowledge base. Paste this into the project instructions (or the Gem instructions) and change the first line to fit your team:

You are the knowledge base assistant for my team.
Answer only from the documents in this project.
If the answer is not in the documents, say "I can't find this in our documents" and name the closest document.
End every answer with the document name it came from.

Naming the source in every answer is not decoration. It lets the person asking open the document and check, which is how you build trust in the tool.

Step 5: Test it with ten real questions

Do not announce anything yet. Write down ten questions your team really asks, including two or three where the answer is NOT in the documents. Then ask them one by one and score each answer:

• Right and sourced: good.
• Right but no source: tighten the instruction.
• Wrong or invented: find out why. Usually two documents disagree, or the answer is buried in a scanned file.
• Not in the documents, and it says so: that is the behavior you want.

If it fails the “not in the documents” questions, that is your signal it is guessing. Make the instruction stricter, fix the files, and test again. Only when it passes most of the ten should a colleague see it. Even then, treat answers on anything legal, financial or safety related as a pointer to the document, not a final decision.

Keep it alive: a ten minute habit

A knowledge base goes stale the same way a shared folder does. Pick a fixed day each month, replace the documents that changed, and add one new question you saw come up. Also keep a short list of questions the assistant could not answer. That list tells you which document is missing, and writing it is usually worth more than adding more AI.

When this is not enough

A project or Gem is a good fit for a small team and a limited set of documents. If you need it to answer customers on your website, or to pull from hundreds of live documents, that is a different job. For the customer side, read AI chatbot for customer support first, and be clear about what the bot may and may not promise.

Start small this week

Pick one area, gather ten clean documents, and run the ten-question test. If it works, you save yourself the next “where is that written down?” message. If it does not, you have learned which documents to fix, which is useful either way. If you want a free starting point for using AI like this in your daily work, join Launchpad below.

If you are choosing which assistant to build on, our Claude vs Gemini 2026 comparison shows how to test both on your real work.