"Our company has decades of accumulated documents and know-how; we want an AI knowledge base where employees can ask questions in natural language" — one of the top enterprise requests of 2025.

The reality we keep observing: most enterprise KB projects are abandoned within 6 months of launch.

Reasons filed in the post-mortem are usually "not enough documents", "employees didn't upload", "model performance was weak". These are surface. The real reasons are five deeper problems — any one of which is enough to slowly kill a KB project.

This piece breaks down those five and offers a minimum viable "Phase 1 KB" that avoids them.

1. Failure 1: Permission drift

The deadliest one.

Typical KB architecture:

  1. Batch-import internal documents (Word / PDF / PPT / Excel) into a vector database
  2. Use an embedding model to chunk and vectorize documents
  3. User asks a question → retrieve relevant chunks by similarity → feed to LLM to generate an answer

Problem: this architecture completely bypasses the business systems' permission model.

Example: in OA, Zhang San (East Sales) can't see North Region customer data. But after importing sales data into the vector DB, Zhang San asks "who are the top 10 customers in North Region" — AI retrieves North customer data by similarity and answers. Information leak.

More subtle issues:

  • After departure: OA revokes access immediately, but the vector DB still indexes what they could see the last time they queried. If the ex-employee can reach the AI (via external API), they can still get data
  • After role change: permissions shift, but the vector DB doesn't sync
  • After document retraction: business side deleted it, but old chunks remain

Permission drift is the #1 cause of KB compliance incidents. We reviewed three such incidents in 2024 — all "an employee saw data they shouldn't via the KB".

The correct approach:

  1. Permissions from source: every query re-checks the original system (Feishu wiki / SharePoint / internal doc system) in real time: "can this employee see this now?". 5-minute cache acceptable
  2. Don't mirror content: the vector DB stores only "this document exists + roughly what it says" as index. Fetch fresh content from source at query time
  3. Agent identity pass-through: AI calls business APIs as the current employee, not as a super-account

Do all three and the permission problem is solved. But most open-source RAG defaults don't do any of them — they're academic prototypes, not enterprise implementations.

2. Failure 2: Stale documents

Month 6 post-launch is the riskiest moment.

Reason: enterprise documents keep updating, but KB data is typically a one-time snapshot import.

At launch:

  • Dec 2024 version of employee handbook
  • Dec 2024 version of product manual
  • Dec 2024 version of sales policy

By March 2025, HR updated the handbook, PM updated the manual, sales director revised quarterly policy — none of which sync to the KB.

Six months later, AI is answering from six-month-old versions. Employee asks "what's the latest Product A spec"; AI gives the old version; employee quotes old, deal lost. Trust collapses. Usage stops within 3 months.

Solution:

  1. Sources must be subscribable: connect to systems with listenable APIs (Feishu wiki API / SharePoint webhook / Confluence REST API); document changes push to KB in real time
  2. Incremental updates, not full rebuilds: process only changed docs, not re-import everything weekly
  3. Version correspondence: AI answers cite "from March 2025 version of employee handbook" — employees know which version
  4. Staleness alerts: documents unchanged for N days get flagged "may be stale"

These must be designed in from the start — not bolted on later.

3. Failure 3: Wrong chunking granularity

Chunking = the size of the slices fed to the embedding model.

Two wrong extremes:

Too coarse: one PDF = one vector. "What's Product A's power rating?" retrieves the whole PDF, most of it unrelated. Context window fills with noise. Answer quality drops.

Too fine: every 200 characters. "Give me an overview of Product A" retrieves dozens of scattered fragments — can't piece together a coherent picture. Like holding dozens of puzzle pieces that don't fit.

Correct approach: chunking per document type.

  • FAQ / Q&A docs: each Q&A pair → one vector
  • Product manuals / technical docs: by chapter or subsection (typically 500-1500 words)
  • Contracts / legal: by clause (one clause → one vector)
  • Long reports: by paragraph + hierarchy preservation (retrieves both details and overall structure)

No "one-size-fits-all" rule. A good KB uses different chunking strategies per document type.

Open-source RAG frameworks' default "sliding window every 500 chars" works poorly on structured documents (manuals, contracts). Most "answer doesn't fit question" KB failures trace to this.

4. Failure 4: No audit

Enterprise KBs must answer three audit questions:

  1. Compliance audit: "Who queried which sensitive documents in the past month?"
  2. Quality audit: "An employee got a wrong answer last week. Which document was it retrieved from?"
  3. Cost audit: "Which departments consumed the most AI calls?"

Most "academic prototype RAG" only logs crude data (call time + question). None support these three.

Consequences of no audit:

  • Compliance gets nervous, imposes various restrictions, eventually forces shutdown
  • Errors can't be traced, KB gets labeled "untrustworthy"
  • Runaway cost goes undetected — surprise bill at month-end

What to do:

  • Complete request logging: every call records requester ID, timestamp, question, retrieved document ID list, generated answer, token count
  • Hash-chain integrity: each log entry contains the previous entry's SHA256; any tamper breaks the chain
  • Exportable structured format: CSV / JSON export on filter
  • Retention: 6+ months, 3-5 years for regulated industries

These audit capabilities must exist in the MVP — retrofitting later costs more than rebuilding.

5. Failure 5: No owner

The most underestimated failure — no one is assigned "responsible for ongoing maintenance" after launch.

Typical scene:

  • IT: "Our job is system stability. Content quality is a business-side issue."
  • Business: "Our job is business outcomes. Document maintenance is HR / admin."
  • HR: "We maintain HR documents only, not others."

In that vacuum, content quality degrades. New docs unopened, old docs unupdated, wrong answers uncorrected, user feedback unprocessed.

Solution: appoint a Knowledge Owner at project kickoff.

Responsibilities:

  • Monthly review of usage data (popular questions, low-satisfaction Q&As, un-retrieved queries)
  • Coordinate with business units for ongoing uploads / updates
  • Handle "wrong answer" reports from employees — trace and fix
  • Quarterly alignment with IT on KB operations

At scale, this is a full-time role. Even starting at 0.5 FTE, it must be explicitly designated. A KB without an owner will decay.

6. Phase-1 KB MVP

Avoiding the five traps, here's a recommended Phase 1 (3-4 months):

Phase 0: Scope (1 week)

  • Pick one priority business scenario ("sales looks up product specs" or "CS looks up return policy") — not 5, not 10
  • Define 50-100 representative questions (what employees most often ask in this scenario)
  • Identify which documents hold answers

Phase 1: Data governance (3-4 weeks)

  • Bring in under 5000 core documents (more isn't better)
  • For each: confirm owner, update date, permission tier
  • Unify document metadata (title, author, department, version)
  • Convert key info in images / PDFs to structured text

Phase 2: Architecture build (4-6 weeks)

  • Document source integration (e.g., full-tenant Feishu wiki scan)
  • Differentiated chunking strategy (by doc type)
  • Permission pass-through + live source check
  • Audit log + hash chain
  • Basic ops dashboard

Phase 3: Pilot (4-6 weeks)

  • Pick 10-20 seed users (business leads, key roles)
  • Evaluate against the 50-100 representative questions
  • Trace wrong-answer root causes; add content or adjust strategy
  • Iterate on feedback

Phase 4: General availability (1-2 weeks)

  • Extend to all target departments
  • Establish Knowledge Owner + ops cadence
  • Weekly usage retros for the first month

Total 3-4 months, ¥500k-1M budget (hardware not included). The realistic rhythm for credible enterprise KBs.

Anyone promising a "2-week AI KB" is in the traps.

7. Closing

An enterprise KB is not a "launch and done" system. It's a product requiring ongoing operations.

Five root failures:

  1. Permission drift (architecture error)
  2. Stale documents (no subscription / update mechanism)
  3. Wrong chunking (no per-type strategy)
  4. No audit (can't answer compliance / quality / cost questions)
  5. No owner (no designated Knowledge Owner)

Each must be solved in both architecture and org design. The technology is the easy part — open source and commercial options are mature. The hard parts are data governance + permission design + ops organization.

Our SiNan (enterprise AI agent gateway) knowledge layer was designed around these five — live Feishu ACL recheck, pgvector + tsvector hybrid retrieval, 60-second doc sync, SHA256 hash-chain audit. If you're planning similar internal capability, see the SiNan architecture or book a technical conversation.