Enterprise Knowledge System — reference architecture

Enterprise Knowledge System — reference architecture

The enterprise knowledge system is the substrate for every other agent in the enterprise. This is what a production-grade knowledge architecture looks like.

The problem with most knowledge systems

Most enterprise knowledge is scattered across wikis, shared drives, ticketing systems, email, and the heads of long-tenured employees. The people who need it cannot find it; the agents that could act on it cannot access it. The result is duplicated work, inconsistent answers, and knowledge that walks out the door when employees leave.

The opportunity is to turn the scattered knowledge into context that both humans and agents can act on, with provenance and access control. The system is not a replacement for the source systems; it is a layer on top that makes the source systems usable for retrieval.

The architecture

The system ingests from the existing knowledge sources: wikis, document management, ticketing, email, internal portals. The ingestion is governed by access control: each document retains its source permissions. The retrieval layer provides context with provenance to both human consumers (search, recommendations) and agent consumers (RAG).

The system is the substrate for every other agent in the enterprise. The customer service agent reads from it for knowledge. The coding agent reads from it for code patterns. The operations agent reads from it for procedures. Without a good knowledge system, every agent has to build its own.

The agent responsibilities

The system is responsible for:

• Ingesting new content as it is created or updated
• Maintaining embedding quality
• Serving relevant context to other agents
• Providing provenance for every retrieval

The system is not responsible for:

• Knowledge that is not in the source systems
• Knowledge that is in the source systems but not ingested
• Knowledge that is ingested but not retrievable
• Permission decisions (those are governed by the source systems)

The flow

1. Ingestion from each source system with access control preserved.
2. Embedding generation with source attribution.
3. Vector storage with metadata for filtering and provenance.
4. Retrieval layer serving both human and agent consumers.
5. Re-ranking and filtering based on user permissions and context.
6. Citation generation for every retrieval.
7. Feedback loop for quality improvement.

The tools

The system uses:

• Ingestion connectors for each source system
• Embedding pipeline
• Vector database
• Retrieval API
• Permission system integration

The models

The system uses:

• An embedding model for semantic search
• A reranker for relevance
• An extraction model for structured data from documents

The verification

The system is verified at three levels:

• Retrieval accuracy (recall, precision, MRR) against a curated test set
• Permission enforcement verified per document with positive and negative cases
• Provenance completeness verified per retrieval
• Freshness SLA verified per source

The human checkpoints

The system does not replace human ownership of knowledge:

• Knowledge base curation is human-owned
• Source permissions are human-governed
• Feedback loop includes human review

The auditability

The system is fully auditable:

• Every retrieval traces to source document and timestamp
• Every ingestion event is logged with source and timestamp
• Every permission check is logged

The deployment options

The system can be deployed as:

• Multi-tenant cloud for standard deployments
• Private cloud or VPC for regulated workloads
• Hybrid: ingestion in customer environment, retrieval in ours

The failure modes

The system can fail in known ways:

• Source system down: stale data, ingestion paused
• Permission misconfiguration: over-revelation or under-revelation
• Embedding quality degrades: retrieval becomes less relevant
• Source content changes but embedding is stale

The failure modes are designed for. The system degrades to “I do not have current information” rather than returning stale data.

What this looks like in production

A new employee asks a question in the internal portal. The system returns a relevant answer with citations. A customer service agent asks a question about a policy. The system returns the relevant policy with the source link. A coding agent looks for a pattern. The system returns the relevant code with the file and commit. Every retrieval has a citation. Every citation has a permission. Every permission is enforced.