Customer Operations Agent — reference architecture

Customer Operations Agent — reference architecture

A production-grade customer service agent is not a chatbot. It is a system with identity, scope, policy, audit trail, and a designed escalation path. This is what that architecture looks like end to end.

The problem with most deployments

Most customer service AI deployments fail for the same reason: they treat the agent as a chatbot, not as a system. The agent has no identity (so it cannot be audited), no scope (so it acts outside its authority), no policy (so it makes inconsistent decisions), no audit trail (so failures cannot be investigated), and no escalation path (so the customer is stuck when the agent does not know). The result is a system that answers some questions correctly, fails the rest silently, and creates trust and compliance issues when it fails.

The architecture

The agent operates inside the existing customer service stack. It has an identity provisioned through the corporate IDP, with the scope and permissions of a customer service role. It reads from the CRM, the ticketing system, the knowledge base, and the customer history. It responds or acts within the policy and the customer’s permissions. When the case is non-standard, it escalates to a human with full context. Every action is logged.

The agent is not a single model call. It is a system that:

• Reads from multiple sources with provenance
• Classifies intent and priority
• Retrieves the relevant knowledge with citations
• Responds or acts within the defined policy
• Escalates with full context when the case is outside its scope
• Learns from corrections and escalations

The agent responsibilities

The agent is responsible for:

• Reading customer history and open tickets
• Classifying intent and priority
• Responding to standard queries using the knowledge base
• Taking routine actions within the policy
• Escalating non-standard cases with full context
• Maintaining the knowledge base based on resolved cases

The agent is not responsible for:

• High-impact actions (refunds, account changes, subscription cancellation)
• Sensitive situations (complaints, fraud, bereavement)
• Anything outside the defined policy
• Anything the customer should talk to a human about

The flow

1. Customer contacts via chat, email, voice or web form.
2. Agent reads customer history from the CRM.
3. Agent reads open tickets from the ticketing system.
4. Agent reads relevant knowledge base articles.
5. Agent classifies by intent and priority.
6. Agent responds or acts within policy and permissions.
7. For non-standard cases, agent escalates with full context to a human.
8. Human agent takes over with the agent’s reading and suggested next steps.

The tools

The agent uses MCP to access:

• The CRM (read customer history, update case)
• The ticketing system (read tickets, create tickets, update tickets)
• The knowledge base (read articles, cite sources)
• The IDP (read customer identity for authentication)
• The communication platform (send response)

The models

The agent uses:

• A frontier LLM for reasoning and generation
• An embedding model for knowledge base retrieval
• A classification model for intent and sentiment
• Smaller models for routine operations (status lookups, etc.)

The verification

The agent is verified at three levels:

• Pre-deployment: scenario-based eval suite covering 100+ test cases
• In production: sampled evals on real interactions
• Continuous: customer satisfaction tracking per interaction, escalation rate and accuracy, resolution rate and accuracy

The human checkpoints

The agent does not bypass the human. The human checkpoints are:

• All high-impact actions require human approval
• All escalations are designed in, not exceptional
• Sensitive situations route to humans by policy

The auditability

Every action is logged with:

• The input (what the agent read)
• The policy (what rule it applied)
• The decision (what it chose to do)
• The output (what it said or wrote)
• The customer context (which customer, which case)

The deployment options

The agent can be deployed as:

• Multi-tenant cloud for standard deployments
• Private cloud or VPC for regulated workloads
• Hybrid: agent in customer cloud, knowledge base in ours

The failure modes

The agent can fail in known ways:

• Knowledge base out of date: agent gives wrong answer
• CRM integration down: agent cannot read customer history
• Policy not defined: agent acts outside scope
• Escalation path not configured: agent cannot reach human

The failure modes are designed for. The agent degrades to “I do not know, escalating to a human” rather than guessing.

What this looks like in production

The customer contacts the service center. The agent reads the customer’s history, the open tickets, the relevant knowledge. The agent responds. The customer does not know the agent was an agent. The interaction is fast, consistent, and accurate. The audit trail is available for every interaction. The human agent focuses on the non-standard cases that require judgement.