← All resources

AI and execution

AI in customer support: what to automate, what to review

An AI summary can read beautifully and still send a team in the wrong direction. A useful sentence such as “Engineering confirmed the fix” becomes a problem if the source actually says, “We think this might fix it.”

The decision about AI in customer support should go beyond whether it can produce a convincing answer. Ask what information it used, what it is being allowed to do and how the team will know whether the result is right.

Separate finding context, proposing work and taking action

These jobs have different consequences. Treating them as one “AI automation” makes it harder to choose sensible controls.

Different jobs need different checks
Job Useful output What to check
Assemble context A concise account of the issue and the relevant evidence. Sources, recency, omissions and whether the reader has permission to see them.
Propose work A suggested investigation, owner or response. Whether the proposal follows from the evidence, fits the team's responsibilities and states uncertainty.
Take action A change in an authorized system or a message sent to someone. Authority, destination, scope, failure handling and proof of what actually happened.

A team might allow an internal summary to appear automatically while requiring approval before sending a customer-facing message. That is a workflow design choice. The right boundary depends on the task, the information involved and the consequences of a mistake.

NIST's Generative AI Profile describes the risk of confidently incorrect generated content and recommends checking sources and citations during evaluation. A readable answer is not evidence that its claims are supported.

Use the consequence of a mistake to choose where people review

Start with a concrete workflow. “Summarize the latest customer conversation” is narrow enough to examine. “Handle escalations autonomously” bundles together interpretation, access, prioritization, ownership and external action.

For each step, ask: if this is wrong, who is affected, how quickly would we notice and can we undo it? These questions help distinguish a draft that someone can correct from a consequential action that needs tighter authorization.

The boundary belongs at the claim as well as the button. A human clicking Send does not help much if the interface hides the uncertainty that person needed to evaluate.

Human review needs something to review

A confirmation dialog that says “Are you sure?” gives someone responsibility without giving them much help. Put the evidence next to the proposed decision.

  • Show where the important claim came from. A reviewer should be able to inspect the source, not just another generated explanation.
  • Keep unknowns visible. “No owner confirmed” is useful. A confidently invented owner creates work for the wrong person.
  • State the proposed effect. Identify the record, message, destination or status that will change. Distinguish preparing a draft from sending it.
  • Allow correction or rejection. Approval should not be the only easy path through the interface.
  • Show the returned result. A proposal, an approved request and a completed action are different states. A failed or uncertain result needs a recovery path.

Keep the source permissions intact as information moves. A summary should not become a shortcut for exposing a restricted customer record to a wider audience. Start an evaluation with fictional or non-confidential material until the relevant access and data-handling arrangements are in place.

Before using an AI tool, establish what information may be sent, which providers process it and what retention settings apply. An approval button on the result does not mean processing waits for approval.

Measure whether AI removed work, including the corrections

A quick generated draft can look efficient while pushing effort into review. Count the checking and rework as part of the workflow.

For a small, permitted evaluation, use representative examples with an expected result. Include routine cases, incomplete context, conflicting information and cases where the system should ask for clarification. Keep some examples separate from those used to tune the workflow.

Review the outcomes that matter to the task:

Supported context
Were the important claims backed by the available sources? What consequential information was omitted or misstated?
Useful proposals
Could the reviewer use the suggested next step, or did they have to reconstruct the issue themselves?
Reliable execution
Did the approved, supported action reach the intended destination? Were failures and uncertain results visible?
Total effort
How much work remained after checking, correcting and following up? Compare that with the team's existing process.

Do not treat a handful of polished demos as an accuracy measurement. Keep track of the failure modes, define what is acceptable for the particular task and check again when the sources or workflow change.

The strongest use of AI is often unglamorous to describe and satisfying to use: the right evidence is already there, the next step is sensible, and the person remains in control of the decision that matters.