Chat analytics that change decisions

Most chat dashboards lead with conversation count and message volume. Both go up when traffic goes up and neither tells you to do anything. They are activity measures presented as performance measures.

Here are four that pass the test: if the number moves, somebody does something differently.

1. Unanswered rate

The share of conversations where the assistant could not answer. This is the direct measure of content gap size.

If it rises, either your content stopped covering what people ask, or your traffic mix changed and a new audience is arriving with different questions. Both are actionable, and the action differs, which is what makes the metric useful. Segment it by landing page to tell them apart.

2. Question categories by distinct askers

Not message counts. Distinct people asking about each topic, ranked. This is the content roadmap, generated rather than debated.

The specific decision it drives: what gets written next. When this list is available, the content meeting is a five minute confirmation instead of an argument about priorities.

3. Handoff rate, split by reason

Total handoff rate on its own is ambiguous, because escalation is sometimes success and sometimes failure. Split by trigger and it becomes readable.

Escalations because the assistant failed twice are a quality problem. Escalations because someone asked about contract terms are a commercial opportunity. If the first category grows, fix content. If the second grows, that is good news and sales should know.

4. Post conversation behaviour

What visitors did after the conversation ended. Did they continue to a pricing page, leave, or come back later. This is the only one of the four that connects conversations to outcomes rather than measuring the conversation in isolation.

It is also the hardest to instrument honestly, and it deserves a caution: correlation here is weak evidence. People who chat are already more engaged, so they were always more likely to continue. Do not present this as a causal conversion lift unless you have actually run a controlled comparison.

What to leave out of the main view

• Total messages. Rises with traffic, decides nothing.
• Average conversation length. Longer can mean engaged or stuck, and averaging destroys the distinction.
• Satisfaction ratings with low response rates. A five percent response rate from self selected raters is not a measurement.
• Deflection rate as a headline. On a marketing site it rewards the wrong outcome.

The review that makes the numbers useful

Numbers tell you where to look. Reading ten actual conversations a month tells you what is happening. Teams that only read the dashboard develop confident theories that the transcripts would have corrected in twenty minutes.

Pick the ten from the categories where the numbers moved. That way the reading is targeted rather than random sampling.