It does not measure the person, it measures their conversations
Whoever handles billing complaints will score worse than whoever resolves configuration questions, and that says nothing about how well they work.
Comparing people without segmenting by question type turns the metric into an injustice with numbers attached.
Why it matters
Used well, it points at where training or documentation is missing.
If three people score low on the same topic, the problem belongs to the topic. Used badly, for rankings or bonuses, it pushes people to close the easy things fast and avoid the hard ones, the exact opposite of what is needed.
How to read it without getting it wrong
Five rules prevent almost every misuse.
- Compare only among people who handle the same type of query.
- Require a minimum number of answers before drawing conclusions.
- Read the comments, not just the score.
- Look at it next to first contact resolution and handle time.
- Use it to decide on training and documentation, not to rank people.
Segment before comparing
By question type and by channel.
Within one segment, differences between people do say something; across segments, they do not.
The minimum volume
With twenty responses, a single negative rating moves the percentage several points.
Below a certain sample the number is noise, and presenting it as if it were not produces absurd conversations.
What else to look at
The comments.
An 80% CSAT with three comments saying the same thing contributes more than the percentage, and it is the only part you can act on.
