Staffing a contact center is expensive. To make the most of your customer service team, you need to make sure they are running efficiently and constantly improving. That is when performance management can come in handy to help you measure an agent’s performance at every level: review chat transcripts, A/B test canned responses, chat shadowing, in addition to various metrics and analytics reports.
But what if you don’t have any experience with this kind of data collection, analysis, and change implementation? How can you be confident about what changes need to be made? How should you know whether your changes have been effective, or how long to wait before making that determination?
In this blog, we’ll discuss how to implement data collection and analysis within the context of an active 5 step process:
1. What to ask (metrics)
2. How to answer (metrics)
3. Getting “Before” right (metrics + data collection)
4. How to react to “After” (data collection + action)
5. Your next “After” (data collection + action, perhaps metrics also)
Step 1: What to Ask
The first step to getting started with performance management is deciding what question you want the data to help you answer. You’ll want to pick something small, simple, and specific. This is because you want there to be as immediate and unambiguous a connection between your actions and the data as possible.
“How can I improve customer satisfaction scores?” though specific, is too big and complicated a question, as multiple factors impact it. “How can I get more customers to complete satisfaction surveys?” or “How can changing routing for billing questions help to reduce wait times and/or transfers?” are better questions, and incidentally can actually help pave the way towards the bigger question you’re driving towards.
Step 2: How to Answer
Once you’ve picked your small, simple, and specific question, you need to figure out how you’re going to measure the extent to which your changes are having an impact. You probably have a strong sense of how you’ll measure progress just from the question you’ve picked, as it’s usually intuitive to figure out how to measure something simple and specific. However, sometimes you might want to go one level deeper.
If you’re looking to have more customers complete surveys, tracking the rate of survey completion is an obvious metric you can use. But, you may want to take it one level deeper and measure whether certain types of site visitors are more or less likely to complete surveys, and target your solution accordingly. If you don’t have a strong sense of who that audience might be, it might be worth looking at visitor monitoring data to identify where the most effective leverage point might be. That said, general survey completions is probably a helpful enough question to guide your performance management efforts, without taking it one level deeper.
Performance management is driven by your vision for how you might improve performance for your team, guided by data collection and experimentation in possible solutions. Going one level deeper with your metrics depends on that vision of yours, and on your comfort level with the data required to determine what’s working and what’s not. So, consider going one level deeper, but by no means feel obligated to if it’s not the right fit.
As a second example- if you’re looking to use routing to reduce wait times and/or transfers, you’ll obviously want to track average wait time and number of transfers per day (or per week, depending on volume), but you may also want to try to identify what causes longer wait times or more frequent transfers.
Perhaps there are certain times of day when staffing is inadequate, but you want to solve that problem with smarter routing rather than with adding more staff to the schedule. In this instance, you’d want to track wait times during this time period apart from wait times during other times of day. When it comes to transfers, perhaps there are certain topics that tier one chat agents are not equipped to handle and so often result in transfers. Rather than resolve that by training all of tier one to handle those specific topics, you want to make it easier to get those conversations to the appropriate agent via routing rather than through a needless transfer. In this case, you might consider tracking transfers and routing for that topic apart from other types of transfers.
The purpose here is to establish the parameters of your problem and how you plan to go about solving it- that tells you what to measure and how to measure it.
Step 3: Getting “Before” right (And Making Mistakes)
Gathering baseline data is the “Before” picture that you’ll compare to the “After” of your implemented solution. This comparison will help you to determine to what extent your solution achieved the desired result (e.g. reducing wait times and transfers). Because of this, it’s important that you understand the scope of your question (what to ask, e.g. can we reduce wait times/transfers?) and your solution (how to answer, e.g. adjust routing) prior to gathering baseline data (the “Before” picture).
If you didn’t establish the scope of your question and your solution before collecting your baseline data, you might not have thought to track transfers for specific topics or transfers during a specific time of day, vs. transfers as a general metric. This kind of oversight can result in less actionable data, as there may not be an obvious impact looking at overall trend data.
But what if the parameters you established as part of your solution’s scope require information that you don’t presently have? For instance, perhaps you don’t currently track the reason for a transfer. This could be useful information if the goal is to reduce the number of transfers. You have two options. First, you could begin gathering that information now, and wait before implementing your proposed change until you’re comfortable with the baseline you’ve set up. Alternatively, you can start collecting data at the same time that you implement your changes (“build the plane as you’re flying it”), and simply use the performance reporting (the “After”) from your first change as your baseline (“Before”) data to precede a second change.
If you’re just getting started with performance management, it’s probably going to be most productive for you to take the first option and start by simply collecting data and getting comfortable with doing that in a new way. Once that data collection has become routine, then move on to the task of managing the new change process that will drive the continuous improvement of your wait times/transfers.
One final note about baseline data: in a pinch, you wouldn’t be the first organization to use industry benchmark data as your rough baseline, prior to actual data collection. If you’re on a tight timetable, you can probably get away with using industry standards as your baseline while you work to generate your own, organization-specific data.
About making mistakes
It’s entirely possible that your first attempt to gather baseline data will miss an important detail or extenuating circumstance.
Maybe many visitors get transferred because they selected the wrong department in a pre-chat survey. If you pay disproportionate attention to how your performance compares to a metric (e.g., number of transfers) rather than paying attention to the quality of metric itself (e.g., are you tracking reason for transfer?), you might continue to overlook an inaccuracy in your data that could easily be remedied (e.g., filtering out quick and harmless transfers resulting from user-error).
It’s much easier to correct this in the benchmark phase rather than retroactively during the performance management phase, because a change in the benchmark phase can cause a change in process that automatically fixes the problem going forward. A retroactive change during the performance management phase might require significant manual effort to correct reporting, or delay reporting until “cleaner” data has been collected for comparison.
Step 4: How to React to “After”
There are really only three types of changes you should be making in response to the data you’ve collected: change practice/process, change data collection, or change priorities.
Changing practice is the most straightforward result of data collection. You had a theory: you could reduce transfers and wait times through better routing. The data supported it: after tracking daily transfers for 2+ weeks, you believe that customers with billing questions should automatically be directed to a specialized customer service rep rather than to general support. Now you just need to move forward and implement the changes necessary (in this case, update your chat/phone/etc software to accommodate this improvement to routing).
You’re ready for Step Five, but don’t stop collecting data yet! You’ll need that data to demonstrate the significant impact your proposed change has had, not only to your superiors, but to your staff who’ve had to accommodate your data collection needs, and potentially to your peers who’ll want to learn from the best practice your process has helped to establish.
When you realize that you don’t feel confident enough about the data you’ve collected to warrant changing practice, but still feel strongly that your hypothesis is correct, it’s time to change the way you collect that data. Perhaps you didn’t track the reason for the transfer when you first started collecting data. As a result, the data seems to indicate that, on average, transfers aren’t impacting service- but that doesn’t align with your real-world experience. You try filtering out quick, user error-based transfers. This change in how you collect the data significantly impacts the average, and now you have the quantitative basis for making a confident improvement to your team’s practices/processes. See “changing practice” above.
Changing priorities happens when, after collecting data, you discover that your hunch was incorrect. While transfers are a painful experience on a daily basis, it is a trivial issue for a small percentage of your customer/visitor population—and as such your time and resources are better allocated towards solving other, bigger problems. This doesn’t mean you did something wrong; this result might mean your improvement efforts in a particular focus area have plateaued. For instance, maybe your routing is fully optimized; there are no other changes you could be making that will result in greater efficiency (without tradeoffs that simply aren’t worth the trouble)- great job getting to that point! You’re ready for Step Five.
Step 5: Your Next “After”
A defining (and also, admittedly, exhausting) component of performance management is that it is continuous; it represents a cultural shift for an organization. That shift being: from a generic, repetitive “incremental raising of the bar” for your team’s or organization’s overall goals, to identifying specific, measurable ways you and your staff can improve your work on a daily basis to meaningfully impact those overarching, strategic goals.
If you’ve made it to this point, you’ve already tested a theory and had it proven right (or demonstrated that your hypothesis doesn’t have the ROI it needs to in order to be worth the trouble of implementing). In either case, you’re ready to test your next theory about how your team can improve. Keeping with our example, you’ve set up smarter routing that now directs customers with billing questions directly to the person or team best-equipped to handle them, rather than clogging tier one and frustrating customers whom tier one is not equipped to help—now what? If the transfer rate is still a problem, try to identify why that might be. Zero in on that “reason for transfer” in your data collection.
If the transfer rate is looking good, find another high-impact opportunity. Maybe have your post-chat survey only ask one question (e.g. “how would you rate our customer service?”), but based on the response to that question, trigger a longer feedback collection process (this could work for getting more information about how to calm an angry customer, or could provide great testimonial content for your marketing team).
Lingering Data Collection Questions
1. How much data do I need to correctly reach a conclusion?
The continuous improvement process is less about scientific rigor and more about making sure your team is paying attention to the right things and working consistently to improve them.
There may be seasonal differences or anomalous spikes or lulls in activity that could impact how long you should be collecting data before reaching a conclusion. For non-statisticians, a good general rule is to gather twice as much data as you plan to compare (e.g., if you want to see how things improved, or didn’t, between weeks two and three, look at data for weeks one through four). This is not proof of impact, but it is evidence that can guide the next few steps in the iterative process of performance management.
2. What are reasonable expectations for what the “Before”/ “After” impact will look like?
This will vary immensely by the situation. Although unlikely, it’s certainly possible that you may have chosen to change something that won’t have any impact at all on the problem you’re trying to solve. It’s best to set the expectations for yourself and for others that you are committed to finding the solution, but that it may take a few tries to figure out what precisely that solution may look like, and then to optimize it.
3. Is the extra effort and infrastructure of data collection worth what it helps us to accomplish?
Yes and no. Yes, in the sense that it helps create a culture of continuous improvement within your organization. Also, yes in the sense that you aren’t going to invest the time and effort into solving a problem that won’t have a tangible benefit. However, it is possible that you may spend so much time trying to crack the code on a business problem that the pain of your problem-solving process is worse than the pain of the initial problem itself. Be sure that the problem you are trying to solve is worth the time and trouble an iterative approach might ultimately represent.
4. How do I get buy-in from staff so that there aren’t data quality issues skewing my results?
This is the question of an experienced change-maker! You need to demonstrate that better data quality helps them directly, through making them better at their jobs, or ultimately making their jobs easier, or both. In our example, logging the reason for transfer ultimately resulting in having to do fewer transfers is an example of data quality making their jobs easier. You might also want to have a look at this article on ICMI, specifically the section on secondary impact. These kinds of conversations can help inform what direction to move in next with your continuous improvement efforts.
