Operational Success Index: Where to Measure Forecast Accuracy

In part one of this article series we established how each team within the workforce management practice affects the outcomes and success metrics of the call center. Last week in part two, we looked at measuring those.

Another common inquiry during our consulting engagements comes from industry best practices for measuring overall success of the call center and workforce management as a whole. From that perspective, we have created another performance index that takes into account several different KPI to determine the overall success of delivering on the call centers caller experience promise. Prior to discussing the new Operational Success Index, we must first review how some metrics are viewed currently.

The majority of the call industry measures their forecast accuracy at the day level. This does not provide the most accurate view of how your operations performed by interval or a true caller experience. Brad Cleveland’s Call Center Management on Fast Forward (i) suggests that measuring at the interval level is a more appropriate approach.  Cleveland suggests in figure 3 that measuring at the day level provides a large delta to the actual experience by interval that callers experience when attempting to contact the call center. Based on that position we have created a new key performance indicator to measure at the interval level, the Interval Average Accuracy (IAA). We calculated this as the average unweighted variance of the day’s intervals.

Figure 3 (ii)

In the above example, you can see that measuring forecast accuracy by the day gives us what appears to be a very good day as the forecast variance is only 1%. However, if we look at the interval level as Cleveland suggests there is a significantly different outcome. For this reason, we suggest using the IAA KPI. This shows that the actual forecast accuracy is at a variance of 9.2%, a far different indication of success than the 1% forecast variance. This gives a truer representation of the caller experience throughout the day, and allows for some root cause analysis around which intervals had the greatest variance.

As part of the root cause analysis, the business needs to review all variables that lead into a forecast to understand what aspect created the variance. For this, we have developed an Operational Success Index that not only provides insight into the forecast variables, but can also be used as lead metric to evaluate the overall success of the operations.

Creating an Index

In order to create an Operational Success Index we must take a look at all the metrics that are important to measuring success in your call center so you can start to provide a root cause analysis of where the issue may lie. These include:

• Shrinkage

o Illness
o Vacation
o Planned
o Unplanned

• Customer Experience Metrics

o Service Level
o Average Handle Time
o Abandonment Rate
o Average Speed of Answer

• Scheduling

o Schedule Adherence
o Schedule Conformance
o Schedule Efficiency

Historically, call centers use these metrics to determine if they were successful. They are also analyzed individually. The approach we are suggesting is to take a view of all these metrics as a whole and weight them according to the strategic importance to your call center.

Depending on the strategic vision that your organization subscribes to, you may find that you want to have a greater emphasis on schedule adherence and conformance as the driver for the overall success of your organization. Through this approach, you would weight those metrics with a greater importance in the overall success index. In figure 4, we have created some sample data to show how creating an index will provide an operational score.

Figure 4

The success indices are weighted based on an example set of strategic importance. The important thing to remember is that they must equal one hundred percent in the weighting.

An Operational Success score gives an indication as to how the call center performed on the previous day. After a historical evaluation, each call center is able to establish an Operational Success range used for targeting purposes. Once the target has been established then call center leadership and the Workface Management team have a common metric that can be utilized to measure the overall success of the call center. If the target range is not achieved then it is an easy exercise to conduct a root cause analysis to determine what aspect(s) of the operation or day’s events caused the target to be missed. Pinpointing which KPI(s) caused the overall Operational Success score to be missed provides a lead indicator for adjusting the strategy for subsequent days or where reforecasting is required.

Matching the results from analyzing the Operational Success metric as well as the degree of influence from figure 1 will give call center leadership greater control on which levers need to be pulled to affect change within the call center.

Conclusion

Understanding how the support teams to call center operations such as workforce management, play in the overall success of the call center is an important step in optimizing your operation. Determining scopes of influence provides a meaningful way to determine purview as well as developing a consequential balanced scorecard for the workforce management team to measure their effectiveness.

By adding an Operational Success matrix to this approach, your call center is better equipped to perform root cause analysis on the day-to-day operations of your call center. This approach to measuring accuracy gives your call center the edge in developing tools that will effectively increase your overall performance.

i: Brad Cleveland, Call Center Management on Fast Forward; succeeding in the new era of customer relationships. Third Edition, 2012 (Colorado Springs, CO, USA ICMI, a registered trademark of UBM) 1753, 1754
ii: Cleveland, 1753. Example Data and approach. Interval Average Accuracy is a new suggested KPI that aligns with this approach.