How to Validate Your Forecasting Model

Call-volume and handle-time forecasts are accurate, but service level is still not being met. Why? There may be an issue with the forecasting model. Let’s take a look at a few different ways to validate the model and provide a clearer picture of what is really occurring in the contact center.

It can be difficult to achieve an accurate forecast on a skill with low volume (example, less than five calls per interval), so it’s best to use skills with higher daily volume for model validation as those with minimal volume don’t lend themselves to effective analysis.

1.    “Back-casting” for validation: Forecasting over a history that has already passed will provide useful information on the health of the forecast. Not all WFM applications will allow for back-casting, so if it’s not a possibility, pull a performance, forecasting accuracy or IDP report for the past period being investigated to begin the troubleshooting process.

When back-casting, or viewing past performance, it’s important to look at the results for each of the following metrics. When analyzing a past forecast, either via back-casting or performance reporting, what is really being analyzed is forecast to actual – how the plan performed compared to what really happened. Analyze:

a.    Call volume – How accurate was the volume forecast?

b.    Handle time – How accurate was the AHT forecast?

c.    Service level – Was the service time accurately predicted? What about the percent answered?

If the volume and handle time forecasts are close to being accurate, but service level is not, it’s time to take the next step and analyze what went on in the contact center for that period.

2.    Data entry: Did all events make it into the WFM application? Before back-casting, make sure all known events are entered in the WFM application. This will help analyze known shrinkage, and rule that out as the culprit. It will also provide ballpark data on any unproductive shrinkage that occurred during that period.

Once you have a good shrinkage number, compare that to what was used in the original forecast. Was the shrinkage used in the original forecast much lower than reality? If so, it’s time to investigate.

3.    Investigate: Check schedule adherence. Were agents where they were supposed to be? If schedule adherence was low, then it’s time to analyze what they were really doing. If all known events are in and adherence is still low, then there’s a high probability of behavioral issues that need to be addressed and agents may not be the only contributors.

Now is a good time to pull those ACD reports for Not Ready/AUX/Unavailable time. Some analysis here will help determine what is causing the discrepancy. Are agents practicing creative call avoidance or are supervisors taking them off the phone for legitimate reasons without notifying the WFM team?

Agents or supervisors going “rogue” can really impact the accuracy of the service level forecast. If this is the case, it’s important to put standardized processes in place so everyone is on the same page, especially if the WFM team is responsible for service level.

Shrinkage is frequently the number one culprit for over-forecasted service levels, meaning the service level was lower than predicted. As such, it’s important to understand the drivers contributing to shrinkage.

Model validation should be done any time there’s a large discrepancy in forecast to actual data, but even if forecasts are accurate, it’s important to revisit that model on a regular basis to make certain all business drivers and unplanned events are considered.

Regularly validating the forecasting model will ensure the staffing plan is on target so business goals are successfully achieved.