The folly of chasing errors
‘Every month we ask each demand manager to identify their ‘top three’ forecast errors and explain what they are doing to prevent their recurrence’ .
This is the kind of practice that we frequently encounter as we talk to companies about how they go about improving the performance of their forecast process…and who could possibly object to such a sensible approach?
In fact, at best this approach is a waste of time; at worst it generates a defensive mentality among forecasting practitioners who – unsurprisingly – will learn to focus their attention on trying to avoid being blamed rather than improving the process.
Why?
Reason Number 1
There will always be 'a top three’ forecast errors, no matter how good or bad the process. The top three could represent three forecasts that are at the extreme limit of what can possibly be achieved and incapable of being improved – so all that ‘investigation will uncover is the existence of unforecastable noise…which will be dressed up as a reason for ‘failure’.
More likely, the top three will be the tip of an ugly iceberg produced by a misfiring forecast process. This is the root cause of the problem, and this should be the focus of attention not the superficial symptoms.
Reason Number 2
What kind of forecasts will appear in the ‘top three’?
The usual suspects will be errors emanating from forecasts of the largest and most volatile products; almost irrespective of the real quality of the forecasts.
The real villains – which may never appear on these ‘most wanted’ lists - may be forecasts of stable products, or lower volumes lines, where poor forecasting will never lead to large absolute errors.
Reason Number 3
The killer reason is perhaps the least obvious one.
Forecasts – or any other kind of process – can only be improved by identifying systematic patterns of error…errors that recur.
This fact was first articulated by Walter Shewhart in 1926 and it has been one of the guiding principles of the quality movement ever since…as seen in the work of Juran, Deming and the rest and most famously the manufacturing practices of the Toyota Motor Corporation.
There will always be individual spikes in process defects due to ‘one-off events’ – Deming called these events ‘special cause variation.’ But, because they may never recur, they provide us with no actionable information that can be used to improve the quality of our process. Instead, we should focus on the routine level of error, or ‘common cause variation,’ as it is only systematic problems that we can only fix.
For forecasters, the lesson is that we need to track errors over time, using a method that helps us identify patterns of error that are indicative of a systematic problem that needs to be fixed so that we are not distracted by random events and noise. And this needs to be done at the most granular level, because it is the quality of low level forecasts that we need to improve, not just our high level performance measures.
Post a comment