Forecasting the Economy
Statistics is difficult, fraught with traps and errors and misconceptions that can lead you down very wrong paths and to very wrong conlcusions.
Downstream of this sort of general difficulty and pain is the use of statistical techniques to make inferences about the future, AKA to forecast.
Forecasting is hard.
Which ones?
We can only have confidence in forecasts that we can explain.
To make a confident prediction, we have a good reason, a good theory, a good explanation of why we think the variable is predictable.
It is impossible to induct your way to this confidence. Remember? If a metaphorical black box spits out the word “forecast” 10^100 times in a row, we can’t say, logically, what the next word will be, or even what that word it is likely to be, without an explanation of why that is so.
Without a good explanation of why the variable being forecasted can be predicted, we can say precisely 0 about how accurate the forecast is likely to be. The meta forecast, the forecast about the forecast, will have the certainty equivalent of a shrug.
Well, definitely not economic ones
One area that generally lacks good explanations of forecast accuracy is economics.
In fact, we have good ideas of why forecasts won’t work.
Central to this is the fact that humans don’t behave in uniform and consistent ways in economic domains, making them very difficult to predict. We are capable of creativity, thus are never truly predictable.
Even if you could predict behaviour given some finite list of inputs, the difficulty then becomes managing that list for every individual. Economic systems involve billions of these individuals interacting hundreds of times a day, across hundreds of countries, in thousands and thousands of sub-markets within those countries. Every economic decision can be impacted by a trillion different things.
This wouldn’t be a problem if we could find abstractions for the behaviour of collections of these individuals. But this also is difficult.
Economic systems are complex environments, characterised by non-linearity, interdependence, self-organization, and emergence. The rub is that you can’t reduce them to individual components, analyse them, and aggregate the conclusions of the analysis (and use it to forecast).
They are unbounded. There is no limit on how much debt a nation can accumulate or how much wealth a singular entity can amass. Models don’t like this.
But what if we were able to come up with these abstractions?
The problem now becomes the fact that people can change their behaviour based on the forecast. Because theory affects policy, that in turn affects outcomes (see the Lucas Critique). Your beautiful forecast might become public domain, and thus actors might behave differently based on your forecast, which may in turn change the variable you were forecasting. We have a nasty recursive spiral: you tweak your forecast based on reaction to that forecast, but now there is a new reaction to your new forecast, so you have to tweak it again to account for that, and so on.
And to top it all off, the domain is open.
Meaning: everything about the system can change entirely within months. There are no rules. We may allow interest rates to go negative. We may ban usury. We may do away with capitalism. We may start using cats as currency. So even if we come up with good theories of how our economy works TODAY, for some political or societal reason, the environment could change overnight, rendering these theories and resulting forecasts worthless.
So just, like, don’t worry about them???
When you’ve been glued to forecasts from the OBR or whoever this week, you’ve been fooling yourself.
If the reasons listed above don’t convince you, go and look for yourself. As in, if you’ve got too much free time, you can go back and look at the forecasts that big, impressive economic institutions deliver. See how accurate they are.
If they do evaluate performance, you’ll usually find it attached to lots of this type of rhetoric:
“Important to note…unexpected events…unforeseen circumstances…very important to note…supply-side shock…re-iterate that this is due to surprise inflation.”
Don’t believe me? I’ll go right ahead and pull direct from the OBR’s latest evaluation report:
The Covid, energy, and interest rates shocks are important context for interpreting the differences between outturn and the three economy forecasts considered in this report. Over 2023-24, our forecasts two- and five-years ahead overestimated real GDP growth, driven by the impact of the shocks and subsequent very low productivity growth, but all three underestimated inflation, primarily due to the unanticipated impacts of the energy price shock.
That’s their excuse for forecasting borrowing 10x lower than it actually was:
In the March 2019 five-year ahead forecast, made before the Covid pandemic, we expected that borrowing in 2023-24 would be £13.5 billion (0.5 per cent of GDP). This was £117.6 [sic] billion (4.2 per cent of GDP) lower than outturn.
Can someone tell me, is that any good? I didn’t even bother to cherry-pick these numbers, I literally found the first chart in the first report I came across and hey look they whiffed by a comically-large margin.
And then also think about how much these forecasts change. As in whole economic outlooks transforming off the back of minor policy changes.
This might be something to do with some questionable incentives. Forecasters are probably inclined to complicate what they’re doing and why because they’re trying to impress their boss, because that’s how it’s always been done, because they’re trying to sell a product, etc.
They also have weak skin in the game. Meaning if their forecasts are waaay off, they’re at their next job, or the results can be blamed on someone else. Their net worth/income/professional rep isn’t fully on the line.