The beginning of the year is filled with forecasts. People love to read them, and as a practical matter, we may use them in preparing capital market assumptions, which are required for the portfolio construction process and other models we use every day. How accurate are these forward looking estimates?
William Sherden reviewed research on forecasting accuracy from 1979 to 1995. He concluded that1:
- Economists cannot predict the turning points in the economy. Of the 48 predictions made by economists, 46 missed the turning points.
- Economists’ forecasting skill is about as good as guessing. Even economists who directly or indirectly run the economy (the Fed, the Council of Economic Advisors and the Congressional Budget Office) had forecasting records that were worse than pure chance.
- There are no economic forecasters who consistently excel in forecasting accuracy.
- Consensus forecasts don’t improve accuracy.
According to Jan Hatzius, the chief economist of Goldman Sachs2:
- The majority of economists didn’t “predict” the three most recent recessions (1990, 2001 and 2007) even after they had begun.
- In November 2007, economists in the Philadelphia Federal Reserve’s Survey of Professional Forecasters called for growth of 2.4 percent for 2008, with only a 3 percent chance of a recession, and only a 1 in 500 chance of the GDP falling by more than 2 percent. GDP actually fell 3.3 percent.
- Since 1990, economists have forecasted only two of the 60 recessions that occurred around the world a year in advance.
So we don’t see the turning points ahead of time. What about the numerical forecasts themselves? The Fed studies their own and consensus forecasts from time to time. In 2015, an analysis of several economic indicators found that using the most simple statistical models (such as a random walk, or a first order autoregressive model) equaled or outperformed the Greenbooks (the estimates used by the Fed) for all time periods modeled except less than one quarter ahead.3
It turns out that the forecasts used by the government, and “consensus” forecasts, have two main ongoing biases: recency bias and optimism.
From FOMC meeting minutes and FRED data:
The above chart shows both recency and optimism. The recency can be seen as near term estimates are quite close to the most recent year’s actual data. The optimism can be seen as every estimate comes in above actual results.
This chart shows consensus earnings forecasts, prepared by Charter Trust Company5:
So there’s some more of the optimism. There are many examples of similar charts for other estimates as well.
This chart from Resolve Asset Management shows the recency bias, using bond yield forecasts and data. Again, the forecasts were from the Fed. Source: (Brooks & Gray, 2003)6:
The CBO also checks on its own forecasts and makes comparisons to around 50 private sector forecasts. Their most recent analysis includes the time period 1982 – 2012. The result of this analysis is that their forecasts deviated from actual outcomes by 1.4 percentage points for real GDP growth and 0.8 percentage points for inflation.7
Edward P. Lazear looked at the CBO data from 1999-2013 and found the GDP estimates to be off by 1.7 percentage points. The reason this is notable is that during that time period, the average GDP was only 2.1%. They didn’t miss by 1.7% of 2.1%, they missed by over 80% of the actual result. On average.
Lazear also found that “History is a better predictor of annual growth than government forecasts. Simply assuming that GDP growth will be 3.1% in each year—the average annual rate for the 30 years that precede the study period—results in an average forecast error of 1.5 percentage points.”8
So we know the forecasts are wrong, but we still need estimates for our models. The research above leaves us with a couple of options that are at least as accurate as the forecasts: simple statistical models, and historical averages. The simplest model may be all the complexity that is needed, and the naïve historical average may not be so naïve after all.
6http://www.investresolve.com/blog/bold-confident-wrong-why-you-should-ignore-expert-forecasts/; History of the Forecasters: An Assessment of the Semi-Annual U.S. Treasury Bond Yield Forecast Survey as Reported in the Wall Street Journal (December 1, 2003)