It’s a given that economic forecasts are rarely correct. In fact, prior to the recent global financial crisis, forecasters consistently underestimated the Canadian economy’s strength, repeatedly allowing the federal government to deliver surprise surpluses. The risk now is that the opposite problem faces policy-makers in the years ahead.

Last month, the federal Department of Finance released an archive of the economic forecasts it has used as part of the budgeting process since 1994. The data cover the average of about 15 private-sector economists’ forecasts in a number of key variables: gross domestic product growth, inflation, benchmark interest rates (treasury bills and 10-year government bonds) in both Canada and the U.S., along with the Canadian jobless rate, the exchange rate for the U.S. and Canadian dollars, and the prices of oil and natural gas.

To get a sense of how well these forecasters have performed, Investment Executive has examined three of the big items they cover: the domestic GDP growth rate, the three-month T-bill rate and the exchange rate between the C$ and US$. IE compared the forecasts published closest to each yearend with the actual results, then measured the relative size of the forecasting error over time. (The forecasts aren’t always made in the same months every year, so IE focused on the ones closest to the end of the year, ranging between December and February.)

An analysis of those forecasts suggests that they usually do a decent job in the short term, under normal conditions. However, forecast accuracy suffers the longer the time horizon for the prediction. And these forecasts are pretty useless at foreseeing unusual events, such as an extraordinary economic downturn.

Even the results forecasted at the end of the year aren’t entirely accurate for some metrics because it takes some time for the data to be finalized — and the preliminary numbers can be subject to later revisions (particularly for economywide data, such as GDP readings, inflation and employment reports, as opposed to single-factor measures such as oil prices or exchange rates). That said, even for the very complex readings, such as GDP, the yearend predictions tend to be accurate.

However, as soon as you ask economists to peer even one year into the future, you start seeing some fairly significant misses — both on the upside and the downside. (The graphs at right show the forecast errors in terms of absolute value to demonstrate the size of the miss without regard to direction.)

Comparing one-year forecasts of GDP growth, T-bill rates and the C$/US$ exchange rate, the economists appear to do best calling the T-bill interest rate. The GDP forecasts demonstrate the widest margins for error.

The data also show that in general, the farther out the date applying to the forecast is from the date the forecast is being made, the wider the gap between the prediction and the actual result. And in some years, the forecast horizon is very distant, extending as far out as six years. Of course, this is to be expected; the farther you try to look into the future, the more assumptions you have to make — and the less certainty there is likely to be to a prediction.

Also, none of the forecasts is useful at all when it comes to anticipating more extreme events, such as the recession that hit in 2008, for which economists’ predictions proved wildly off the mark — although, their exchange rate calls were much more accurate.@page_break@That downturn was brought on by an exceptional global financial crisis that most economists didn’t foresee. It wasn’t a slow-building cyclical move that forecasters could be expected to see on the horizon, so it’s not surprising that their predictions were so far off. However, their forecasts also proved much too optimistic ahead of the far less dramatic 2001 economic downturn. Conversely, they’ve also been too conservative ahead of years of very robust economic growth.

One reason for these large forecasting errors in certain years appears to be that these forecasts don’t anticipate as wide a range of possible outcomes as actually occurs. Looking four and five years into the future, economists can only assume that things will trend toward their historical norms.

Indeed, if you look at the actual GDP growth rates over the past 15 years, the results have a standard deviation of two, whereas the forecasts only one year out from those same beginning years have a standard deviation of slightly more than one. This means there is much greater variability in the actual results than in the one-year forecasts.

Three years out, the standard deviation for these forecasts drops below 0.4; five years out, it’s down to just 0.23. In other words, the farther into the future that economists are asked to project, the smaller the range of their forecasts. Indeed, the average long-term forecast for GDP growth has been about 3% (the average forecast for two, three, four, five and six years out).

What will be interesting is how this practice adjusts to what some economists believe is sure to be a secular shift into a lower-growth phase as both households and governments are forced to curb their spending amid growing debt loads. In addition, inexorable demographic forces will limit the economy’s growth potential by constraining labour force growth and increasing the demands on the health-care system and public pensions.

Already, there are signs of this shift taking place in these forecasts. For instance, the latest GDP forecast (released in June) calls for GDP growth of 2.8% in 2012 and 2013, sliding to 2.5% in 2014 and 2015, and to 2.4% in 2016.

These predictions will surely be proven wrong, but the trend suggests that economists see growth potential declining. What remains to be seen is if this outlook is pessimistic enough. In fact, a recent report from the Organization for Economic Co-operation and Development predicts that GDP growth in Canada will slump to just 1.6% in the years ahead as the effects of demographic change take hold.

In the past, forecasters apparently have been too conservative in their estimates of economic growth, leading to persistent surprise budgetary surpluses. For example, in 2005, a research paper by the International Monetary Fund observed that Canada’s budget forecasts had meaningful errors and that “public and private forecasters were repeatedly surprised by the strength of the Canadian economy and fiscal performance.”

The IMF study suggested that it may be tough to weed out these sorts of errors in a system that relies on private-sector forecasts as part of the budgeting process. The study compared Canada’s budget forecasting process with those of other countries, observing that Canada leans more heavily on private-sector economists for macroeconomic forecasts than many other countries.

Furthermore, the IMF study found that a reliance on outside forecasters may boost the credibility of government projections, but that this sort of broader consultation “could imply the use of less systematic forecasting techniques, which may make it more difficult to pinpoint the cause of forecast errors.”

Budget forecasters may have been too cautious to predict strong economic growth in the past. However, the risk in the years ahead is that they may be too cautious in marking down the economy’s potential, giving policy-makers a false sense of security. IE