Coding Errors and Challenging Assumptions: Why the Debate over Reinhart-Rogoff Matters
Last week, the economics blogosphere was ablaze with commentary on an economics paper from 2010 called “Growth in the Time Debt.” The paper was by Carmen Reinhart and Kenneth Rogoff, both of Harvard, and has come to be known as just “Reinhart-Rogoff.” What’s so big about a three-year-old economics paper? Well, most of the current calls for austerity in the U.S. and around the world cite this paper as the major influence in cutting government spending…oh, and the conclusions of the paper turn out to be wrong.
You see the paper examined the relationship between economic growth and the relative level of a country’s indebtedness. Their key finding was that once a country’s public debt was over 90% of their GDP, there was a strong negative relationship with the country’s economic growth – about 1% lower than otherwise. The study was cited by U.S. House Budget Committee Chairman Paul Ryan and European Union officials in their efforts to combat increasing budget deficits; the implication being that more government borrowing was negatively affecting economic growth, and that by lowering government indebtedness, a country’s economy could be unshackled.
And then, three researchers from UMass Amherst got ahold of the Reinhart-Rogoff data and attempted to replicate the results. Their resulting paper (nicely summarized here) found some odd exclusions of data, an “unconventional weighting of summary statistics,” and coding errors. And “coding errors” is a nice way of saying that when Reinhart and Rogoff were working in Excel, they didn’t drag their formula down to include all of the data they should have.
The result? There doesn’t appear to be any magic line at 90% debt-to-GDP levels – at least using their dataset. Other research on this topic has been mixed. An IMF paper from 2010 found “some evidence” of 90% threshold, but a 2012 IMF paper found no threshold. Researchers from the Bank for International Settlements (BIS) found the threshold to be 85%.
But the errors are only part of the issue. The assumptions that policymakers made were the other part.
Let’s assume for a sec that the Reinhart-Rogoff data was immaculate and that there is a relationship between 90% debt-to-GDP levels and slacking economic growth. Reinhart-Rogoff initially only suggested that there was “correlation.” When pundits and policymakers got ahold of the study, they started to imply “causation” – high government debt causes slow economic growth. You may have had a science or statistics professor tell you, “Correlation doesn’t imply causation.” Well, now you can see it in action. You’ve got a problem assuming that 90% debt-to-GDP causes slower economic growth, because there’s every reason to believe that it works the other way.
Take this as an example. High perspiration levels are correlated with days where the temperature is over 90 degrees Fahrenheit (32 Celsius). You could grab that data and say that high perspiration causes the temperature to go over 90 degrees…but you’d be wrong. The causation works the other way, but correlation doesn’t imply causation. To put it into Reinhart-Rogoff context, perhaps slow economic growth causes a country’s government to borrow more money, which increases the debt-to-GDP ratio. As Arindrajit Dube at the Roosevelt Institute points out, when the economy slows, the government must pay for more unemployment insurance and at the same time receives less tax revenue with fewer people working—both lead to a higher deficit.
There could also be another causative factor that the data doesn’t capture. I can cite data that shows that as ice cream sales increase, the number of sunburns also increases. And then I could assert that ice cream causes sunburns…and I’d be wrong. The assertion ignores the fact that ice cream is sold more during hot summer months, and that people are outside more during the same months and are more likely to get sunburned. Ice cream sales and sunburns are correlated; one doesn’t cause the other. For Reinhart-Rogoff, this may mean that there’s another variable that’s related to both government debt and economic growth that’s holding sway over them.
So even if Reinhart and Rogoff’s data was perfect, it’s dangerous to leap to the conclusion that high government debt causes slower economic growth. It’s dangerous because you’re creating fiscal and economic policy based on a relationship that may or may not exist. That can put the livelihoods of real people into jeopardy.
So the lessons from this are 1) correlation doesn’t imply causation, and 2) check, double-check, triple-check, and quadruple-check that your Excel equation captures ALL of the data that it needs to. By the way, we all make mistakes. Here is link from Bloomberg listing some other data fails that are much more serious than Reinhart –Rogoff’s Excel gaff.
The information in this article has been derived from sources believed to be accurate. Information derived from sources other than Principal Global Investors or its affiliates is believed to be reliable; however, we do not independently verify or guarantee its accuracy or validity.
The information in this article contains general information only on investment matters and should not be considered as a comprehensive statement on any matter and should not be relied upon as such. The general information it contains does not take account of any investor’s investment objectives, particular needs or financial situation, nor should it be relied upon in any way as a forecast or guarantee of future events regarding a particular investment or the markets in general. All expressions of opinion and predictions in this document are subject to change without notice.
Subject to any contrary provisions of applicable law, no company in the Principal Financial Group nor any of their employees or directors gives any warranty of reliability or accuracy nor accepts any responsibility arising in any other way (including by reason of negligence) for errors or omissions in this article.