How can we measure political risk when we can’t measure political risk?

A Martin Wolf column in the Financial Times earlier this month discusses how political risk, and political developments more generally, are impacting global markets. It’s a good article, but one odd item jumped out at me.

At the end of the article, Wolf includes a chart of the Geopolitical Risk (GPR) index, developed by Dario Caldara and Matteo Iacoviello, both economists at the Federal Reserve (more info in the paper “Measuring Geopolitical Risk” by Dario Caldara and Matteo Iacoviello at https://www2.bc.edu/matteo-iacoviello/gpr.htm).

This index says that the November 2015 terrorist attacks on Paris created a level of political risk nearly double to the Arab Spring, more than Iraq invading Kuwait, and that the fall of the Berlin Wall was a time of below-average geopolitical risk.

It is developed by two PhD economists at the US central bank, cited by an award-winning writer in one of the world’s leading newspapers — all signals that this is as well-constructed an index as could be expected.

And yet it says that a set of terrorist attacks that killed 130 people, while certainly awful, indicated more political risk than revolutions that ended the rules of four authoritarian leaders and led to two civil wars, or one state invading another, or the end of the most visible symbol of the Cold War.

Clearly, something is off with this index.

Or is something off on any index that attempts to capture political risk?

The GPR index is built on the frequency of articles in 11 newspapers that include geopolitical risk words (full description of their methodology and the table of words are here). Of these newspapers, 6 are based in the US, 4 in the UK, and 1 in Canada.

As such, this index is not measuring geopolitical risk. It is measuring US/UK/Canadian media attention to geopolitical topics.

This leads to predictable skews. Anything that happens in a major city with easy newspaper coverage and high interest (like the Paris attacks) will be overrepresented; the opposite for areas with difficulties for coverage. Ongoing events (like the Syrian civil war) don’t reflect the constant levels of political risk as they become normalized and slotted into existing coverage patterns. Other than for obvious run-ups to wars — such as those before the Gulf War and Iraq War — this methodology reflects events rather than anticipating them, which is what political risk is usually meant to be aimed at.

Now, this certainly does not mean that this index is useless. The authors acknowledge many of the limitations of the methodology. They chose this approach because it allows for a transparent and replicable data set. Their design serves an important purpose by facilitating research into the correlation between geopolitical events and economic and financial patterns.

But it does show the difficulty of using any single, purportedly objective, metric as the basis of political risk forecasting, when even a well-crafted index only tracks past political risk coverage.

The GPR index also highlights the difficulty in measuring political risk, especially since that term includes so many different elements of varying relevance for an organization.

To a mining company, civil unrest near a mine may be the most important political risk, but receive no mention in the international press. For a military policy planner, the risk of a war in any part of the world in the next 20 years may have little connection to today’s events. A theoretical index that is perfect for each one would be of little to no use for the other.

The GPR index, like many other political risk ratings or metrics, is a useful indicator for a political risk analyst. But it is nonetheless measuring only a proxy for political risk. The global phenomenon of political risk is made up of numerous discrete risks — wars, expropriations, trade barriers — that are unevenly distributed across countries and sectors. A metric can serve as a window, but will always be a limited and single viewpoint.

There is a bright side to all this, however.

The GPR index is a reminder that the question for political risk analysts is not the extraordinarily difficult task of measuring political risk writ large. It is measuring the political risks that matter to their work, using metrics when and if they are applicable. It’s still difficult, but at least gives a starting point for narrower, and hopefully more feasible, analysis.

Note: This article was originally posted on October 31, 2018