Skip to content
Home » Blog » The Accuracy of Country Risk Ratings

The Accuracy of Country Risk Ratings

This article summarizes Chapter 5 of Country Risk: The Bane of Foreign Investors by Norbert Gaillard (Springer, July 2020)

 

This chapter focuses on the shocks that may affect firms involved in business relations abroad. There are three main reasons for studying shocks. First, they are an observable embodiment of country risk. Second, they can be identified and measured more easily than latent risks, whose effects may be difficult to perceive or may become evident only in the long term. Third, shocks enable one to test the accuracy of country risk indicators.

Selecting a country risk component as a potential shock is based on the analysis conducted in Chapter 3. The selection is contingent on four factors: (i) the availability of reliable information about the focal component; (ii) the extent to which such information can be quantified; (iii) the possibility of observing a sharp decline in the component’s measure within a short period of time (typically a year or less); and (iv) the shock’s capacity to affect the business operations of a wide array of firms or sectors in the short to medium term.

I find that eight components – accounting for five of the seven broad components of country risk analyzed in Chapter 3 – satisfy these four criteria: major episodes of international political violence, major episodes of domestic political violence, expropriation acts, high-inflation peaks, deep economic depressions, significant restrictions on capital flows, sovereign debt crises, and exceptional natural disasters.

For each component, I determine a specific threshold beyond which its measure’s decline is significant enough that one can reasonably deem the component to have been affected by an actual shock. My efforts to set unbiased thresholds are supported by academic research, professional publications, and press articles.

The eight shocks materialized several times during the period under consideration (1985–2014), resulting in a number of country risk crises.[1] In the discussion to follow, one such crisis corresponds to one specific shock experienced by one country a given year. The crises identified will allow me to test the accuracy of country risk indicators.

The eight idiosyncratic shocks resulted in 298 country risk crises. The next step is to aggregate the data so that no country-year observation is counted more than once. For example, Albania was hit by two distinct shocks in 1991 – a deep economic depression and a sovereign debt crisis – but only one crisis is counted: “Albania:1991”. Hence my final sample consists of 272 observations.

An examination of these 272 crises reveals that 117 nations were hit by at least one of the eight shocks. There were 23 countries that experienced at least four crises during 1985–2014: Argentina, Belize, Bulgaria, the Democratic Republic of Congo, Ecuador, Egypt, Indonesia, Iraq, Jamaica, Kazakhstan, Kyrgyzstan, Liberia, Mexico, Nigeria, Pakistan, Peru, Romania, Russia, Sierra Leone, Turkmenistan, Ukraine, Venezuela, and Zimbabwe.

Now, I analyze how well the ratings issued during 1984–2013 by Euromoney, International Country Risk Guide (ICRG), the Organisation for Economic Co-operation and Development (OECD, the Heritage Foundation, the Fraser Institute, and the World Economic Forum anticipated the country risk crises observed during 1985–2014. After presenting the methodology of these raters, I examine their performance.

I focus on six simple criteria to measure the performance of country risk indicators:

  • The average rating assigned to a country one, two, and three years before it was hit by a shock.
  • The average percentile ranking assigned to a country one, two, and three years before it was hit by a shock.
  • The percentage of countries in the “risk-free” rating category one, two, and three years before they were hit by a shock.
  • The percentage of countries in the top 25% of all ranked countries one, two, and three years before they were hit by a shock.[2]
  • The percentage of countries in the “safe” and “relatively safe” rating categories one, two, and three years before they were hit by a shock.
  • The percentage of countries in the top 50% of all ranked countries one, two, and three years before they were hit by a shock.[3]

An important finding is that certain types of shocks are much harder to anticipate than others. Table 1 shows which types of shocks most frequently led to a misrating.

 

Table 1 Misratings and types of shocks

Misratings by causal type of shock (raw number) Misratings by causal type of shock (as % of all misratings)
Major episode of international political violence 0 0.0
Major episode of domestic political violence 1 1.4
Expropriation 19 27.5
High inflation 1 1.4
Deep economic depression 8 11.6
Restriction on capital flows 23 33.3
Sovereign debt crisis 13 18.8
Exceptional natural disaster 4 5.8

Note: A misrating occurs when a crisis country is assigned a rating in the “risk-free” category. The six country risk raters considered were responsible for a total of 69 misratings that occurred either one, two, or three years prior to a crisis. Values in the last column do not total 100% because of rounding.

Source: Author’s calculations.

 

More than 60% of misratings were caused either by an expropriation or by a sudden control of capital flows. The latter is extremely difficult for country risk raters to anticipate when one considers that it accounts for only 5.4% of the 298 shocks observed during 1985–2014.[4] In Sections 5.4.2.2 and 5.4.2.3, I make some recommendations that can facilitate better anticipation of these two shocks.

Examining the accuracy of the ratings assigned by the six raters reveals some performance gaps. It appears that the Fraser Institute’s Economic Freedom of the World index is less accurate than other indicators owing to the high proportion of crisis countries in its top ratings and rankings. The OECD country risk ratings are probably only the fifth best. The main weaknesses of the OECD metrics are the same as those observed for the Fraser Institute. Discriminating between the four remaining raters is more difficult. That said, the accuracy of ICRG ratings and of the Heritage Foundation’s Index of Economic Freedom ratings is compromised by their excessively high percentages of crisis countries in the “safe” and “relatively safe” categories. For these reasons, Euromoney and the World Economic Forum’s Global Competitiveness Index (GCI) outperform (if only slightly) their counterparts. Euromoney’s “risk-free” rating category and the GCI’s top-ranked group (i.e., the top 25%) are especially reliable.

 

 

[1] Determining the period to be examined is constrained by the availability of (i) country risk ratings and (ii) data on the country risk components.

[2] The selection of this threshold is supported by discussions I had with economists and analysts.

[3] Idem.

[4] All the misratings caused by capital controls involved the same countries (i.e., Argentina in 2001, Iceland in 2008, and Cyprus in 2012).

Leave a Reply