2008-03-12

It must be all the fish they eat

Mafalda once noted that you hardly ever hear about Norway in world news as dried fish isn't as newsworthy as wars. Now the Global Peace Index has shown that, indeed, Norway is the most peaceful country on Earth. (And Iraq is the least peaceful, surprise, surprise.)

Well, having strolled around in Oslo, even late at night, I can well believe that and certainly the Norwegians deserve the distinction, but I can't help but think a bit more about how the measurement has been made. It is scientific in the sense that the scoring is presented and the motivations for the choices of parameters given. The authors note that there is a certain level of arbitrariness in assigning scores.

I'd like to pick on a few of these arbitrarities.

Neither the Vatican, San Marino, Liechtenstein, Luxemburg, Monaco nor Andorra have been included, nor are Niau, Tuvalu, Cocos Islands or Surinam. Maybe they were considered to be too small to count, but if so, that does say something about reasons for peacefulness. (And Afghanistan, Belarus, and Eritrea have not been included either.)

Much of the ranking depends on the nations' scores on various 5-level scales. Having a discretised scale introduces various quantification errors. Ideally I should now have done a perturbation analysis to see if changing a score by a single unit would change the ordering between, say, Norway and New Zealand (the current number two on the peacefulness list), but I have not been able to find the exact weighting formula that has been used. (Possibly it is given somewhere on the site, but not sufficiently obviously to me.) The methodology section indicates that various numbers have been “banded” into these discretised scores. It's not obvious why one would want to do that, as retaining continuous data would give higher data quality, in that we would not be throwing information away. Furthermore, if any of these variables are assessments, rather than quantised continuous data, then we are abusing the numbers.

To wit: Using the numbers, say, 1–5 to grade something introduces an ordinal scale: we can say that “1” is better than “2”, but we cannot really tell if the difference between “1” and “2” is equal to the difference between “3” and “4”. That's why computing grade point averages really is bogus, it's not a well-defined operation. For that you would need at least an interval scale, where we are guaranteed the each step is the same size as any other step.

So the moral is: if Flight of the Conchords are more peaceful than KLM then you may not have been qualified for university.

No comments: