Scientific superpowers and Simpson's paradox

By | November 25, 2012

I have always loved rankings, charts of results and performance measures. Recently I stumbled across Thomson Reuter's Essential Science Indicators - a scientometric database that ranks countries, institutions and individual scientists according to their publication output and citation impact. Wow! This can help me to answer my big question: Are United States the scientific center of the Universe? Specifically, do Americans publish more and do they publish more influential publications then say Europeans or Asians?

First, I simply had a look at the first 14 countries ranked by the numbers of all scientific papers published between 2002 and 2012:

Well, at first glance it looks pretty clear - USA rocks in both number of papers and citations. China is the second most productive country but it is definitely not the second most cited. The two megapowers are followed by the usual suspects: Germany, Japan, England, France and so on. But these absolute numbers of high scientific productivity can be very well caused by population sizes of the countries. We still don't know if say an American is better scientist than a German. So I collated some data on population sizes of the countries (from CIA factbook),  I selected a set of interesting "superpower-ish" countries and calculated number of papers published per citizen in each of them:

Now it starts to be interesting. China sinks way down with an average production of 0.7 scientific papers per 1000 inhabitants during the last ten years. United States also sink a little bit - Canada, Australia, Israel, Norway and some EU countries (the boxplot - it shows median and quartiles) publish more. Switzerland is at the very top with 25 papers per 1000 inhabitants during the last 10 years.

Ok, but are the papers published by the most productive countries any good? Let's have a look at the mean number of citations per published paper:

The top countries remain roughly the same, but Unites States jump up. And now none of the EU countries produces as well cited papers as the United States! The US are actually contested only by Switzerland. Sad news for the EU is that an average American paper is cited by six (!!!) more papers than an average European paper.

But can't this be just an artifact of how the data are aggregated? Maybe if we compared US with the whole EU it would make more sense. Or similarly, how about comparing performances of individual US states and individual EU countries? The Essential Science Indicators do not provide data on individual US states and so I manually pulled them out of Web of Science. Unfortunately, I was only able to get data on number of published papers. Web of Science does not do citation reports for reports of more than 10,000 papers (or so).

Here is what I found:

Maybe not the most obvious, but still, this is an example of a contra-intuitive statistical phenomenon called Simpson's Paradox! To demonstrate it, I also show separate data on Japan and the UK. Simpson's paradox means that the median (calculated across the states) of per-capita publication rate in the US is higher than the per-capita publication rate in the UK. But when US is taken as a whole (i.e. we divide number of all American papers by number of all Americans), US has lower publication rate than the UK. Similar situation can be seen when we compare EU and Japan. So a region's relative scientific productivity depends on how the data were aggregated.

To summarize, I have shown that:

1. China is the second most productive country in terms of volume of scientific papers. However, this is caused mostly by its enormous population size. Per-capita publication rate and citation impact of Chinese publications have been low.

2. EU countries publish similar per-capita quantities of scientific papers as United States. However, EU countries cannot compete with the US in quality of the papers.

3. USA produce the highest volume of the top-quality research in the world and they really are the scientific center of the world.

4. The data on scientific performance of World's superpowers show interesting cases of Simpson's paradox.

(Visited 448 times, 1 visits today)

One thought on “Scientific superpowers and Simpson's paradox

  1. ilyas guven

    You don’t take into account the language barrier. If Russia had spoken English it would have many more cited papers or if the US had spoken Russian it would have fewer. The citations alone don’t mean anything.


Leave a Reply

Your email address will not be published. Required fields are marked *