Flawed study: Global patterns of terrestrial vertebrate diversity and conservation

Recently, a paper by C. N. Jenkins, S. L. Pimm and L.N. Joppa called "Global patterns of terrestrial vertebrate diversity and conservation" appeared in PNAS. The authors claim that they have refined global map of priority conservation areas to a grid of 10 km x 10 km resolution. They call this an improvement as most of the previous global studies used pretty coarse resolution of 100 km x 100 km. The paper also sells some strong statements about the distribution of global conservation priorities.

The paper is methodologically flawed.

I find it troublesome given the whole setting of the study: high-profile journal, big names, pressing theme of global importance. Not only are the authors' reputations at stake (S.L. Pimm is a leading conservation scientist) - the credibility of the discipline can be harmed. Moreover, the paper will probably be heavily cited and it will cost some effort and time to fix the disinformation that it will have spread.

Here is why I have such a negative opinion:

  • It is well established that expert-drawn range maps hugely overestimate species occurrences at resolutions finer than 100 km x 100 km. Actually, the same journal had published a seminal paper (Hurlbert & Jetz 2007) on that issue and there are other studies that demonstrate the same point, which is quite straightforward in principle, but which the authors completely ignore when producing their global maps. They simply proceed and sell their quick and dirty product.
  • At several places the authors claim that the grains of 100 km x 100 km and coarser "degrade the raw data and obscure crucial patterns of diversity in regions of rapid species turnover", that "such a coarse scale unnecessarily blurs the data, most importantly the data on where species occur" and "generalizing the data to a grain of 100 km x 100 km destroys vital information". These statements are misleading. They depend on how one defines¬†precision and what exactly is meant by the metaphoric "blurring". One can argue that the situation is exactly the opposite to what the authors say: First, fine grain maps of richness based on range maps are actually less precise because of the overstated species occurrences. Second, coarsening-up the grid resolution is not blurring information, it is just more correctly representing the uncertainty about where exactly species live - a species is simply considered to live somewhere within the 100 x 100 km grid cell, but we don't know where. Jenkins et al. may call it blurring, I call it honesty.
  • The authors present five take-home messages at the end of the paper. Because of the limitations outlined above I consider them largely unsupported. The most problematic is the message that: "Protection levels for priority areas are greater than the global average but still are insufficient". Any reported percentages and species numbers are heavily dependent on the correct representation of species presences and absences at the 10 km x 10 km grain. We know that the presences of species in the protected areas based on expert range maps are almost certainly overstated. So it just all falls apart. Another message is titled "Spatial grain of the analysis matters". Ok, it does matter, but in a completely different way than the authors claim in the paragraph. And not a single sentence in this paragraph is supported by the results (or by other studies). It is all just randomly generated opinions.
  • The authors seem to be aware of these issues at least superficially. They sweep them away with an argument that "although these limitations are real, range maps currently are still the best data available for assessing very large areas for large numbers of taxa". Well, saying that "the limitation are real" does not exactly sound like a reason to proceed regardless of the limitations, without even trying to account for them by modelling them.

An additional observation, which actually applies to other studies in large-scale ecology: The authors have fallen into the trap of inferring their messages directly from the data - they consider the data to be the truth. Their study clearly demonstrates the danger of using a third-party data without any insight into their elementary properties. But more importantly, this data-oriented approach is philosophically problematic: Even "good" data are not a perfect representation of reality, they are usually just one of many possible realizations of the underlying processes, and are then filtered (and biased) through observation. Reality can only be revealed by a proper hypotheses or models that are fitted to the data. The best inference is not derived from data alone, it is derived from a model whose parameters are conditional on the data.

I recommend that the authors start their methodological rehab by thinking about my last point. To be specific: They should have not produced hi-res maps of the inappropriate raw data. Instead, they should have provided hi-res predictions of a model that explicitly accounts for scale dependency of species richness and for uncertainty in species distributions.

Does it seem too "challenging" (to cite the authors)? Well, welcome to the world of ecology.

5 Responses

  1. Brad Hawkins
    Brad Hawkins June 27, 2013 at 3:45 pm |

    Give them hell, Peter. Just don't expect it to make any difference.

    Reply
  2. Fabricio Villalobos
    Fabricio Villalobos June 29, 2013 at 3:40 am |

    Well put, Peter. I immediately thought the same when I saw the paper and went thru the methods trying to find something more than just their so-called "improvement" of using a finer resolution grid. One good thing they could have done (and maybe more appropriate for a high-profile journal), if they really wanted to "refine" their priority assessments, was to model species distributions and downscale their data with an appropriate model (if data were available and they were willing to take the "complicated" step) ( Keil et al. 2013 on ). You may have been modest, but I think using your framework could a real improvement to perform conservation assessments.

    Reply
  3. Falko Buschke
    Falko Buschke July 26, 2013 at 1:18 pm |

    I haven't read the entire PNAS paper (but the high res figures really are beautiful), so I can't make well-informed comments. Nevertheless, I am curious to here your opinion on something, Petr.

    The whole "hook" of the paper was that it reduced the discrete grid ten-fold. What next, another ten-fold reduction? Surely we should stop reducing the ranges into smaller and smaller discrete cells and just analyses them continuously?

    It's pretty easy. In ArcMap, just use the feature to polygon tool to determine the "sampling units" from the ranges themselves and then count the number of polygons that overlap these units. This will cut out the need for any grid.

    On the downside, the units will all be different sizes and any analyses with environmental data will still be limited by the resolution of the climate raster. On the plus side, however, the units will be ecological units (not arbitrary grids), which actually represent Robert Ricklef's idea of a disintegrated community of overlapping ranges instead of arbitrary discrete point estimates of assemblages.

    Of course, none of this solves the issue about data quality.

    That said it is also essential to understand exactly what this is measuring regardless of data quality issues. First, it is important to know that the range maps are extent of occurrence (EOO) and not area of occupancy (AOO), so we must understand that the richness reported by these methods are the maximum possible number of species (Observed species plus the so-called "dark-diversity") and not the actual number of species occurring within the region. Second, one might argue (I know I do) that what this is measuring is the richness of the regional species pool and not local species richness. Should this matter for conservation prioritisation? I think it should.

    In short, I was wondering if you had given any thought to analysing distributions continuously, without using any grids and, if not, do you think it is an avenue worth pursuing?

    Reply
  4. Marten
    Marten August 16, 2013 at 8:35 am |

    Well, a nice read. I also thought about this paper but more in the way how does it improve the question all papers mentioned here are made for: The conservation of species. Petr, I see your point and can only second Fabricio (heja man! ;O) to write a comment, re-analyze or whatever reply is appropriate (this is the only way to improve science and enhance discussions, this blog is nice but the audience is very limited)...anyway BUT still how do the results change the conservation policy. They don't (probably never) because the much more important question is and here Petr you can bring things forward (maybe): don't we already conserve what we want to conserve in conservation areas in those "newly defined" hotspots? Before new cons. areas will be opened or existing ones will be enlarged, politicians would like to know what is in the areas they support and what not!

    A method discussion is nice but for politicians not interesting enough. And PNAS is a journal for a broader audience...there are more things to be kept in mind when papers are published in those high rank journals...

    Interested in hear your thoughts...

    sorry for bothering with this :)
    cheers
    M

    Reply

Leave a Reply