Flawed study: Global patterns of terrestrial vertebrate diversity and conservation

By | June 26, 2013

Recently, a paper by C. N. Jenkins, S. L. Pimm and L.N. Joppa called "Global patterns of terrestrial vertebrate diversity and conservation" appeared in PNAS. The authors claim that they have refined global map of priority conservation areas to a grid of 10 km x 10 km resolution. They call this an improvement as most of the previous global studies used pretty coarse resolution of 100 km x 100 km. The paper also sells some strong statements about the distribution of global conservation priorities.

The paper is methodologically flawed.

I find it troublesome given the whole setting of the study: high-profile journal, big names, pressing theme of global importance. Not only are the authors' reputations at stake (S.L. Pimm is a leading conservation scientist) - the credibility of the discipline can be harmed. Moreover, the paper will probably be heavily cited and it will cost some effort and time to fix the disinformation that it will have spread.

Here is why I have such a negative opinion:

  • It is well established that expert-drawn range maps hugely overestimate species occurrences at resolutions finer than 100 km x 100 km. Actually, the same journal had published a seminal paper (Hurlbert & Jetz 2007) on that issue and there are other studies that demonstrate the same point, which is quite straightforward in principle, but which the authors completely ignore when producing their global maps. They simply proceed and sell their quick and dirty product.
  • At several places the authors claim that the grains of 100 km x 100 km and coarser "degrade the raw data and obscure crucial patterns of diversity in regions of rapid species turnover", that "such a coarse scale unnecessarily blurs the data, most importantly the data on where species occur" and "generalizing the data to a grain of 100 km x 100 km destroys vital information". These statements are misleading. They depend on how one defines precision and what exactly is meant by the metaphoric "blurring". One can argue that the situation is exactly the opposite to what the authors say: First, fine grain maps of richness based on range maps are actually less precise because of the overstated species occurrences. Second, coarsening-up the grid resolution is not blurring information, it is just more correctly representing the uncertainty about where exactly species live - a species is simply considered to live somewhere within the 100 x 100 km grid cell, but we don't know where. Jenkins et al. may call it blurring, I call it honesty.
  • The authors present five take-home messages at the end of the paper. Because of the limitations outlined above I consider them largely unsupported. The most problematic is the message that: "Protection levels for priority areas are greater than the global average but still are insufficient". Any reported percentages and species numbers are heavily dependent on the correct representation of species presences and absences at the 10 km x 10 km grain. We know that the presences of species in the protected areas based on expert range maps are almost certainly overstated. So it just all falls apart. Another message is titled "Spatial grain of the analysis matters". Ok, it does matter, but in a completely different way than the authors claim in the paragraph. And not a single sentence in this paragraph is supported by the results (or by other studies). It is all just randomly generated opinions.
  • The authors seem to be aware of these issues at least superficially. They sweep them away with an argument that "although these limitations are real, range maps currently are still the best data available for assessing very large areas for large numbers of taxa". Well, saying that "the limitation are real" does not exactly sound like a reason to proceed regardless of the limitations, without even trying to account for them by modelling them.

An additional observation, which actually applies to other studies in large-scale ecology: The authors have fallen into the trap of inferring their messages directly from the data - they consider the data to be the truth. Their study clearly demonstrates the danger of using a third-party data without any insight into their elementary properties. But more importantly, this data-oriented approach is philosophically problematic: Even "good" data are not a perfect representation of reality, they are usually just one of many possible realizations of the underlying processes, and are then filtered (and biased) through observation. Reality can only be revealed by a proper hypotheses or models that are fitted to the data. The best inference is not derived from data alone, it is derived from a model whose parameters are conditional on the data.

I recommend that the authors start their methodological rehab by thinking about my last point. To be specific: They should have not produced hi-res maps of the inappropriate raw data. Instead, they should have provided hi-res predictions of a model that explicitly accounts for scale dependency of species richness and for uncertainty in species distributions.

Does it seem too "challenging" (to cite the authors)? Well, welcome to the world of ecology.

(Visited 1,264 times, 1 visits today)

6 thoughts on “Flawed study: Global patterns of terrestrial vertebrate diversity and conservation

  1. Brad Hawkins

    Give them hell, Peter. Just don't expect it to make any difference.

  2. Fabricio Villalobos

    Well put, Peter. I immediately thought the same when I saw the paper and went thru the methods trying to find something more than just their so-called "improvement" of using a finer resolution grid. One good thing they could have done (and maybe more appropriate for a high-profile journal), if they really wanted to "refine" their priority assessments, was to model species distributions and downscale their data with an appropriate model (if data were available and they were willing to take the "complicated" step) ( Keil et al. 2013 on ). You may have been modest, but I think using your framework could a real improvement to perform conservation assessments.

  3. Falko Buschke

    I haven't read the entire PNAS paper (but the high res figures really are beautiful), so I can't make well-informed comments. Nevertheless, I am curious to here your opinion on something, Petr.

    The whole "hook" of the paper was that it reduced the discrete grid ten-fold. What next, another ten-fold reduction? Surely we should stop reducing the ranges into smaller and smaller discrete cells and just analyses them continuously?

    It's pretty easy. In ArcMap, just use the feature to polygon tool to determine the "sampling units" from the ranges themselves and then count the number of polygons that overlap these units. This will cut out the need for any grid.

    On the downside, the units will all be different sizes and any analyses with environmental data will still be limited by the resolution of the climate raster. On the plus side, however, the units will be ecological units (not arbitrary grids), which actually represent Robert Ricklef's idea of a disintegrated community of overlapping ranges instead of arbitrary discrete point estimates of assemblages.

    Of course, none of this solves the issue about data quality.

    That said it is also essential to understand exactly what this is measuring regardless of data quality issues. First, it is important to know that the range maps are extent of occurrence (EOO) and not area of occupancy (AOO), so we must understand that the richness reported by these methods are the maximum possible number of species (Observed species plus the so-called "dark-diversity") and not the actual number of species occurring within the region. Second, one might argue (I know I do) that what this is measuring is the richness of the regional species pool and not local species richness. Should this matter for conservation prioritisation? I think it should.

    In short, I was wondering if you had given any thought to analysing distributions continuously, without using any grids and, if not, do you think it is an avenue worth pursuing?

    1. petrkeil Post author

      Hi Falko,

      1. On Robert Ricklefs' "disintegrated community": Both diversity and community composition are defined per unit of area. They are is inseparable from the concept of area and spatial delimitation, they are extensive properties in a physical sense. Now we are 100% free to study any areas we want - the spatial delimitation (grain, resolution, scale, extent) will be always arbitrary. In fact, there is a continuous gradient of infinite number of scales going from a dimensionless point all the way up to the whole planet. Historically, the founding fathers termed discrete categories such as alpha diversity, regional diversity and so on, and I agree that these are completely arbitrary - exactly because ANY spatial delimitation is to some degree arbitrary (but at the same time necessary if we want to study diversity). I also think that it is wrong to say that diversity and species composition at the relatively fine scales should be dismissed as epiphenomena (as Ricklefs suggests) - in such case any extensive quantities in ecology would have to be abandoned, not only the concept of community. My position is that we should focus on the whole spatial scaling relationships, rather than on particular spatial delimitations.

      2. On EOO vs AOO, and the "potential occurrences" in a region. No, EOO does not tell anything about the potential of the species being at a particular point location within an expert-drawn range polygon. The only information that EOO really gives you is where the species does NOT live (which is outside of the EOO). But within the EOO? It can live everywhere, or nearly nowhere. You just don't know. In general, the concepts of EOO and AOO are sloppy exactly because they often ignore the scaling relationship of occupancy, which is the occupancy-area relationship (also termed scale-area relationship).

      3. Should species pool matter for conservation prioritization? Maybe yes, maybe no. There are all of the other spatially delimited "pools" - there is almost never a single and well defined species pool. There is infinite number of possible pools, depending on the scale that you consider. Which one is more important than the other?

      3. On analyzing distributions continuously: I think that this is an old desire that stems from the fact that when you go outside for a walk, there are no grids around you. But without grids (or other spatially delimited spatial units), there is NO biodiversity! It does not exist any more. Yes, you can track and draw continuous movements of animals in an African savanna, you can draw points representing positions of each individual of a plant in a meadow. And that is all ok. But if you are interested in biodiversity and community composition, or even in the probability of occurrence of individual species, then I don't know of any way to avoid spatial units.

  4. Marten

    Well, a nice read. I also thought about this paper but more in the way how does it improve the question all papers mentioned here are made for: The conservation of species. Petr, I see your point and can only second Fabricio (heja man! ;O) to write a comment, re-analyze or whatever reply is appropriate (this is the only way to improve science and enhance discussions, this blog is nice but the audience is very limited)...anyway BUT still how do the results change the conservation policy. They don't (probably never) because the much more important question is and here Petr you can bring things forward (maybe): don't we already conserve what we want to conserve in conservation areas in those "newly defined" hotspots? Before new cons. areas will be opened or existing ones will be enlarged, politicians would like to know what is in the areas they support and what not!

    A method discussion is nice but for politicians not interesting enough. And PNAS is a journal for a broader audience...there are more things to be kept in mind when papers are published in those high rank journals...

    Interested in hear your thoughts...

    sorry for bothering with this 🙂

  5. Alec Christie

    Really great post that my colleague forwarded to me. Just about to use similar IUCN data for plotting bat diversity and I'll be paying head to the 2 degree coarseness warning of (Hurlbert & Jetz 2007). Did you ever write a reply or a follow-up paper after publishing this post? Would be interested in it as I think it's an important issue


Leave a Reply

Your email address will not be published. Required fields are marked *