AUTHORS: Helen R. Sofaer*, U.S. Geological Survey; Catherine S. Jarnevich, U.S. Geological Survey; Jennifer A. Hoeting, Colorado State University; Curtis H. Flather, U.S. Department of Agriculture Forest Service
ABSTRACT: Species distribution models are an important source of information used to guide conservation and management decisions. The decisions guided by distribution model outputs include management prioritization in space and among species, and the design of search and surveillance efforts. Regardless of their intended use, models are typically assessed based on discrimination performance, which is the ability to differentiate between occupied and unoccupied locations. Our work evaluates whether discrimination performance is an effective indicator of a model’s utility for decision-making in different contexts, and we consider additional, complementary metrics. In a validation study based on Breeding Bird Survey data, we show that species distribution models with excellent discrimination performance and overall temporal transferability nevertheless fail to accurately predict range expansion and contraction. These models therefore fail to identify the species that are most vulnerable to global change or that pose the greatest invasion risk. Using simulations, we evaluate precision-recall curves as an additional method of performance assessment, and consider variation with species prevalence. Precision is the probability that a species is present, given a prediction of presence, and so may be particularly useful for assessing model utility for guiding search and surveillance. We suggest that models should be assessed along the dimensions of their intended use, and that model developers communicate appropriate uses to decision makers.
