Regional Credibility Measures
Although the BBS provides a huge amount of information about regional
population change for many species, there are a variety of possible
problems with estimates of population change from BBS data. Small sample sizes,
low relative abundances on survey routes, imprecise trends, and missing data
all can compromise BBS results. In the complicated model fitting procedures presently used for the BBS, it is possible that convergence failures in regions with these deficiencies can produce results that are inaccurate as well as imprecise. Often, users do not take these problems into
account when viewing BBS results, and use the results inappropriately.
To provide some guidance to interpretation of BBS data, we have implemented
a series of checks for some attributes that we view as cause for caution in
interpretation of BBS results. We categorize BBS data in 3 credibility
categories:
This category reflects data with an important
deficiency. In particular:
- 1. The regional abundance is less than 0.1 birds/route (very low abundance),
- 2. The sample is based on less than 5 routes for the long term (very small samples), or
- 3. The results are so imprecise that a 5%/year change (as indicated by the half-width of the credible intervals) would not
be detected over the long-term (very imprecise).
Users should be aware that a variety of circumstances may lead to imprecise results. For example, imprecise results are sometimes a consequence of a failure of the models to converge in those local areas, even though the model performs adequately in larger regions.
This category reflects data with a
deficiency. In particular:
- 1. The regional abundance is less than 1.0 birds/route (low abundance),
- 2. The sample is based on less than 14 routes for the long term
(small sample size), or
- 3. The results are so imprecise that a 3%/year change (as indicated by the half-width of the credible intervals) would not
be detected over the long-term (quite imprecise), or
This category reflects data with at least 14 samples
in the long term, of moderate precision, and of moderate abundance on routes.
Note:
- 1. Due to changes in the way N of samples (in this analysis, it is defined as the N of routes on which the species
occurred), relative abundance (taken directly from the hierarchical model results), and the precision (half-width of
the credible intervals), these categories are slightly different than those used in earlier analyses. We will refine these
groupings based on user comments and our evaluation of the limitations of the hierarchical model analysis.
- 2. Even data falling in the category may
not provide valid results. There are many factors that can influence the
validity and use of the information, and any analysis of BBS data should
carefully consider the possible problems with the data. As noted above, judging whether technical issues associated with model convergence are leading to imprecise results can be difficult in analyses based on many strata, but these categories help users to screen for suspect results.
- 3. We are occasionally asked to identify which deficiency is causing the flag.
However, the point of the codes is to provide a quick and simple set of cautions to users,
and we are resisting the notion of setting up a complicated series of codes. To determine
why the code exists, look at the results. All of these deficiencies (abundances, precisions, etc)
will be evident from the results we present.