![[logo] US EPA](http://www.epa.gov/epafiles/images/logo_epaseal.gif){kind=logo}
Development and Application of Multimetric Indexes
The Clean Water Act requires all states to define water quality standards for their surface waters (Ransel, 1995). The standards include designated uses, criteria to protect those uses, and a prohibition against degradation of existing uses. States must assess the condition of water bodies and determine whether they support or fail to support their designated uses. Water bodies that fail are then listed as impaired (NRC, 2001). Currently, most states have narrative biological criteria in place, but are mandated to develop numeric criteria for biological condition (Karr, 1991; USEPA, 2002).
Multimetric indexes were created to fill this role as numeric assessment tools and are used by most states for this purpose (USEPA, 2002). The relevance and defensibility of the decision to list a site as impaired depends on both the biological meaning of index values and the statistical precision of the index. Decline in biological condition is a continuous process, and drawing a line of impairment, or defining biological criteria, at any single point along its range will inevitably be somewhat arbitrary. The most common approach for drawing this line defines impairment in terms of deviation from values observed for reference sites (Hughes, 1995; USEPA, 2002). A second approach uses statistical power analysis to calculate the change in biological condition an index can detect for a given statistical model and defines impairment in terms of that measurable change (Fore, 2002a). Other authors have recommended that better definitions of beneficial use should drive this process and that definitions of impairment should be tied to societal values, such as the support of salmonid spawning (NRC, 2001).
Biological criteria depend on the definition of reference sites
Many states currently define biological impairment in terms of multimetric index values observed at reference, or minimally disturbed, sites. Sites of unknown condition are then sampled and judged against this standard. Defining a set of reference sites may be arbitrary, either in terms of which sites are included or which criteria are used to define reference condition. For the MAIA study, we have seen that hand-picked reference sites did not necessarily match objective criteria for reference condition. This lesson learned in the Mid-Atlantic is relevant to states as they develop criteria for reference condition and rules for defining impairment. Currently, states vary both in the way they characterize reference condition and the way they define deviation from reference condition. Furthermore, the line of impairment ranges from the index value that corresponds to the 5th to the 95th percentile of reference sites (USEPA, 2002).
For the MAIA fish index, three methods were used to define reference sites that represented increasingly stringent criteria for reference condition in that more conditions had to be satisfied (McCormick et al., 2001). The least restrictive criteria were based on chemical criteria and RBP habitat measures; the moderately restrictive criteria also included measures based on watershed land use; and the most restrictive criteria included all these as well as Bryce et al.'s condition class. The value of the fish index that represented the 25th percentile for reference sites was selected as the line of impairment for each method. For the three sets of reference sites, the values of the fish index were very similar and their average was used to define reference condition for fish assemblages.
Patterns of index variability were similar across assemblage types
Statistical precision is an important feature of any monitoring tool because it determines the ability of an indicator to detect change should it occur. A highly variable indicator must show a large change in value before the change is statistically significant. Lack of sensitivity translates into an inability to sound an alarm that will protect resources from degradation. Statistical power analysis can be used to estimate the magnitude of change that an indicator can detect. Two commonly used statistical models for power analysis are the t-test and regression (Peterman, 1990; Carlisle and Clements, 1999; Fore et al., 2001). Given EMAP's focus on estimating trends through time, a regression model with index regressed against year is arguably the more relevant statistical model for power analysis (Stevens, 1994; Larsen et al., 2001; Hughes et al., 1998; Urquhart et al., 1998). Results from the two approaches indicate that the MAIA multimetric indexes had adequate precision to distinguish between two and five categories of biological condition and could detect between 1.5% and 2.5% change per year after five years of monitoring.
Both approaches for estimating statistical power to detect change use estimates of variance components derived from ANOVA. For power analysis based on a t-test to compare two sites with three replicates each, within-year variance is used to calculate the minimum detectable difference for index values at two sites (MDD; Zar, 1984). By dividing the range of the index by the MDD, one can calculate the number of categories of biological condition that the index can detect (Fore et al., 1994; Fore et al., 2001; Fore 2002a; Blocksom, 2003). The regression model uses the within-year variance as well, but also uses estimates of variance associated with site x year interaction and year-to-year variability to calculate statistical power (Larsen et al., 1995; Urquhart et al., 1998).
The relative magnitude of the variance components illustrates which temporal influences were relatively more important for each index. The percentage of total variance associated with repeat visits to the same site, that is, all sources of variance besides that associated with site differences were approximately similar across assemblages (13-20%). For fish and diatom assemblages, the multimetric index tended to have less variance associated with repeat visits than did its component metrics (Figure 6; McCormick and Peck, 2000; Fore, 2002b). USEPA guidelines for biological indicators recommend that the overall variability associated with repeat sampling within a single year, or the error component, not exceed 10% (McCormick and Peck, 2000). Only the fish index met this target with 6% of total variance associated with error; at 17%, the diatom index was furthest from the mark and the invertebrate index was close with 13% (Table 4).
Indexes for each assemblage differed in the way that the "nuisance" variance was allocated to each of the different sources, i.e., year-to-year differences, site x year interaction, and repeat sampling within year (measurement error). For the diatom index, most of its variance was associated with repeat visits to a site while year-to-year variance was nearly zero. In contrast, fish and invertebrate indexes did tend to change together across years (year-to-year variance). This difference may be related to longer life cycles for fish and invertebrates compared with diatoms.
Based on statistical power calculations for the t-test model, the fish index
was most precise and could detect approximately 5.5 categories of biological
condition, the invertebrate index could detect 4 categories, and the diatom
index 2.4. The diatom index had the largest percentage of its total variance
associated with error; therefore, it follows that it had the lowest statistical
power to detect differences.
Figure 6. Variance components for the invertebrate index and its metrics. Variance associated with repeat visits to the same site during the same year ("error") and from year to year ("year") was lower for the index than for its metrics. Approximately 17% of the index's variability was associated with repeat visits (see Table 3).
Table 4. Components of variance expressed as a percentage of the total variance for diatom, invertebrate, and fish multimetric indexes. Variance associated with site differences, year-to-year differences, site x year interaction, and repeat visits within years are shown for each index.
| Diatom | Invertebrate | Fish | |
| Site | 80.4 | 83.3 | 86.8 |
| Year | 0 | 2.1 | 1.5 |
| Site x year | 2.7 | 1.6 | 5.6 |
| Error (repeat visits) | 16.9 | 12.9 | 6.2 |
<< previous page
next page >>