Jump to main content.


Assessment Limitations

Limitations -- Features of the assessment that limit the kinds of conclusions that should be drawn result from:
  • gaps in data;
  • limitations in computer models used;
  • default assumptions used routinely in any risk assessment; and
  • limitations in the overall design of the assessment (intended to address some questions but not others).
The following are important specific limitations to recognize:

The results apply to geographic areas, not specific locations. The assessment focused on variation in air concentration, exposure and risk between geographic areas such as census tracts, counties and states. All questions asked, therefore, must focus on variations between county or larger geographic areas. They cannot be used to identify “hot spots” where the air concentration, exposure and/or risk might be significantly higher within a county. In addition, this kind of modeling assessment cannot address the kinds of questions an epidemiology study might, such as the relationship between asthma or cancer risk, and proximity of residences to point sources, roadways and other sources of air toxics emissions.

The results do not include impacts from sources in neighboring countries (i.e., Canada or Mexico). Since the assessment did not include the emissions of sources in Canada and Mexico the results for States which border either of these countries would not reflect these potentially significant sources of transported emissions.

The results apply to groups, not to specific individuals. Within a census tract, all individuals were assigned the same ambient air concentration, chosen to represent a typical ambient air concentration. Similarly, the exposure assessment used activity patterns that do not fully reflect variations between individuals. As a result, the exposures and risks in a census tract should be interpreted as being only typical values rather than as means, medians, etc. They are likely to be values in the midrange for the census tract, and so typical here means something like “in the midrange” of values for all individuals in the census tract.

The results are restricted to 1996. The assessment used emissions data from 1996. This was chosen for two reasons. First, it was the only available, reliable, data set on emissions that was sufficiently complete to allow the analysis. Second, it provides a baseline estimate of ambient air concentration, exposure and risk against which results of future assessments may be compared to determine the trend in air quality. Significant emission reductions have taken place since 1996: (i) mobile source regulations are being phased in over time, (ii) EPA has issued air toxics regulations for major industrial sources, (iii) there are State and industry initiatives, and (iv) some facilities may have closed.

The results do not reflect exposures and risk from all compounds. The assessment examined only 33 selected air toxics compounds. While these were chosen to represent the air toxics of most concern, the actual risks in a census tract may be higher due to the presence of air toxics compounds not considered in this assessment. It is particularly significant that the assessment did not quantify risks from diesel exhaust. This is because EPA does not have a unit risk estimate at this time (for more information, see the qualitative discussion on risk from diesel PM.)

The results do not reflect all pathways of exposure. The assessment included only risks from inhalation of the air toxics compounds. It did not consider air toxics compounds that might then deposit onto soil, water, food, etc, and therefore enter the body through ingestion or skin contact. Consideration of these other routes of exposure should have the effect of raising the exposure and risk.

The assessment results reflect only compounds released initially into the outdoor air. The assessment did not include exposure to air toxics compounds produced indoors, such as from stoves or out-gassing from building materials or evaporative benzene emissions from cars in attached garages. For some compounds such as formaldehyde, these indoor sources can contribute significantly to the total exposure for an individual, even if only inhalation exposures are considered. In addition, the assessment did not consider toxics released directly to water and soil.

The assessment does not fully reflect variation in background ambient air concentrations. The assessment uses background ambient air concentrations that are average values over broad geographic regions. Much more research is needed before an accurate estimate of background concentrations at the level of census tracts, or even at the higher geographic scales (counties, states, etc), can be made. Since background levels are significant contributors to the overall exposure in this assessment, the lack of detailed information on variations in background exposures probably causes the amount of variation in total exposure and risk between census tracts to be smaller than would otherwise be the case.

The assessment might systematically underestimate ambient air concentration for some compounds. The ASPEN model used to estimate ambient air concentration has been shown in this assessment to underestimate the measured concentration in many cases. This would tend to result in an underestimation of the exposure and risk. In any event, the effect of this issue is unknown at present.

The assessment used default, or simplifying, assumptions where data were missing or of poor quality. Data on some of the quantities used in the modeling for emissions and dispersion of air toxics compounds (such as stack height, facility location, etc) were not available or were flawed. When this happened, they were replaced by default assumptions. For example, a stack height for a facility might be set equal to stack heights at comparable facilities; the location of the facility might be placed at the center of a census tract; etc. This introduces uncertainty into the final predictions of ambient concentration, exposure and risk, as discussed in the section on Uncertainty.

The assessment may not accurately capture sources that have episodic emissions (e.g., wildfires and prescribed burning). The ASPEN model assumes emission rates are uniform throughout the year. Some sources have variable rates of emissions which occur within only a few days or weeks each year (episodic). For example, the emissions from prescribed fires, which typically last for about a week are averaged over the entire year.

Many of the cancer risk estimates have a built-in margin of safety. The parameter used to convert from exposure to cancer risk (i.e. the Unit Risk Estimate or URE) is based on default science policy processes used routinely in EPA assessments. First, some air toxics are known to be carcinogens in animals but lack data in humans. These have been assumed to be human carcinogens. Second, all the air toxics in this assessment were assumed to have linear relationships between exposure and the probability of cancer (i.e. effects at low exposures were extrapolated from higher, measurable, exposures by a straight line). Third, the URE used for some air toxics compounds represents a maximum likelihood estimate, which might be taken to mean the best scientific estimate. For other air toxics compounds, however, the URE used was an “upper bound” estimate, meaning that it probably leads to an overestimation of risk if it is incorrect. In addition, it has been assumed that this URE continues to apply even at the low exposures considered in this assessment. It is likely, however, that this linear model over-predicts the risk at exposures encountered in the environment. The cancer risk estimates produced in this assessment, therefore, should be considered “upper bound” in the science policy sense.

All of the noncancer risk estimates have a built-in margin of safety. All of the Reference Concentrations (RfCs) used in the assessment in estimating a Hazard Quotient (HQ) are conservative, meaning that they represent exposures which probably do not result in any health effects, with a margin of safety built into the RfC to account for sources of uncertainty and variability. A value of HQ greater than 1, therefore, should not necessarily be taken to indicate that a health effect is expected. The values of HQ produced by this assessment are, therefore, considered “upper bound” in the science policy sense.

Taking into account all of the above limitations, the results of the national-scale assessment can provide answers to the following kind of question:

How do ambient air concentrations, inhalation exposures and/or “upper-bound” estimates of risks from inhalation for a typical individual in a census tract as of 1996 vary between broad geographic areas for the 33 air toxics compounds considered when indoor sources are excluded?
It is important to keep this question in mind when interpreting all results, and to ensure that the results are not used to answer other questions for which they are not suited. In addition, these limitations prevent the EPA from using the results of the assessment to determine contributions from specific sources or to set regulatory requirements. They provide, instead, a basis for informing decisions about priorities in the air toxics program and guiding the collection of additional data that could lead eventually to regulatory decisions.

 

Download a printer friendly version of this page

Return to the Limitations, Variability, and Uncertainty Page

Frequent Questions
Glossary | Site Map


Local Navigation


Jump to main content.