EFED's Summary of the SAP Review of Ecological Risk Assessment Methodologies
April 14, 1997
On this Page
- Summary of SAP Recommendations
- Major Issues and Discussion
The FIFRA Scientific Advisory Panel (SAP) met for three days, May 29-31, 1996, to address several scientific issues regarding the Office of Pesticide Program's (OPP) ecological risk assessments and guidelines. One of the major topics explored with the Panel was OPP's ecological risk assessment methods and procedures. OPP requested the Panel to review and comment on how this office could improve these methods and procedures.
Two case studies were presented to the Panel, "The Ecological Risk Assessment of Carbofuran Flowable Products" and "The Corn Cluster Risk Assessment". OPP asked the Panel specifically to focus on the methodology used to assess the risk of the pesticides evaluated in these analyses and not to address the individual chemicals.
This paper provides a summary of the comments and major issues (Scientific Advisory Panel, 1996) identified by the SAP.
Summary of SAP Recommendations
The Panel in general seemed to recognize the value of our current assessment process, which can be viewed as a screening level assessment, and approved of the methodology. The Panel also provided several insightful and useful comments and recommendations on how to improve the methods and procedures used to help move beyond a screening level assessment. These comments are summarized below and provide the basis for the four overarching issues identified in Section III of this paper.
The Panel suggested that the assessments need to be expanded to encompass not only invertebrates, fish and birds, but mammals, reptiles, amphibians, and critical forage insects; all important parts of the food chain.
The Panel suggested that direct lethality is only one impact; pesticide impacts on aquatic and terrestrial organisms cannot be fully understood without accounting for sublethal and indirect effects, which can also significantly impact populations.
The Panel suggested that the exposure and effects methodologies used do not provide the depth of understanding of the chemical and biological mechanisms that a environmental risk assessment requires.
The Panel identified several steps in OPP's risk assessment process that involve extrapolations from laboratory data and suggested that the validity and scope of these extrapolations have not been adequately investigated and validated to date.
The Panel raised concerns that OPP, due to data gaps, is "trapped" in deterministic risk assessments and should move toward probabilistic types of assessments. (Currently, we say that the levels of concern (LOCs) have been exceeded. The Panel believes that we should move towards providing the following type of characterization: "60% of the time, 15% of the population of X species will be killed.")
The Panel suggested that the methodologies and specific endpoints used have a number of limitations, some severe, in relation to their utility in risk assessments. They also suggest that there are gaps in the methodologies used that must be filled to accomplish effective and comprehensive risk assessments.
The Panel suggested that more emphasis should be given to the dose-response slopes in characterizing the hazard of individual chemicals, between chemicals, and between formulations.
The Panel indicated that the Risk Quotient is a qualitative parameter and not a highly quantitative measure of ecological risk, particularly since exposure and environmental fate are not determined.
The Panel noted that the two assessments failed to address the significance of the fate properties and routes and means of transport of the pesticide in relation to exposure, particularly for terrestrial assessments.
The Panel noted that the four Corn Cluster pesticides were similar in fate and toxicity parameters. They raised the question on how dissimilar chemicals (e.g. different fate properties or less toxic) would influence the risk estimates.
The Panel suggested that establishing an estimated environmental concentration (EEC) based solely on application rate needed justification.
The Panel recommended that the differences in the toxicity and fate among formulations of chemicals and between chemicals needed to be considered to fully evaluate the risk of pesticides to non-target organisms. Relying on estimates of toxicity and exposure based on active ingredient (AI) values, which EFED assumes to mean application rate and/or type of formulation, may lead to erroneous conclusions and may need further testing.
The Panel believed that incident reports are not useful for quantitative comparisons in a weight of the evidence approach. They believed that incident reports were very important for establishing ecological concern for chemical use, but should be used mainly to justify further investigations tailored to specific chemical formulations and uses.
The Panel strongly recommended that OPP reinstate critical field testing and related mesocosm studies.
Major Issues and Discussion
There appear to be four major issues that arise from these comments and recommendations. They are discussed in more detail in sections A through D.
Overview of the 4 Issues
The Panel identified several areas in the assessments that could be expanded to present a more complete perspective of the potential environmental perturbations for the chemicals reviewed. They suggested a more thorough analysis could be developed if available environmental fate data were combined with effects information and the life histories of exposed species, especially for terrestrial organisms. They further identified additional data that would help refine the assessments.
Extrapolation and Validation
The Panel identified several steps in the risk assessment process that involve extrapolations. They suggested that the validity and scope of these extrapolations have not been adequately investigated to date.
The Panel suggested that the methodologies and specific endpoints used have several limitations, sometimes sever, in relation to their utility in risk assessments. They recommended that OPP develop the necessary databases and methodologies to conduct probabilistic assessments of risk.
Guideline Field Testing and Research
The Panel strongly recommended that OPP reinstate critical field testing and related mesocosm studies. They also strongly encouraged field research to validate the models currently used by EFED and those developed for future use.
While not specifically addressing the issue of risk characterization, a number of the issues the Panel raised appear to roughly fall under this heading. The Panel in several of their comments suggested that further use of the available data can be made to provide better insight into the potential environmental effects for the chemicals reviewed (Risk Characterization).
A prominent point of discussion appears to be the integration and use of environmental fate data in the selection of exposure levels. For terrestrial exposure, several questions addressed the validity of estimating exposure on the basis of AI without consideration of the formulation and environmental fate parameters. That is, degradation of the pesticide may have a significant impact on exposure. This is not currently factored in the calculation of environmental exposure.
For aquatic exposure determinations, point estimates for exposure are selected from a temporal distribution of values for use in the exposure portion of aquatic assessments. However, additional refinements are needed. For example, tools for spatial probabilistic exposure assessment have been under development in conjunction with the Exposure Modelling Work Group and the National Science Foundation.
The Panel stressed the need to incorporate and report the slopes of dose- response curves in the assessments, which are easily calculated from the results of the basic toxicity tests. They indicated that the shallowness or steepness of these curves can provide significant insight into the potential effects of a chemical and needs to be factored into the weight of the evidence.
The SAP further suggested that the assessments need to be expanded to more completely address important parts of the food chain such as reptiles, amphibians, and critical forage insects. They also raised a concern about mortality as the major input used in the assessment process. They stated that the cost of this convenience may be immense in establishing risks and embarking on mitigation, especially when the goal is to proceed from individual toxicity measurements to an understanding of population effects. Parameters such as biomarkers (e.g., cholinesterase, P450 levels, hormone levels, reproductive indices or birth abnormalities) should be used as endpoints.
The SAP implied that "rough probabilistic statements" (interpreted to mean risk characterization) could be developed based on current fate, exposure, and effects data, overcoming some of the problems inherent in assessments based solely on quotients calculated from point estimates. The Panel further implied, that even considering the shortcomings of the current data bases and conceptual assumptions, further elaboration and description of "risks" based on the potential environmental fate of the chemicals could be made in the assessments.
The Panel also suggested expanding the scope of the data requirements to include additional toxicity testing, life-stages, species and chemical formulations.
Extrapolation and Validation
The second major issue raised by the Panel concerns extrapolation, which has often been raised in the past from sources inside and outside our program. This needs to be addressed to reduce the uncertainty of results from the methods used to estimate effects.
Several steps in the assessment process involve extrapolations from laboratory data, both for effects and exposure, to the field. Other steps in the assessment process involve extrapolations of field effects data, field exposure data or exposure modeling results from one crop to another, one formulation to another, one application method to another, and one region of the country to another. The validity and scope of these extrapolations have not been adequately investigated to date. That is, there has been little real world data collected to validate the model predictions.
Further, if OPP is to move from the deterministic (hazard assessment) mode to probabilistic risk assessments, as encouraged by the SAP, the functional relationships between the above parameters need to be better defined as well as several other parameters currently given limited attention in evaluating potential impacts. These include duration of exposure, extrapolation from AI to formulated products, severity of effects, proportion of organisms affected, and population and ecosystem level considerations.
The third major issue identified by the Panel was that OPP, due to data gaps, is still "trapped" in the deterministic mode of assessing effects of pesticides to non-target organisms rather than providing probabilistic assessments for the chemicals of concern. The Panel tacitly suggested that the current test methodologies and specific endpoints used by OPP in its model assessments were designed to support the relative simplistic process of hazard assessment, not risk assessment. The Panel indicated that this approach has a number of limitations, and its utility in risk assessments is of questionable value. They also point out that gaps in the current methodologies must be filled to accomplish effective and comprehensive risk assessments.
The Panel's suggestion, that the limitations of the assessment methodologies used in the two examples are of questionable value in risk assessments, may need to be explored further. It appears that the Panel is using a more stringent definition of "risk assessment" than the Agency. The Agency defines ecological risk assessment as the process that evaluates the likelihood that adverse ecological effects may occur or are occurring as a result of exposure to one or more stressors. And the Agency defines "likelihood" to mean that risk may be expressed as a qualitative likelihood of effects or as quantitative probabilities. The Panel from their comments and discussion appears to use the more formal definition of risk assessment, which is:
A set of formal scientific methods for establishing the probabilities and magnitudes of undesired effects resulting from the release of chemicals. Risk assessment includes quantitative determination of both exposure and effects (Society of Environmental Toxicology and Chemistry, 1987).
While this distinction is not explicitly stated by the Panel, it helps to explain a number of the Panel's comments and particularly a number of the implications of the Panel's comments.
OPP's pesticide assessment process, referred to as a "risk assessment, " calculates a quotient by comparing the toxicological endpoint of interest (normally the LD50 or LC50 of the most sensitive species) to an estimate of the exposure concentration (normally without accounting for degradation) while factoring in available field data and incident reports. When asked how can these "risk assessment" methods be improved, and given that the methods do not establish the probabilities or magnitudes of effects as the formal definition states, the response that the assessments do " . . . not provide a powerful enough analysis to estimate overall risk" is understandable. In fact the Panel's comments may be somewhat polite in that the assessments do not address risk as defined in the more formal definition.
Also worthy of note, the Panel seems to have been under the impression that OPP adopted the assessment approach outlined and discussed at a previous joint Science Advisory Board (SAB) and SAP meeting in August of 1995. At this Joint SAB and SAP meeting, OPP presented the report titled, "Aquatic Dialogue Group: Pesticide Risk and Mitigation". This report addresses the procedures for conducting risk assessments.
The procedures outlined in this report are presented as probabilistic based. While this latter point may warrant discussion, the impression of the Panel that OPP had adopted this approach may have added to the ambiguity of the definitions of "risk". Unfortunately, this report was not mentioned until after the discussions between the Panel and OPP had ended and was referred to during the public comment period by industry representatives. Therefore, OPP did not have an opportunity to clarify the status of the report at the meeting. OPP was not aware the Panel was under the belief that this approach had been adopted until the written comments were received.
While the Panel makes several other comments that appear to support this interpretation of their definition of "risk", their recommendation is for OPP to move from the deterministic mode to probabilistic assessment. The Panel acknowledged that available data are inadequate to support this approach, but is silent on the amount and type of data needed for probabilistic assessments.
Guideline Field Testing and Research
The fourth major issue raised by the Panel was field studies. The Panel addressed two issues, which are
the reinstatement of field studies to support registration and reregistration, and
field studies that need to be designed to validate the assessment models that are used.
(This latter point overlaps with their comments on extrapolation and validation.)
The Panel emphasizes that for pesticides used on corn and other field crops, work is needed to clarify exposure, validate toxicities, and establish site- specific and bench mark data for species critical to the regulatory decision-making process. Therefore, they concluded,
. . . we strongly urge the Agency to reinstate critical field testing and related mesocosm studies for chemicals to monitor chemical use, test paradigms and modeling algorithms and to obtain base line data on the interactions between chemicals, habitats and toxicities to birds, small mammals, aquatic organisms and other appropriate wildlife.
The recommendation to reinstate field tests to monitor chemical use addresses the need for field studies to support registration and reregistration. While, the latter part of the recommendation, to reinstate field studies to test paradigms and modeling algorithms and obtain baseline information, addresses the validation issue.
In reference to the validation issue, the Panel implied that the uncertainties associated with OPP's present methods must be considered if improvements are to be made in assessment methods. There is a great need to better understand the functional relationship between the tools used to estimate effects, toxicity and exposure estimates, and actual effects under field conditions. These relationships most likely would be the foundation for any model, deterministic or probabilistic. Therefore, if advances are to be made, they need to be better defined. In the absence of this research, which the Panel suggested is long over due, the questions associated with present methodologies will persist even if more sophisticated methodologies are developed.
EFED agrees with many of the issues and recommendations made by the SAP. They have raised meaningful questions about our current risk assessment methodologies and the need for validation. EFED is looking forward to addressing these issues and recommendations.
Scientific Advisory Panel (1996) A Set of Scientific Issues Being Considered by the Agency in Connection with EPA's Ecological Risk Assessment of Carbofuran Flowable Products and with EPA's Corn Cluster Ecological Risk Assessment. Review conducted in an open meeting held on May 29 - 31. Pages 37 - 49.
Society of Environmental Toxicology and Chemistry (1987) Workshop Report. Research Priorities in Environmental Risk Assessment. August 16 - 21, 1987.