Evaluation Guidelines for Ecological Toxicity Data in the Open Literature

On this page:
Attachments

Memorandum

May 16, 2011

SUBJECT: Evaluation Guidelines for Ecological Toxicity Data in the Open Literature

FROM: /s/ Donald J. Brady, Ph.D., Director, Environmental Fate and Effects Division, Office of Pesticide Programs

TO: All Managers and Staff of the Environmental Fate and Effects Division

Ecological effects data for pesticides are provided by the registrants as part of the 40 CFR Part 158 guideline requirements. In addition to the information furnished by the registrants, ecological effects data contained within the open literature are considered in ecological risk assessments conducted in the Office of Pesticide Programs. Based on an agreement with the U.S. Fish and Wildlife Service and National Marine Fisheries Service, the Office of Pesticide Programs uses the U.S. Environmental Protection Agency's Ecotoxicity Database (ECOTOX) as its search engine to obtain relevant data on the ecotoxicological effects of pesticides.

The purpose of this document, which is effective immediately, is to replace the interim guidance entitled "Evaluation Criteria for Ecological Toxicity Data in the Open Literature", which was prepared and issued by the Environmental Fate and Effects Division (EFED) in July 2004. Thus, this OPP-wide document supersedes the 2004 guidance. It provides updates as well as additional information and clarification to assist in the screening, reviewing, and incorporating the available open literature from ECOTOX.

This guidance was developed by the Endangered Species Registration Review Workgroup. For further information, please contact Anita Pease or other members of the Workgroup.

Endangered Species Registration Review Workgroup

David Bays, AD
Shannon Borges, BPPD
James Breithaupt, AD
Mark Corbin, EFED
Kevin Costello, PRD
William Eckel, EFED
Catherine Eiden, PRD
Melissa Grable, EFED
Mark J. Huff, EFED
Stephanie Irene, EFED
Russell Jones, BPPD
Stephen Morrill, BPPD
Edward Odenkirchen, EFED (Co-Chair)
Melissa Panger, EFED
Anita Pease, EFED
Mohammed Ruhman, EFED
Dana Spatz, EFED
Katrina White, EFED

Evaluation Guidelines for Ecological Toxicity Data in the Open Literature

May 9, 2011

PROCEDURES FOR SCREENING, REVIEWING, AND USING PUBLISHED OPEN LITERATURE TOXICITY DATA IN ECOLOGICAL RISK ASSESSMENTS

  1. Introduction

    • 1.1 Purpose

      Ecological effects data for pesticides are provided by the registrants as part of the 40 CFR Part 158 guideline requirements. In addition to the information furnished by the registrants, ecological effects data contained within the open literature are considered in ecological risk assessments conducted in the Office of Pesticide Programs (OPP). Based on an agreement with the U.S. Fish and Wildlife Service and National Marine Fisheries Service (collectively referred to as the "Services"), OPP uses the Environmental Protection Agency's (EPA) Ecotoxicity Database1 (ECOTOX) as its search engine to obtain relevant data on the ecotoxicological effects of pesticides.

      The purpose of this document is to provide further information and clarification to assist in screening, reviewing, and incorporating the available open literature from ECOTOX. Although parts of this guidance document are specific for OPP's Environmental Fate and Effects Division (EFED), the overall concepts and principles relative to consideration of open literature studies to support ecological risk assessments are applicable across OPP. This generic guidance was developed to assist OPP scientists and is intended for use in OPP's problem formulations and ecological risk assessments completed as part of the Registration Review and in support of endangered species litigation. Although this guidance document may also be used to screen and evaluate open literature for ecological risk assessments in support of new uses (Section 3), special local needs (Section 24c), and emergency exemption (Section 18) petitions, it should be noted that ECOTOX searches including associated data tables generated by EPA's Office of Research and Development's Mid-Continental Ecology Division (ORD/MED) are currently available only for chemicals being assessed as part of the Registration Review process and/or endangered species litigation. For non-Registration Review and non-litigation-related assessments, risk assessors are encouraged to query the public version of ECOTOX and screen/review the data based on similar guidelines provided for ECOTOX searches conducted by ORD/MED. In general, ECOTOX searches for chemicals being evaluated as part of Registration Review and/or endangered species litigation should be available to the risk assessment teams approximately 6 months prior to the assessment due date, depending on the complexity of the chemical and the time period between previous ECOTOX searches, if any.

      1 ECOTOX is a publicly available database summarizing the ecological effects of single chemicals to aquatic and terrestrial plants and animals ( http://cfpub.epa.gov/ecotox/). ECOTOX was developed by EPA's Office of Research and Development's Mid-Continental Ecology Division (ORD/MED), which routinely conducts literature searches for pesticides undergoing Registration Review as well as for litigation-related endangered species assessments.

    • 1.2 Background

      In July of 2004, an Interim Guidance, entitled "Evaluation Criteria for Ecological Toxicity Data in the Open Literature (Interim Guidance)" was completed by the Environmental Fate and Effects Division (EFED). The interim document provides procedures for identifying, selecting, and acquiring toxicity effects data published in the open literature (USEPA, 2004a). The Interim Guidance document is divided into two phases. Phase I of the Interim Guidance presents the ECOTOX and OPP acceptance criteria, as well as the criteria for those papers that are excluded or rejected from the ECOTOX database. Phase II of the Interim Guidance provides information on the categorization of studies and determination of their use in risk assessment. In addition, the Interim Guidance provides a summary of the roles and responsibilities of ORD/MED, EFED's Environmental Information Services Branch (EISB), and the risk assessor.

      Although the information provided in the 2004 Interim Guidance has been useful in providing methods for selection of appropriate open literature studies, there are inconsistencies among OPP risk assessors regarding the use of open literature data to address an existing data gap, the classification of open literature studies, and the use of these studies in an ecological risk assessment. In addition, documentation of reviewed open literature (i. e., open literature review summary or OLRS) is oftentimes not completed and rarely sent to EISB for tracking on the Storage Area Network (SAN) drive.

      This OPP-wide document is intended to supersede the EFED Interim Guidance by including all of the open literature acceptance criteria for effects data (Phase I of the Interim Guidance) as Attachment I and updating Phase II of the Interim Guidance by providing further information and clarification on how to categorize and incorporate studies from the open literature into problem formulations and ecological risk assessments completed by OPP. This updated guidance document is not limited to EFED, but is intended to ensure consistent consideration, use, and documentation of information in the open literature by OPP risk assessors in evaluating the potential pesticide effects on listed (threatened and endangered) and non-listed non-target organisms. Ultimately, the utility of open literature studies is in large part determined through the best professional judgment of the reviewer and cannot be completely prescribed in any guidance document.

      The flowcharts depicted in Figures 1 and 2 summarize the screening, review, and documentation process for open literature studies from the ORD/MED ECOTOX search (Figure 1) and other submitted non-guideline open literature studies that are cited in the OPP Information Network (OPPIN) bibliography (Figure 2).

      ESA Open Literature Figure 1
      Figure 1 - Screening, Review, and Documentation Process for ECOTOX Open Literature
      ESA Open Literature Figure 2
      Figure 2 - Screening, Review, and Documentation Process for Other Submitted Non-Guideline Open Literature Data Cited in OPPIN

    • 1.3 Organization of the Document

      This guidance document is divided into the following three sections:

  2. Screening the ECOTOX Open Literature

    In addition to data generated from guideline studies submitted by the registrant, ecological risk assessors in OPP consider data from the open literature that ORD/MED locates using the search strategy discussed in Attachment I. Studies located and coded into ECOTOX must meet acceptability criteria, as described below and in Attachment I. The intent of the acceptability criteria is to ensure data quality and verifiability. As part of this search strategy, potentially useful open literature papers and respective citations are sorted into one of the four following categories:

    1. Papers that are accepted by ECOTOX and OPP;

    2. Papers that are accepted by ECOTOX, but not accepted by OPP;

    3. Papers that are rejected by ECOTOX and OPP; and

    4. "Other" papers.

    Attachment I presents the ECOTOX and OPP acceptance criteria, as well as the criteria for those papers that are excluded or rejected from the ECOTOX database.

    The purpose of this section of the guidance document is to discuss the screening process used to identify potentially useful open literature papers that fall within these four categories and provide directions on how to obtain and/or order potentially useful papers. The screening criteria for accepted papers described below in Section 2.1 are taken from the 2004 Interim Guidance; however, the additional information on endpoint refinement for under-represented taxa, rejected papers, consideration of papers in the "Other" category, and instructions for obtaining and ordering relevant papers is new in this guidance document.

    • 2.1 Accepted Papers by ECOTOX and OPP

      In order to be accepted and included in the ECOTOX database, papers must meet the following minimum criteria:

      1. The toxic effects are related to single chemical exposure;

      2. The toxic effects are on an aquatic or terrestrial plant or animal species;

      3. There is a biological effect on live, whole organisms;

      4. A concurrent environmental chemical concentration/dose or application rate is reported; and

      5. There is an explicit duration of exposure.

      In addition to the criteria listed above, the following criteria, which are discussed in further detail in Attachment I, are applied by OPP as a further screen of acceptability:

      • 6. Toxicology information is reported for a chemical of concern to OPP;

      • 7. The article is published in the English language;

      • 8. The study is presented as a full article;

      • 9. The paper is a publicly available document;

      • 10. The paper is the primary source of the data

      • 11. A calculated endpoint is reported;

      • 12. Treatment(s) are compared to an acceptable control;

      • 13. The location of the study (e.g., laboratory vs. field) is reported; and

      • 14. The tested species is reported and verified.

      Data from open literature papers that pass the screens of acceptability by ECOTOX and OPP are available to risk assessors in the form of a Microsoft® Excel spreadsheet and in three types of summary tables generated by the Microsoft® Access database.

      The Excel spreadsheet and summary tables provide all of the accepted data and may include multiple endpoints or rows of information from the same paper. The information contained in the Excel spreadsheet can be sorted by any of the relevant attribute columns-(i.e., taxa, genus, endpoint, concentration/dose, study duration, etc.) contained within the spreadsheet. It should be noted that all updates or "refreshes" to the Excel spreadsheet will include both the updated data as well as any older searches; updated information can be found in separate worksheets that are named according to the date of the "refresh". The risk assessor should ensure that all previously dated ECOTOX searches have been reviewed by consulting the most recent ecological risk assessment where open literature data have been considered and checking the date of the latest ECOTOX search provided in that assessment. Further guidance on how to sort the data and format the Excel spreadsheet table for inclusion as an appendix to the Registration Review risk assessment is provided in Attachment 2 of this document.

      If requested, EISB will provide three types of summary tables to the risk assessor. The three types of Microsoft ® Access database summary tables include the following:

      • Table 1: Studies with Concentrations and Doses Similar to Registrant-Submitted Studies

      • Table 2: Studies on Underrepresented Taxa

      • Table 3: Studies on Sublethal Effects

      Depending on the preference of the risk assessor, either the Excel spreadsheet and/or associated Access database summary tables may be used to screen and obtain papers for consideration and use. Questions regarding the Microsoft® Access database summary tables should be directed to EISB staff.

      The purpose of screening the accepted papers is to quickly identify toxic responses or endpoints that are more sensitive than those reported in the registrant-submitted studies and/or capture underrepresented receptors and sublethal endpoints that are not contained in current EPA guideline studies. In addition, a new endpoint refinement strategy has been added, whereby the risk assessor is directed to search the open literature for less sensitive endpoints for under-represented taxa including aquatic- and terrestrial-phase amphibians, freshwater mollusks, and reptiles. Further discussion of the endpoint refinement strategy for under-represented taxa is provided in Section 2.1.2. In addition, for studies in which the experimental design includes exposure to both single chemicals and mixtures, it is possible that data on mixtures may be available in accepted studies. In order to determine whether an accepted ECOTOX paper may contain additional data on mixtures, the risk assessor must search the codes provided in the bibliography of accepted papers by ECOTOX for the term "MIXTURE". Additional studies on mixtures that do not contain data on exposure to single chemicals may also be identified in the list of ECOTOX rejected studies (see Section 2.2 for further discussion). All papers that are coded with the term "MIXTURE" should be identified as part of the screening process and considered in the risk assessment.

      All papers identified as potentially relevant should be obtained or ordered, as discussed in Section 2.4. Although this document provides additional guidance on factors to consider when selecting open literature papers for review, it is expected that ecological risk assessors will use best professional judgment and provide rationale for deciding which papers to review for potential inclusion in the risk assessment. Further discussion of the review and use of open literature in risk assessments is provided in Sections 3 and 4, respectively.

      • 2.1.1 Consideration of Most Sensitive Endpoint

        Open literature studies that are comparable to guideline studies and report more toxic or lower endpoints than the registrant-submitted studies should be identified from the Excel spreadsheet and/or the Table 1 summary tables. Attachment 3 provides a blank table of the lowest acute and chronic toxicity values from registrant-submitted guideline studies, which should be completed by the risk assessor prior to identifying more sensitive endpoints from the open literature. This table may also be used to identify potential data gaps in the available registrant-submitted studies.

      • 2.1.2 Consideration of Under-represented Taxa

        Taxonomic groups not typically tested in required guideline studies include, but are not limited to, the following: aquatic-phase and terrestrial-phase amphibians, freshwater mussels, reptiles, mammals other than rodents, terrestrial invertebrates other than honey bees, and terrestrial plants not typically tested in the guideline seedling emergence and/or vegetative vigor studies (i.e., ferns, conifers, and other woody vegetation). Open literature studies that contain potentially useful toxicity data or measurement endpoints on taxonomic groups that are not specifically tested in registrant-submitted studies should be identified from the Excel spreadsheet and/or the Table 2 summaries.

        For the purposes of assessing risk to listed species, it is recognized that further endpoint refinement, beyond those specified in Attachment 3, is needed in order to address direct effects for listed species and to focus mitigation options on those species and pesticide use areas of concern. Based on the June 17, 2010 EFED Chemical Review Process (CRP) meeting, it was decided that refinement of ecological effects endpoints should occur at the preliminary risk assessment stage of Registration Review. Therefore, an endpoint refinement strategy was developed in order to gather open literature data for certain under-represented taxa for which large numbers of listed species exist and sensitivity to pesticides may be under- or overestimated based on the use of data from surrogate taxa. The taxonomic classes selected for endpoint refinement include aquatic-phase amphibians, freshwater invertebrate mollusks (i.e., bivalves and snails), reptiles, and terrestrial-phase amphibians. Amphibian, freshwater mollusk, and reptile classes were chosen for refinement because open literature data with routes of exposure comparable to guideline studies may exist for these classes, and there are large numbers of listed species within each class (i.e., 25 species for amphibians, 105 species for freshwater mollusks, and 40 species for reptiles). In order to ensure that the best available information is used to determine whether direct effects to listed species within these taxonomic classes may occur, both more and less sensitive endpoints, as compared to the registrant-submitted data for surrogate species, should be considered for aquatic-phase amphibians, freshwater invertebrate mollusks (i.e., bivalves and snails), reptiles, and terrestrial-phase amphibians.

        Listed species classes including freshwater and estuarine/marine fish, non-molluscan freshwater invertebrates, estuarine invertebrates, aquatic plants, birds, mammals, terrestrial invertebrates and terrestrial plants are not included in the endpoint refinement strategy because the combination of available data from guideline studies and more sensitive endpoints from the open literature are assumed to provide the best available and most protective endpoint data for listed species that are categorized within these classes. However, it is up to the discretion of the risk assessor to further refine endpoints for these other taxa, based on the best available information, and the type of assessment being completed. For example, if only one listed species of beetle is being evaluated as part of a litigation-related assessment for an insecticide, and acceptable data for the order, Coleoptera, are available, and show less sensitivity to the insecticide relative to guideline data for honey bees (order = Hymenoptera), it may be appropriate to use the Coleoptera endpoint as a surrogate for direct effects to the listed beetle. In general, however, it is not recommended that endpoints be refined for these additional taxa for Registration Review assessments, given the diversity and number of taxonomic orders and classes of listed species being assessed.

        With respect to those open literature papers that are identified as less sensitive than surrogate taxa for amphibians, freshwater mollusks, and reptiles, it may not be necessary to review all studies with endpoints that are less sensitive. Assuming that no "more sensitive endpoints" are identified for quantitative use, the risk assessor should start with the open literature study with the lowest of the "less sensitive" endpoints and work upwards (in order of ascending endpoints) until a "quantitative" endpoint (described further in Section 3.2.2) is identified. Once a "quantitative" endpoint is identified, reviews of the remaining "less sensitive" studies for amphibians, freshwater mollusks, and/or reptiles are not necessary, unless the risk assessor intends to use the full range of data (from least sensitive to most sensitive) in a species sensitivity distribution. All open literature data from under-represented taxa that are used to derive risk quotients and/or are used in species sensitivity distributions as part of the risk assessment refinement must be reviewed and determined to be scientifically valid for quantitative use as described further in Section 3.

        Table 1 provides a summary of the open literature search methodology and applicable columns of the Excel spreadsheet to locate papers that may contain additional endpoints for aquatic- and terrestrial-phase amphibians, freshwater mollusks, and reptiles.

        Table 1
        Refined Endpoint Open Literature Searches for Registration Review Assessments
        Added Endpoints 1,3 for Direct Effects Refinement Method for Searching Open Literature 2 Applicable Excel Spreadsheet Columns Surrogate value from registrant-submitted data 3
        Aquatic-phase amphibians Search for papers containing endpoints for freshwater aquatic-phase amphibians that are more or less sensitive than lowest registrant-submitted study for freshwater fish Phylum = Chordata;
        Class = Amphibia;
        Habitat = Aquatic;
        Media = FW        		 Lowest 96-h LC50 and NOAEC from freshwater fish data
        Freshwater invertebrate mollusks (bivalves and snails) Search for papers containing endpoints for freshwater invertebrate mollusks that are more or less sensitive than lowest registrant-submitted study for freshwater invertebrates Phylum = Mollusca;
        Class = Bivalvia (bivalves) and Gastropoda (snails);
        Habitat = Aquatic;
        Media = FW        		 Lowest 48-h EC50 and NOAEC from freshwater invertebrate data
        Reptiles Search for papers containing endpoints for reptiles that are more or less sensitive than lowest registrant-submitted study for birds Phylum = Chordata;
        Class = Reptilia;
        Habitat = Terrestrial and aquatic (sea turtles);
        Media (for aquatic) = FW and SW        		 Lowest LC/LD50 and NOAEC from avian data
        Terrestrial-phase amphibians Search for papers containing endpoints for terrestrial-phase amphibians that are more or less sensitive than lowest registrant-submitted study for birds Phylum = Chordata;
        Class = Amphibia;
        Habitat = Terrestrial        		 Lowest LC/LD50 and NOAEC from avian data

        1 Both acute and chronic endpoints in units comparable to surrogate values from registrant-submitted data should be reported.

        2 When considering data from the open literature, the risk assessor should be mindful that the route of exposure tested in the open literature study is comparable to that tested in the guideline study (i.e., direct contact, dietary, etc.).

        3 In the event that no additional data are available in the open literature, acute and/or chronic endpoints are based on registrant-submitted data for surrogate taxa including freshwater fish for aquatic-phase amphibians, freshwater invertebrates for freshwater mollusks, and birds for reptiles and terrestrial-phase amphibians.

      • 2.1.3 Consideration of Sublethal Effects

        Open literature studies that contain test results for sublethal endpoints that report more sensitive or lower endpoints than registrant-submitted guideline studies should be identified from the Excel spreadsheet and/or the Table 3 summaries. Examples of sublethal data from the open literature include, but are not limited to, growth, reproductive, hormonal, biochemical, cellular, osmoregulatory, and behavioral (i.e., loss of equilibrium) endpoints.

    • 2.2 Rejected or "Excluded" Papers by either ECOTOX and/or OPP

      Open literature papers that are rejected by either ECOTOX and/or OPP, based on the criteria summarized in Section 2.1, are not included in the Excel spreadsheet or the Summary Tables 1, 2, and 3, discussed above; however, separate bibliographies of citations for these papers will be provided to OPP risk assessors. Studies that do not meet the criteria for ECOTOX are designated in the bibliography as "Excluded by ECOTOX"; studies that meet the ECOTOX screen but do not meet the conditions established by OPP are designated in the bibliography as "Accepted by ECOTOX but not OPP". Both bibliographies of papers rejected or excluded by ECOTOX and/or OPP contain rejection codes, which may be cited to explain exclusion of the paper from consideration. The list of excluded papers should be reviewed by the risk assessor to determine whether the study may include information useful to the risk assessment. For example, a number of the ECOTOX-rejected studies may include data on both environmental mixtures as well as multiple-a.i. product mixtures. All papers that are coded with the rejection code "MIXTURE" should be identified as part of the screening process and considered in the risk assessment. (Further discussion of the review of open literature papers containing data on environmental mixtures and/or multiple a.i. product mixtures is provided in Section 3.3.) Other papers included in the ECOTOX-rejected category that may be useful to the risk assessor include studies with data related to modeling, monitoring, incident reports, and review articles. Although a potentially useful source of ecotoxicological data, review articles that do not represent the primary source of data should not be screened, categorized, and reviewed as open literature.

      As part of the Registration Review and endangered species litigation risk assessments, risk assessors must provide citations with corresponding rejection codes for all ECOTOX references not considered as part of the assessment. This bibliography of references not considered in the assessment should be provided in an appendix and include the appropriate list of citations falling under the headings "Excluded by ECOTOX" and "Accepted by ECOTOX but not OPP", along with their respective rejection codes. An explanation of the OPP acceptability criteria and rejection codes for ECOTOX should be provided as a preface to the bibliography of references not considered. Attachment 4 provides an example of the type of language that should precede the list of papers not considered. Additionally, Attachment I-C includes a list of ECOTOX rejection codes.

    • 2.3 Consideration of Papers in the "Other" Category

      In addition to receiving lists of citations for "acceptable" (i.e., acceptable for ECOTOX and OPP), "not acceptable" (i.e., acceptable for ECOTOX but not OPP), and "rejected" or "excluded" (i.e., not applicable for ECOTOX or OPP) papers, a fourth file of citations called "Other" is provided to the risk assessor as part of the ORD/MED ECOTOX search. The "Other" category includes a list of citations for papers that fit into one of the following four categories:

      1. Target: toxicity of chemical on intended pest including efficacy studies;

      2. On Order: potentially acceptable but publication has not been received;

      3. To Code: applicable but not coded; and

      4. To Skim: potentially acceptable but not evaluated.

      Papers included in the "Other" category are not coded into ECOTOX and do not appear in the spreadsheet or associated Access database summary tables of "accepted" data. Depending on the chemical, citations for "target" data are routinely included, whereas citations from the other three categories of "on order", "to code", and "to skim" are encountered less frequently. In an effort to address the omission of information included in the "Other" category, all citations listed in the ECOTOX file name "Other" should be included in the appendix of ECOTOX papers not considered. In addition, the language provided below should be added to the effects section of the risk assessment, following the description of ECOTOX screening criteria, as appropriate for herbicides/defoliants/plant growth regulators or insecticides/fumigants/insect growth regulators.

      Suggested language for Herbicides/Defoliants/Plant Growth Regulators

      "Open literature toxicity data for "target" terrestrial plant species, which include efficacy studies, are not currently considered in deriving the most sensitive endpoint for terrestrial plants. Efficacy studies do not typically provide endpoint values that are useful for risk assessment (e.g., NOAEC, EC50, etc.), but rather are intended to identify a dose that maximizes a particular effect (e.g., EC100). Therefore, efficacy data and non-efficacy toxicological target data are not included in the ECOTOX open literature summary table provided in Appendix X [insert Appendix where ECOTOX Excel spreadsheet is provided]. The list of citations including toxicological and/or efficacy data on target plant species not considered in this assessment is provided in Appendix XX [include all citations listed under "Target Species" in the ECOTOX file name "Other" in the appendix of ECOTOX papers not considered]."

      Suggested language for Insecticides/Fumigants/Insect Growth Regulators

      [The risk assessor should provide a summary of the ECOTOX open literature data for bees, non-insect invertebrates (i.e., soil arthropods, worms, etc.) butterflies, and beetles, which are available for all litigation and registration review chemicals].

      "Open literature toxicity data for other "target" insect species (not including bees, butterflies, beetles, and non-insect invertebrates including soil arthropods and worms), which include efficacy studies, are not currently considered in deriving the most sensitive endpoint for terrestrial insects. Efficacy studies do not typically provide endpoint values that are useful for risk assessment (e.g., NOAEC, EC50, etc.), but rather are intended to identify a dose that maximizes a particular effect (e.g., EC100). Therefore, efficacy data and non-efficacy toxicological target insect data are not included in the ECOTOX open literature summary table provided in Appendix X [insert Appendix where ECOTOX Excel spreadsheet is provided]. For the purposes of this assessment, "target" insect species are defined as all terrestrial insects with the exception of bees, butterflies, beetles, and non-insect invertebrates (i.e., soil arthropods, worms, etc.) which are included in the ECOTOX data presented in Appendix X. The list of citations including toxicological and/or efficacy data on target insect species not considered in this assessment is provided in Appendix XX [include all citations listed under "Target Species" in the ECOTOX file name "Other" in the appendix of ECOTOX papers not considered]."

    • 2.4 Obtaining and Ordering Relevant Papers

      ECOTOX reports of accepted papers generated by ORD/MED include an Excel summary spreadsheet of the data and an associated bibliography that identifies the ECOTOX or "E" number and the author/title of the paper containing ecological endpoints for the chemical under review.

      Once the risk assessor has determined that the open literature paper should be reviewed, based on the screening criteria discussed in Section 2.1, image copies of these papers may be available from the following three different sources in OPP:

      1. the EFED SAN drive under ECOTOX Retrieved Articles and Reviews;

      2. the EFED Library Scanned Articles on the SAN drive; or

      3. through Documentum if the study has been previously submitted to the Agency and assigned an MRID number.

      Otherwise, the paper can be ordered through ORD/MED as discussed in Section 2.4.4.

      • 2.4.1 SAN Drive

        As a first step, the risk assessor should attempt to locate papers of interest from the SAN drive. For example, the EFED SAN drive is located in the "H:/EFEDADMINSCANS on Documentum-Test-DCOPPDCOC03 (W2032pccth016)" folder labeled "ECOTOX Retrieved Articles and Reviews". This folder contains thousands of ".pdf" file copies of ECOTOX articles that EFED has retrieved from previous search requests. All ".pdf" files are named by the ECOTOX reference number preceded by an "E". ECOTOX reference numbers less than six digits are preceded by zeros to allow them to line up numerically on the SAN drive.

        Again, in EFED, all completed OLRSs are stored in the "EFED ECOTOX Reviews" folder on the EFED SAN drive. The electronic files for completed reviews are named by the ECOTOX reference number preceded by the letter "E". For reviews of papers that contain data on more than one chemical, the file name also includes the PC Code of the chemical for which the review was written. Further description of file naming for OLRSs is provided in Section 3.3.

      • 2.4.2 Library Scanned Articles

        If the papers cannot be located in the ECOTOX retrievals on the EFED SAN drive, an alternative source of papers can be accessed via the "Information Resources" folder on the G: Drive by opening the EFED Library Database and searching by the title or author of the document using the Microsoft® Access database search tools. There are two main tables in the database; Articles and Books. The Articles table should be searched first, followed by the Books table, as appropriate. This type of search can be accomplished by placing the cursor in the title field and then using the A-Z tool in the toolbar, which will alphabetically organize all titles and/or the author, as appropriate. Scroll down to the title and/or author of interest, and look to the left for the J-Number, or in some cases, the MRID, of the title/author of interest. Many article titles contained in ECOTOX have been previously obtained or provided to the Agency and may already be scanned and contained in our Library collection making them available immediately.

        If the title/author of interest has a corresponding "J number" in the Articles table, the article may be obtained from the SAN drive under the "EFED Library-Scanned Articles and Publications" folder. If the title/author of interest is located in the Books table, the entry should also include a shelf location in the EFED library located in the EFED File Room on the 12th Floor of Potomac Yard One South. The EFED library also contains collections of a number of journals frequently cited by ECOTOX. These include, but are not limited to, the Journal of Environmental Toxicology and Chemistry (1982-2008), Environmental Science and Technology (1992-2003) and the Journal of Mammology (1962-2004). It should be noted that journals frequently cited by ECOTOX are not always acceptable and should be reviewed similar to other open literature. Any ECOTOX citation listing an EPA Publication number or Department of Interior technical paper or scientific report number may be available in the Federal publications area of the EFED Library shelves. EISB library staff are available to locate publications as needed. Also, journal articles may also be available through the EPA Desktop Library using the EPA IntraNet (http://intranet.epa.gov/desktop/science.htm).

      • 2.4.3 Documentum

        A third possible source of ECOTOX open literature papers is the OPP Document Center via Documentum. To access Documentum, click on the desktop icon entitled "OPP Documentum Login" and type your normal windows login name and password.

        In order to determine if the paper is available in Documentum, the risk assessor can use the OPPIN database query tool by searching on the author to determine if the ECOTOX article title has previously been submitted by registrants or received by the Agency and assigned a Master Record Identification (MRID) number. Images of titles with MRIDs may be retrieved instantly through Documentum or by request from the Document Center. To order a paper from the OPP Document Center, go to the OPP@Work website (http://intranet.epa.gov/pesticides/), click on "Technology and Information Services", then "Information Center", then "Request Documents from the Information Center", fill out the form, and press the "submit" icon.

        If the ECOTOX article has been assigned an MRID number, it may have been previously reviewed by EFED, and an open literature review summary may exist in the EFED files under the "EFED ECOTOX Reviews" folder. If previously reviewed by OPP as a Data Evaluation Record (DER), the file would be located in the chemical files, by PC code, in the SAN drive. Conversely, an MRID number may be assigned to an open literature study with no associated review or DER. Open literature studies that have been formally submitted to the Agency must be reviewed as though they had been captured by ECOTOX, and the same filters used by ECOTOX for screening purposes should be applied to these additional studies (see Figure 2 and Section 4.2 for further description). As such, if a registrant-submitted open literature study is not consistent with the guidelines identified in this document, then the study would not be considered for use in the risk assessment.

      • 2.4.4 Requesting ECOTOX Open Literature Papers

        If the desired papers are not contained in-house, a list of ECOTOX numbers for needed papers should be sent by the reviewer to the EISB Contract Representative, who will in turn request that they be provided by ORD/MED. The turnaround time for this can vary from one day (if articles are already imaged by ORD/MED) to about two weeks. EISB will forward any articles received from ORD/MED directly to the ERB scientist, and ensure that all articles are scanned and stored in the SAN drive for future reference.

  3. Reviewing the Open Literature

    All open literature papers that are identified as potentially useful based on the screening process discussed in Section 2 should be reviewed, classified, and documented as depicted in Figure 1.

    A description of the open literature study classifications, guidelines for study reviews, and completion/documentation of open literature data summaries is provided in Sections 3.1 through 3.3, respectively.

    • 3.1 Study Classifications

      Open literature studies that may provide additional information on existing measurement endpoints, effects on non-guideline or underrepresented taxa (i.e., aquatic- and terrestrial-phase amphibians, reptiles, freshwater mollusks, etc.), and sublethal effects that are quantitatively linked to survival, reproduction, and growth should be reviewed and classified as to their usefulness in a risk assessment. The three general categories for classifying open literature studies are:

      • Quantitative: Appropriate for use in RQ calculations;

      • Qualitative: Not appropriate for quantitative use, but is of sufficient quality, relevant to issues of concern in the risk assessment, and can be used descriptively in the risk characterization; and

      • Invalid: Inappropriate for use in RQ derivation (quantitative use) and in risk description (qualitative use) because it is of insufficient quality and lacks scientific defensibility.

      Further general guidelines for reviewing open literature studies and identifying the study classification as "quantitative", "qualitative", or "invalid" are provided below in Section 3.2.

    • 3.2 Guidance for Open Literature Study Review

      Given that a collective source of definitive guidance has not been available to OPP science staff to consistently differentiate open literature studies into the three categories listed above, the following collective guidance has been developed. This guidance is based on:

      1. the available guidance for review of registrant-submitted studies in response to the 40 CFR Part 158 data requirements and

      2. on guidelines provided by the Office of Water in reviewing open literature as part of the Ambient Water Quality Criteria (AWQC) derivation (Stephan et al., 1985).

      The risk assessor must also use best professional judgment, in addition to the considerations discussed below to determine the appropriate study classification for open literature studies. While a single factor may result in the down-grading of a study to invalid (e.g., excessive control mortality), more typically, several factors combine to render the study of questionable utility.

      The EPA Pesticide Assessment Guidelines Subdivision E Assessment Guidelines serve to identify the effects of pesticides to non-target fish and wildlife (USEPA, 1982). The guidelines specify when a test is required, the testing standards that should be met, and the [raw] data that should be reported. General information that should be considered as important in determining the utility of an open literature study in risk assessment and guidance to determine whether the study meets guideline testing requirements include the following:

      • Nature of the test substance (percent active ingredient; manufacturer).

        The study must indicate the exact nature and source of the pesticide. The percent active ingredient and/or the purity of the test compound (i.e., identification of the test material as TGAI) should be reported. If a vehicle is used, the vehicle should not interfere with the absorption, distribution, metabolism or the elimination (ADME) of the test substance nor alter the behavior/response of the test organisms. Studies which rely on solvents should include solvent controls to document that the solvent did affect measurement endpoints.

      • Species, age, sex, size, and life stage and source of the test species.

        The health of the test organism should be reported. The test organisms, to the extent possible, should be of uniform weight, size and age, and have no history of pre-exposure to pesticides or other contaminants. Observed diseases and treatment must be reported.

      • The number of organisms tested per concentration and the number of concentrations or dosage levels evaluated.

        This type of information should be reported and be sufficient to yield statistically sound results. An inadequate number of test organisms per test level can also produce unreliable results. The appropriate comparable guideline study Standard Evaluation Procedure (SEP) and/or Office of Prevention, Pesticides, and Toxic Substances (OPPTS) 850 test guideline (http://www.epa.gov/opptsfrs/publications/OPPTS_Harmonized/850_ Ecological_Effects_Test_Guidelines/Drafts) should be consulted for further information on the adequate number of test organisms per test level.

      • Exposure method, route, and frequency of administration and length of the treatment period.

        The total volume of material administered (test substance plus carrier) to each organism or in feed or water at each time administration is made must be reported. In addition, the frequency of administration and duration of the exposure must be reported. For all studies, the exposure conditions must be clearly described and documented. Where measured concentrations are not available and/or they fluctuate markedly during the study period, it is difficult to characterize exposure to the extent that a dose-response (cause/effect) relationship can be documented. Typically, OPP rejects studies where measured concentrations deviate more than 20% and/or if variability between treatment group-measured concentrations is sufficiently high to render treatment means statistically indistinguishable. Additional factors to be considered when reviewing studies with high variability between treatment group measured concentrations are provided in Attachment 5. Where only nominal concentrations are reported, the reviewer must consider whether the test compound is subject to degradation, volatilization, partitioning and/or a combination of these properties such that exposure levels may be considerably different than the reported nominal values (discussed further below). Additionally, the reviewer must consider whether test conditions may not sufficiently preclude exposure to other chemicals that could potentially confound the study. In such cases, the reviewer should consider the variability associated with the measured endpoints from the controls.

      • Controls.

        A suitable number of controls must be run to test whether study conditions are adequate. Control performance should be used as an indicator of whether study conditions and animal performance are adequate. To this end, controls must be run concurrent with the study; failure to do so would invalidate the study. As mentioned previously, studies which rely on co-solvents should report concurrent solvent controls. As an indicator of study conditions, control performance in terms of mortality and disease should be carefully evaluated to determine the adequacy of the study. Mortality of greater than 10% in controls for most test species is sufficient to invalidate acute studies. Ideally, studies should also report the measured concentrations of test chemical in the controls. Studies reporting test chemical residues in the controls > LOD are invalidated as the ability of the study to discriminate a treatment effect may be compromised.

      • Performance of test species.

        Normal development times (where available) should be compared to those reported for the test species. Where the development time for the control animals differs substantially from normal reported values, the reviewer must determine whether study conditions have impaired the animals' ability to thrive. In cases where development time is substantially different than what is typically observed for the test organisms, the study should be invalidated as the study's ability to distinguish treatment effects is uncertain.

      • Macroscopic observations of the test animals.

        During the course of the study, a detailed description of the nature, incidence, time of occurrence, severity, and duration of all observed toxic effects, including death and any other abnormal or unusual signs and symptoms (i.e., sublethal effects) should be reported.

      • Husbandry conditions.

        Guideline studies have been developed using particular species to establish conditions under which the test organisms are most likely to thrive and where husbandry conditions will not confound the interpretation of the study. Reviewers must be cognizant of husbandry conditions and verify whether the environmental conditions of the study are adequately described and/or addressed to ensure that the test organisms are not adversely affected. This description should include the number of animals per cage or test container (i.e., biological loading rate); nature and composition of bedding used for mammalian studies (if available); ambient temperature and humidity; photoperiod; description of the diet; source of the animal feed; dimensions of the test container; source of the dilution water and a description of its chemical characteristics; description of the toxicant delivery system and flow rate expressed as the average water volume of test solution passing through each test chamber in 24 hours. Loading rate of the study must be sufficient to ensure adequate husbandry conditions. Reviewers should consider whether static or flow-through conditions are adequate to support the number of test animals. Control performance and the variability associated with these measures should be used as an indication of the test environment suitability.

      • Statistical method used to derive the test endpoints.

        Verification of the statistical analysis is an integral part of the data evaluation process. As such, studies should provide descriptive statistics that report measures of central tendency (e.g., means, medians) and measures of dispersion (e.g., standard deviations, standard errors) along with associated sample sizes (N values). The report should state which methods of statistical comparison (e.g., t-test, ANOVA, chi square) were used and the presumed nature or the data (parametric versus nonparametric).

      • Information necessary to provide a complete and accurate description of test procedures and evaluation of the test results.

        Each report should include a summary of the data, a description of the statistical analysis of the data, and a statement of conclusions drawn from the analysis that allows the reader to independently understand the conclusions of the author.

      • Important information missing from the study.

        Inconsistencies or deviations with recommended methodologies, as discussed in the appropriate comparable guideline study SEP and/or 850 guideline for each of the respective studies, should be addressed. SEPs and/or 850 test guidelines can provide additional measures of gauging the reliability of study conditions.

      Additional information to consider when reviewing aquatic open literature studies include the following:

      • Test chemical properties (solubility, Kd and Koc, vapor pressure/Henry's law constant).

        This information is needed to determine whether actual concentrations may differ substantially from nominal and where it would be critical to have measured values throughout the conduct of the study. The draft OPPTS Guideline 850.1000 (US EPA, 1996) provides useful guidance on the design and conduct of aquatic studies with difficult to test substances and should be considered when determining the acceptability of a study. The solubility and stability of the test material must be known for the conditions under which it is tested in order to provide scientifically defensible information. The representative analysis must be conducted with the specific media for which it is used during the test, i.e., the analysis should be performed under test conditions, and the limit of detection (LOD) and limit of quantification (LOQ) should be identified. In aquatic studies, low solubility as evidenced by precipitates or films in/on the water without centrifuging and/or filtering of the water samples prior to analysis would not adequately characterize exposure. The water solubility of the compound relative to treatment concentrations should also be considered. Additionally, aerated treatment units where the compound is likely to be volatile without measured concentrations are also likely to result in an invalid study because exposure is not adequately characterized. Typically, OPP has required that measured concentrations must be > 70% of nominal except for aquatic plant studies. For chemicals where stability, solubility, volatility, and/or sorption may be issues, chemical measurements at study initiation and termination are considered critical.

      • Water quality.

        Water quality parameters (e.g., dissolved oxygen, temperature, pH, etc.) should be reported. In addition, biological loading rates must be suitable for the test container and not compromise water quality during the study. Generally, for water column tests, mean dissolved oxygen concentrations should not drop below 60% saturation for prolonged periods, unless justification can be provided that dissolved oxygen suppression does not interfere with the interpretation of the study results.

      • Negative and solvent control performance.

        As previously discussed, control performance must be evaluated to determine whether environmental conditions and solvent selection are sufficient. EFED has developed guidance (USEPA, 2008) for aquatic studies for determining whether negative and solvent control performance is adequate. This guidance should be considered when determining whether an aquatic toxicity study is valid (G:\Teams and Panels\Tech Teams\Aquatic Biology(ABTT)\Solv Neg Control Guidance Memo signed.pdf).

      • Endpoint Selection.

        Endpoints from open literature studies should be comparable to those used as inputs to the Risk Quotient (RQ) method for expressing risk (see Section V.B.2 of the Overview Document; USEPA, 2004b). Briefly, these endpoints would include acute EC50, LC50, or LD50 values and chronic NOAEC values for animal and aquatic plant studies and EC25, EC05 and/or NOAEC for terrestrial plant studies.

      • 3.2.1 Guidance for Invalidation of Open Literature

        Invalid open literature studies are those that are not considered scientifically sound and do not provide useful/reliable information. These can include studies that were performed under conditions that deviated so significantly from the recommended protocols that the results are not useful in a risk assessment. The Rejection Rate Analysis (US EPA, 1994) provides information on factors that most frequently result in the rejection of ecological effects guideline studies required for pesticide registrations.

        Aquatic studies commonly placed in this category include those where the test system was aerated, test vessels were constructed from materials other than glass, or where there were problems with solubility or volatility of the test material such that the reviewer cannot be sure that the organisms were exposed to nominally designated residues. Also, a study where the test material was not properly identified can be invalidated. In addition to the reasons discussed in Section 3.2, a list of factors that could result in invalidation of open literature data is provided in Attachment 5.

      • 3.2.2 Guidance for Differentiating Between Quantitative and Qualitative Open Literature Studies

        If a study is not invalidated based on the guidelines described above and provided in Attachment 5, a determination is made regarding whether the information provided in the study is adequate for "quantitative" or "qualitative" use in risk assessment. For OPP's purposes, "quantitative" means the data from the study can be used to calculate RQs in risk assessment. "Qualitative" refers to data that are not adequate for RQ calculation but can be used as lines of evidence to support risk conclusions for listed and non-listed species.

        To be used quantitatively or qualitatively in risk assessment, data must be from open literature studies that are considered scientifically valid (i.e., not classified as invalid), where the toxic effects are related to single chemical exposure, and a quantitative relationship exists between the measurement endpoint and the Agency's assessment endpoints of growth, mortality, and/or reproduction. To be used quantitatively, the endpoint(s) reported in the open literature must meet all of the following guidelines:

        • The endpoint from the open literature study must be lower (i.e., more sensitive) than the lowest endpoint from a comparable registrant-submitted study for all taxon with the exception of aquatic-phase and terrestrial-phase amphibians, freshwater mollusks, and reptiles or derivation of species sensitivity distributions for other taxonomic groups. For amphibians, freshwater mollusks, and reptiles the endpoint from the open literature study may be either more or less sensitive than the corresponding endpoint for surrogate taxa, which include freshwater fish for aquatic-phase amphibians, freshwater invertebrates for freshwater mollusks, and birds for terrestrial-phase amphibians and reptiles. In addition, if the risk assessor intends to derive species sensitivity distributions, the full range of data (from least to most sensitive) should be considered;

        • The open literature endpoint is reported in (or has the ability to be converted to) units that can be compared to modeled EECs (e.g., lb a.i./acre for terrestrial plants);

        • The open literature endpoint is consistent with those endpoints currently used to derive RQs (e.g., LC50 for acute exposure in fish); and

        • Sufficient information must be provided in the open literature to substantiate and/or independently evaluate whether the study conclusions (i.e., dose-response) and endpoints are accurate.

        Depending on the measurement endpoint, the same evaluation criteria as those used in registrant-submitted guideline studies for similar endpoints should be used to gauge the utility of the open literature study. If a scientifically valid study does not meet any one of these four guidelines, the data from the study should be classified as "qualitative". OPP recognizes that the fourth criterion (i.e., sufficient information must be provided [in the citation] to substantiate whether the conclusions/endpoints are accurate) requires best professional judgment. The most reliable means of determining whether study conclusions can be verified is through access to the raw data; however, it is recognized that very few open literature papers provide this type of information. Therefore, the quantitative use of open literature requires that the study provide a relatively comprehensive understanding of the conditions under which the study was conducted and of the data generated by the study. The study should report relatively detailed measures of the variability associated with the data and the methods used to analyze the data. Reviewers should note whether the statistical tests used in the study are appropriate to the design of the study, the nature of the measurement endpoint, and of the data generated in the study. Tests using parametric statistics should indicate whether the conditions for such tests (i.e., normal distribution and homogeneity of variance) have been met.

        Where raw data are not available to verify the study endpoints, the reviewer must discuss the uncertainties associated with quantitative use of the data relative to studies where raw data are provided. Consideration must be given as to the extent to which endpoints are aligned with other lines of evidence. Open literature values that are inconsistent with similar measures of toxicity should be carefully scrutinized to determine their reliability. Depending on the importance of the open literature study to the risk assessment conclusions, attempts should be made to obtain missing information from the study, including the raw data, if possible.

        Beyond the guidelines discussed above and those addressed in Section 3.2, distinguishing between data that can be used quantitatively versus qualitatively will largely depend on professional judgment.

        The reviewer should also consider that although studies may be deemed of insufficient quality to be used quantitatively in risk assessments, information from qualitative studies may be used as additional lines of evidence in the risk description section of the assessment. It should be noted, however, that qualitative data should not be used in lieu of quantitative data to derive risk quotients in the risk estimation section of the assessment.

    • 3.3 Completion of Open Literature Review Summaries (OLRSs)

      The purpose of completing the OLRS is to ensure an efficient and consistent process for documenting reviews of the ECOTOX open literature and avoiding duplicative and possibly conflicting efforts associated with study reviews (see EFED policy memo for Reviews of Scientific Literature provided as Attachment 7).

      The open literature review summary (OLRS) form provided in Attachment 6 should be completed for open literature studies that meet all of the following criteria:

      • The toxic effects are related to single chemical exposure;

      • Endpoints are more sensitive than those from registrant-submitted guideline studies with the exception of those derived from aquatic-phase and terrestrial-phase amphibians, freshwater mollusks, and reptile studies or endpoints used in derivation of species sensitivity distributions for other taxonomic groups. For amphibians, freshwater mollusks, and reptiles the endpoint from the open literature study may be either more or less sensitive than the corresponding endpoint for surrogate taxa, which include freshwater fish for aquatic-phase amphibians, freshwater invertebrates for freshwater mollusks, and birds for terrestrial-phase amphibians and reptiles; AND

      • There is a peer-reviewed, quantitative relationship between the measurement endpoint and the Agency's assessment endpoints of survival, reproduction, and/or growth which are known to impact species at the population level.

      With respect to under-represented taxa including aquatic- and terrestrial-phase amphibians, freshwater mollusks, and reptiles, endpoints that are more or less sensitive than the standard surrogate taxa (i.e., freshwater fish for aquatic-phase amphibians, freshwater invertebrates for freshwater mollusks, and birds for terrestrial-phase amphibians and reptiles) should be considered for formal review and OLRS documentation in the initial screen. However, as previously discussed in Section 2.1.2, it may not be necessary to review all studies with under-represented taxa endpoints that are less sensitive than their standard surrogates, unless the risk assessor intends to use the full range of data (from least sensitive to most sensitive) in a species sensitivity distribution. All open literature data from under-represented taxa that are used to derive risk quotients and/or are used in species-sensitivity distributions as part of the risk assessment refinement must be reviewed and determined to be scientifically valid for quantitative use.

      OLRSs are not required for studies on mixtures and/or sublethal effects where a peer-reviewed quantitative relationship between the sublethal measurement endpoint and the Agency's assessment endpoints of survival, reproduction, and/or growth is not established. Although OLRSs are not required for studies on mixtures and sublethal endpoints that cannot be quantitatively linked to the assessment endpoints of survival, reproduction, and growth, a brief description of data on mixtures and sublethal endpoints must be provided in the risk assessment. Further description regarding inclusion of mixture data and sublethal effects endpoints in the risk assessment is provided in Section 4.2.3.

      In order to ensure completion and submittal of OLRSs, a requirement has been added to the EFED Review Panel Document Quality Concurrence and Tracking form to indicate that the risk assessors have completed and submitted OLRSs for all endpoints that are classified as "quantitative", "qualitative", or "invalid". This form must be signed by the Branch Chief and at least one senior reviewer prior to distribution of the risk assessment to the EFED Review Panel.

      The procedures for completion and submittal of OLRSs for endpoints that are classified as "quantitative", "qualitative", or "invalid" are described below in Sections 3.3.1 through 3.3.3, respectively.

      • 3.3.1 Completion and Submittal of OLRSs for Endpoints Used Quantitatively

        Review summaries of open literature data that are used quantitatively (i.e., to derive RQs) should include all available information that would normally be included as part of the current guideline/non-guideline Data Evaluation Record (DER) templates. Although the OLRS should include the same type of information included in the DER templates, it is expected that they will be reduced in length and detail as compared to standard DERs because raw data are generally not available for review and only the data in the published study are being evaluated. Using the form provided in Attachment 6, the basic study requirements, such as use of solvent and negative controls, control mortality rates (or other issues with controls that could affect the study validity), number of test concentrations, number of treatments (e.g., 3 applications, 7-day application interval), water quality parameters, and suitable replication should be verified and reported in the OLRS. The review should document all statistically or biologically significant effects. In addition, the duration of exposure, the magnitude of the effect, and the test concentration (nominal, measured, and time-weighted average, if it can be determined) at which the effect was observed should be documented. Each documented endpoint should specify the affected taxa and/or individual species. In addition, the reviewer is encouraged to include relevant figures and tables from the study that include key findings; table and figure captions should properly cite the relevant publication if the figure and/or table is copied from the publication. All open literature studies that are classified as "quantitative" and used to derive RQs must undergo two levels of internal peer review, including a primary review of the study and secondary review by senior staff, within OPP.

        An MRID number must be obtained for any open literature study in which the endpoint is used quantitatively. The purpose of assigning an MRID to the open literature study is to ensure that the study is documented as part of the study bibliography for the chemical in OPPIN and electronically available via Documentum. In order to obtain an MRID number for an open literature study, an electronic copy of the study should be provided via email to the Alternate Contracting Officer Representative (ACOR) of the Data Management Contract in the Information Technology Resources and Management Division (ITRMD). Currently, the ACOR point of contact in ITRMD is Teresa Downs (703-305-5363). The electronic file of the open literature study should be emailed to Teresa Downs at downs.teresa@epa.gov. MRID numbers are typically assigned within 2 to 10 days. Once the MRID is assigned to the open literature study, the citation for the study will appear in the OPPIN bibliography. In addition, a ".pdf" file of the study will be available in Documentum approximately ten days after the MRID is assigned to the study.

        Once the risk assessor has obtained an MRID number and completed the OLRS (including two levels of internal peer review), the OLRS should be saved as a file name that begins with the letter "E" followed by the six-digit ECOTOX number, then the PC code of the chemical for which the review was completed followed by the assigned MRID number. For example, the file name of the study review for ECOTOX paper 81457 for atrazine (PC code 080803) with an MRID number of "12345678" would be "E081457 PC080803 MRIDI2345678". Completed OLRSs for data used quantitatively must be submitted to the OPP EFED Tracking Team. When the OLRS is submitted electronically to the OPP EFED Tracking Team, the subject heading of the email should include the following text: "Submittal of Quantitative Open Literature Review (insert file name; insert DP Barcode) - classify as 'Acceptable Non-Guideline' Study in OPPIN". The OLRS should identify the DP Barcode associated with the review under the "Purpose of Review" heading. Risk assessors should work with their branch Risk Assessment Process Lead (RAPL) to obtain DP Barcodes for quantitative open literature studies reviewed as part of the Registration Review process and/or litigation-related assessments.

        Submittal of the electronic OLRS to the OPP EFED Tracking Team will ensure the following:

        • that the OLRS is documented as part of the ecological effects data for the chemical file (by PC code) and available electronically as a ".pdf" file on the SAN drive or as hard copy in the green PC code folder in the EFED library;

        • that the associated quantitative endpoint from the reviewed open literature study is entered into EFED's Ecotoxicity Database, and

        • that the OPPIN citation indicates that the study is acceptable as a non-guideline study.

      • 3.3.2 Completion and Submittal of OLRSs for Endpoints Used Qualitatively

        OLRSs must also be completed using the form provided in Attachment 6 for open literature studies that include endpoints to be used qualitatively in the risk assessment. To the extent possible, OLRSs for qualitative endpoints should include the same type of information and level of detail as reviews that are completed for quantitative endpoints. In addition, OLRSs for qualitative endpoints must include a description of the study limitations which preclude its quantitative use (note that secondary review by senior staff is not required for studies that are classified as "qualitative"). MRID numbers are not required for open literature studies that are classified as "qualitative." OLRSs for qualitative studies should be saved with corresponding file names that begin with the letter "E" followed by the six-digit ECOTOX record number. For papers that include data on multiple chemicals, and the risk assessor's review is limited to a single pesticide or active ingredient, the file name should also include the PC code of the chemical for which the review was completed. For example, if ECOTOX paper 81457 includes data for multiple herbicides, and the review was limited to atrazine (PC Code 080803), the file name of the review would be "E081457 PC080803". Electronic files of the OLRS for qualitative endpoints should not be submitted to OPP EFED Tracking Team, but rather emailed to Brian Montague (and cc'd to the EISB Branch Chief). Completed "qualitative" open literature study reviews will be converted into ".pdf" files and copied to the SAN drive.

      • 3.3.3 Completion and Submittal of OLRSs for Invalid Open Literature Studies

        For those open literature studies that are classified as "invalid", OLRSs must be completed using the form provided in Attachment 6; however, the length and level of detail relative to "quantitative" and "qualitative" reviews should be significantly reduced. The abbreviated OLRS for invalid studies should be condensed into 1-2 pages and focus on the limitations of study which preclude its use in risk assessment. Detailed description of the experimental design is not required for studies that are classified as "invalid". Further guidance on invalidation of open literature studies is provided in Section 3.2 and Attachment 5. OLRSs for "invalid" studies should be saved with corresponding file names that begin with the letter "E" followed by the six-digit ECOTOX record number. For papers that include data on multiple chemicals, and the risk assessor's review is limited to a single pesticide or active ingredient, the file name should also include the PC code of the chemical for which the review was completed. In addition, the file name should include the term "invalid"; inclusion of the study classification in the file name will allow the risk assessor to quickly determine that the study is invalid without opening the file. For example, if ECOTOX paper 123456 included data for multiple herbicides, and the review was limited to atrazine (PC Code 080803) and determined to be invalid, the file name of the review would be "E081457 PC080803  invalid" For invalid studies, submittal of OLRSs to EISB and availability of ".pdf" files on the SAN drive are identical to those described above in Section 3.3.2 for "qualitative" studies. Also similar to "qualitative" studies, MRID numbers are not required for open literature studies classified as "invalid."

  4. Use of Open Literature in Problem Formulation and Risk Assessment

    • 4.1 Use of Open Literature Data in Registration Review Problem Formulation

      As of May 2010, ECOTOX open literature searches conducted by ORD/MED will not be provided to the risk assessment teams prior to the problem formulation stage of Registration Review. The ECOTOX searches will be delivered to the teams approximately 6 months prior to the preliminary risk assessment due date, depending on the complexity of the chemical and the time period between previous ECOTOX searches, if any. The purpose of limiting the ECOTOX search to the preliminary risk assessment phase is to maximize resources on the best available information at the time of the actual risk assessment, given that the problem formulation written in support of Registration Review may precede the actual risk assessment by several years.

      There may be situations where the ORD/MED ECOTOX search is available for a particular chemical prior to the problem formulation stage, such as those for which endangered species litigation assessments have recently been completed. In the event that an ORD/MED ECOTOX search is available prior to the problem formulation, a bibliography of the accepted and rejected papers, based on the most recent ECOTOX search, should be provided as an appendix to the problem formulation, with a statement in the Analysis Plan section that indicates the data from accepted papers and those rejected papers containing mixtures data will be considered as part of the risk assessment. In cases where an ECOTOX search has been previously completed, it should also be stated in the Analysis Plan of the problem formulation that the Agency will consider any new information from the open literature, based on an updated "refresh" of the ECOTOX search to be completed prior to the preliminary risk assessment. In addition, registrants should be encouraged to formally submit any other relevant open literature to the Agency that could potentially be used to satisfy data requirements where data gaps exist and/or inform our understanding of the ecological effects toxicity profile of the active ingredient, degradates, or formulations which are the subject of Registration Review. As stated previously, registrant submissions of open literature will be subject to the same standards described in this guidance for review of relevant open literature studies obtained from ECOTOX. For appendix bibliographies with a large number of citations, the ".pdf" file of the citations may be attached as a separate CD to the problem formulation.

      If no ECOTOX search is available at the problem formulation stage, it should be clearly stated in the Analysis Plan that the ORD/MED ECOTOX search will be conducted prior to and included as part of the preliminary risk assessment. In addition, registrants should be encouraged to formally submit open literature data that are intended to satisfy the Agency's data requirements where data gaps exist and/or provide additional information on the ecotoxicity profile of the active ingredient, degradates, or formulations, as discussed above. Also, as discussed above, these submissions will be subject to the same standards described in this guidance for other open literature studies. Finally, a hyperlink to the publicly available version of ECOTOX (http://cfpub.epa.gov/ecotox/) should be provided and accompanied by a statement indicating that all open literature data included in ECOTOX will be considered in the preliminary risk assessment, provided it passes the screening criteria discussed in Section 2 of this guidance document.

      • 4.1.1 Use of Registrant-submitted Open Literature Studies to Fulfill Data Requirements

        Open literature submitted by the registrant to OPP may be used to fulfill a data requirement, provided the study has been reviewed and a determination has been made that the study, by itself, is sufficient to address the purposes of the guideline. Generally, an open literature study that is not acceptable for quantitative use according the guidelines provided in this document will not satisfy the Agency data requirements. In addition, if access to actual study reports and/or raw data is necessary for EPA to reach sound conclusions, public literature citations or submissions, by themselves, will not be deemed adequate to satisfy Agency data requirements. Applicants may utilize appropriate public literature found through ECOTOX or through other open literature to satisfy the Agency's data requirements. However, OPP's ECOTOX searches cannot, by themselves, serve to fulfill data requirements for applicants; applicants have an independent obligation, pursuant to EPA regulations codified in 40 CFR Part 152.80-.99, to demonstrate to EPA that they have fulfilled all necessary data requirements. In summary, it is the applicant's responsibility, not EPA's, to provide submissions and/or citations that are intended to satisfy the Agency's data requirements for pesticides.

        In the event that screening, review, and documentation of the ECOTOX open literature has occurred prior to the problem formulation, for example, in a previous endangered species assessment, and data that are classified as "quantitative" or "qualitative" have been described in previous assessments, non-guideline studies from the ECOTOX open literature may not be used to fulfill a data requirement unless they have been formally submitted by the applicant and deemed as adequate to satisfy the data requirement, as discussed above. Although ECOTOX open literature data previously reviewed by OPP, but not formally submitted by the applicant, may not be used to satisfy a data requirement, these data should be briefly discussed in the effects section of the problem formulation.

      • 4.1.2 Discussion of Endpoint Refinement in Registration Review Problem Formulation

        Currently, acute and chronic endpoints for aquatic and terrestrial receptors (summarized in Attachment 3) are identified and summarized in the Registration Review problem formulation; these endpoints are based on the most sensitive endpoint from registrant-submitted and open literature ecological effects data on surrogate taxa. However, as previously discussed in Section 2.1.2, the methodology for deriving endpoints has been updated to include endpoint refinement for certain under-represented taxonomic groups including aquatic-phase amphibians, freshwater mollusks (i.e., bivalves and snails), terrestrial-phase amphibians, and reptiles, where both more and less sensitive data from the open literature are considered. Although the actual refinement of endpoints for specified taxa will not occur until an updated ECOTOX search is provided as part of the preliminary Registration Review risk assessment, the problem formulation section should convey this refined approach for endpoint derivation. The language provided below should be included as a preface to the summary of aquatic and terrestrial effects data in all Registration Review problem formulations:

        Suggested problem formulation language for endpoint refinement

        "Consistent with the process described in the Overview Document (USEPA, 2004), the Agency has routinely relied on a surrogate species approach. Toxicological data generated from surrogate test species, which are intended to be representative of broad taxonomic groups, have been used to extrapolate the potential effects on a variety of species (receptors) included under these taxonomic groups. For example, guideline data for freshwater fish have typically served as surrogates for aquatic-phase amphibians and submitted studies for birds have served as surrogates for terrestrial-phase amphibians and reptiles. For the purposes of Registration Review, the Agency will consider the best available data in an effort to provide additional refined endpoints for specified taxonomic groups for which guideline data on surrogate species are typically not available and large numbers of listed species occur. These taxa include aquatic-phase and terrestrial-phase amphibians, freshwater mollusks, and reptiles. Based on an updated ECOTOX search, which will be conducted as part of the Registration Review preliminary risk assessment, EPA will consider both more and less sensitive endpoints for amphibians, freshwater mollusks, and reptiles, as compared to the registrant-submitted data for surrogate species (i.e., freshwater fish are surrogates for aquatic-phase amphibians, freshwater invertebrates are surrogates for freshwater mollusks, and birds are surrogates for terrestrial-phase amphibians and reptiles). In situations where no additional open literature data are available to refine endpoints for amphibians, freshwater mollusks, and reptiles, the Agency will rely on registrant-submitted data and open literature for surrogate test species, as specified above."

    • 4.2 Use of Open Literature Data in Registration Review and Endangered Species Litigation Risk Assessments

      The extent to which open literature data classified as either "quantitative" or "qualitative" should be used in the risk assessment is discussed below in Sections 4.2.1 and 4.2.2, respectively. In addition, discussion of open literature data on mixtures and sublethal measurement endpoints that are more sensitive than guideline studies but not quantitatively linked to the Agency's assessment endpoints of survival, reproduction, and/or growth is provided in Section 4.2.3. Open literature studies that pass the initial screen discussed in Section 2 and are determined to be "invalid" based on the risk assessor's review should not be included in the risk assessment; however, the OLRS or other type of review summary that explains the rationale for invalidation should be provided as an appendix to the risk assessment. As previously discussed, OLRSs of studies that are classified as "invalid" should be submitted to EISB in order to document the study's invalidation.

      All Registration Review and endangered species litigation risk assessments should include an appendix that provides the ECOTOX Excel spreadsheet and associated bibliography of accepted papers, including submitted non-ECOTOX open literature studies, as discussed above. Instructions for formatting the ECOTOX Excel spreadsheet for inclusion as an appendix to the risk assessment are provided in Attachment 2. In addition, the citations of all ECOTOX and submitted non-ECOTOX open literature not considered in the risk assessment because they were either rejected by the ECOTOX and/or OPP screen or included in the "Other" category must also be included as a separate appendix to the risk assessment. This separate appendix must also include the rationale for rejection of those studies that were not included. The rationale for rejection is normally included in ECOTOX bibliographies of open literature studies that are rejected based on either the ECOTOX and/or OPP criteria. For those submitted non-ECOTOX studies that are rejected, the risk assessor must add the appropriate rejection code along with the study citation and associated MRID number (from the OPPIN bibliography) to the appendix. The rationale for not including a study can consist of the rejection codes alone, provided they are properly defined in a key. The open literature rejection codes and an example of the type of language that should accompany the list of papers not included in the risk assessment are provided in Attachment 4.

      In some situations, open literature studies may be available that have not been captured through ECOTOX; in many cases, these studies have been submitted to the Agency and are assigned MRID numbers. Open literature studies that have been formally submitted to the Agency must be reviewed as though they had been captured by ECOTOX, and the same filters and standards used by ECOTOX and discussed in this guidance for screening purposes should be applied to these additional studies. The process for reviewing open literature studies that have been formally submitted to the Agency and not captured in the ECOTOX search is depicted in Figure 2. In order to ensure that all submitted non-ECOTOX open literature data are considered, EISB will cross-walk the ecological effects bibliography of submitted studies from the OPPIN database with all ECOTOX searches. Based on this cross-walk, EISB will generate a bibliography of all non-ECOTOX open literature citations in OPPIN, including the corresponding MRID for the study and a Document ID number. The information contained in this bibliography will allow the risk assessor to locate the study and any previous documentation of the study reviews. Previous reviews of non-ECOTOX open literature, if available, are located in the SAN drive under the folder corresponding to the PC Code for the chemical of interest. The study review can be located based on the Document ID number for the MRID study of interest, which is included in the OPPIN non-ECOTOX bibliography provided by EISB. In the event that a review has not been completed for the submitted non-ECOTOX open literature study, the original paper should be obtained via Documentum by searching on the MRID number. Once the original paper is obtained, the risk assessor should apply the same filters used to screen and review the ECOTOX studies, determine its utility in the risk assessment, and complete and submit the appropriate OLRSs. In addition, the citations and MRID numbers for non-ECOTOX open literature studies should be provided in the appended list of references for open literature that are either accepted or rejected, as appropriate.

      • 4.2.1 Use of "Quantitative" Open Literature Data in Risk Assessment

        Endpoints from the open literature that are classified as "quantitative" should be used to calculate RQs in the risk estimation section of the risk assessment. Section 3.2.2 provides further information on the guidelines for classifying an open literature study as "quantitative". A complete list of the taxonomic groups for which quantitative endpoints should be derived, including refined endpoints for under-represented taxa (which are highlighted and bolded) is provided in Table 2. For each of the taxa-specific endpoints, Table 2 also summarizes the corresponding taxonomic group of listed species to which the endpoint applies for listed species under the jurisdiction of the U.S. Fish and Wildlife Service (USFWS) and the National Marine Fisheries Service (NMFS). It should be noted that the classes identified in Table 2 are representative of taxonomic groups for which listed species occur. When searching the open literature data for more sensitive endpoints, the risk assessor should also consider taxonomic groups and/or classes that are representative of the broader taxonomic groupings of non-listed species. For example, when considering the open literature data for non-listed aquatic invertebrates, more sensitive endpoints for all aquatic invertebrates, including, but not limited to aquatic insects, annelids, ostracods, polychaetes, mayflies, caddisflies, stoneflies, etc. should be considered.

        Table 2
        Taxa-specific Endpoints and Applicability to Listed Species
        Taxa-specific Endpoints Method for Endpoint Selection 1 Class 2 Effect Type Endpoint USFWS Listed Species NMFS Listed Species
        Freshwater fish Most sensitive from guideline freshwater fish study or quantitative open lit. Actinopterygii and others (e.g., Chondrichthyes) Acute LC50 Fishes Marine and Anadramous Fish 3
        Chronic NOAEC
        Aquatic-phase amphibians
        [under-represented taxa]        		 More or less sensitive quantitative open lit endpoint (search for freshwater amphibian studies) as compared to FW fish endpoint. If no open lit, use FW fish endpoint as default Amphibia Acute LC50 Amphibians NA
        Chronic NOAEC
        Freshwater invertebrates
        (non-mollusks)        		 Most sensitive from guideline freshwater invertebrate study or quantitative open lit. Brachiopoda
        (shrimp and daphnia)
        Malacostraca
        (amphipods, isopods, & crayfish)        		 Acute LC/EC50 Crustaceans NA
        Chronic NOAEC
        Freshwater invertebrate - mollusks (bivalves and snails)
        [under-represented taxa]        		 More or less sensitive quantitative open lit endpoint (search for freshwater bivalve and gastropod studies) as compared to FW invert endpoint. If no open lit, use FW invert endpoint as default Bivalvia
        Gastropoda        		 Acute LC/EC50 Clams & Mussels
        Snails        		 NA
        Chronic NOAEC
        Estuarine/marine fish Most sensitive from guideline estuarine/marine fish study or quantitative open lit. Actinopterygii and others (e.g., Osteichthyes) Acute LC50 NA Marine and Anadramous Fish 3
        Chronic NOAEC
        Estuarine/marine invertebrates Most sensitive from guideline mysid and/or Eastern oyster study. Search open lit for quantitative endpoint that is more sensitive than endpoint from guideline study (search for saltwater gastropoda and/or anthozoa studies). If no open lit, use Eastern oyster endpoint for abalone and mysid endpoint for coral Malacostraca (mysid); Bivalvia (Eastern oyster); Gastropoda (abalone); Anthozoa (coral) Acute LC50 NA Marine Invertebrates: Abalone
        Coral
        Chronic NOAEC
        Vascular aquatic plant Duckweed endpoint or more sensitive quantitative endpoint from open literature (search for aquatic, plantae vascular studies) Kingdom: Plantae
        Class: Lilopsida (duckweed) and others        		 Acute EC50, NOAEC or EC05 Flowering plant subset Marine plants:
        Johnson's seagrass
        Nonvascular aquatic plant Most sensitive from green algae, BG cyanobacteria, freshwater diatom, and marine diatom guideline studies or more sensitive quantitative endpoint from open literature (search for aquatic, plantae, non-vascular studies) Kingdom: Plantae Acute EC50, NOAEC or EC05 NA NA
        Birds Most sensitive from guideline avian study or quantitative open lit. Aves Subacute dietary LC50 Birds NA
        Acute oral LD50
        Chronic NOAEC
        Reptiles
        [under-represented taxa]        		 More or less sensitive quantitative open lit endpoint (search for terrestrial and aquatic reptilian studies) as compared to avian endpoints. If no open lit, use avian endpoints as default Reptilia Subacute dietary LC50 Reptiles Marine turtles
        Acute oral LD50
        Chronic NOAEC
        Terrestrial-phase amphibians
        [under-represented taxa]        		 More or less sensitive quantitative open lit endpoint (search for terrestrial amphibian studies) as compared to avian endpoints. If no open lit, use avian endpoint as default Amphibia Acute LC50 Amphibians NA
        Chronic NOAEC
        Mammal Most sensitive from guideline mammalian study or quantitative open lit. Mammalia Acute oral LD50 Mammals Marine mammals
        Chronic NOAEC
        NOAEL
        Terrestrial invertebrates Most sensitive from guideline acute contact honey bee study, OECD arthropod studies, or quantitative open lit (search for insecta and/or arachnida studies). If no open lit or OECD study, use honey bee endpoint as default. Insecta
        Arachnida        		 Acute LD50 Insects
        Arachnids        		 NA
        Terrestrial monocot
        (listed)        		 Most sensitive monocot NOAEC or EC05 from guideline seedling emergence and/or veg vigor study Liliopsida
        Lecanoromycetes (lichens)        		 Acute EC25, NOAEC or EC05 Flowering plant subset
        Lichens 4        		 NA
        Terrestrial dicot
        (listed)        		 Most sensitive dicot NOAEC or EC05 from guideline seedling emergence and/or veg vigor study. For ferns, it is proposed that the most sensitive vegetative vigor endpoint be used (seedling emergence endpoint is not relevant for ferns because they lack seeds) Magnoliopsida (dicots); Pteridopsida and Lycopodiopsida (ferns); Pinopsida (conifers/cycads); Lecanoromycetes (lichens) Acute EC25, NOAEC or EC05 Flowering plant subset
        Ferns
        Conifers & cycads
        Lichens 4        		 NA

        1 When considering data from the open literature, the risk assessor should be mindful that the route of exposure tested in the open literature study is comparable to that tested in the guideline study (i.e., direct contact, dietary, etc.).

        2 Identified classes correspond to taxonomic groups for which listed species occur. When searching the open literature data for more sensitive endpoints, the risk assessor should also consider taxonomic groups and/or classes that are representative of non-listed species. For example, when searching the open literature data for non-listed aquatic invertebrates, more sensitive endpoints for aquatic insects, annelids, ostracods, poly haetes, mayflies, cadd flies, stoneflies, etc. should be considered.

        3 The most sensitive of the freshwater and estuarine/marine fish endpoints should be used for anadromous listed fish under NMFS jurisdiction (i.e., salmonids).

        4 The most sensitive of the monocot and dicot vegetative vigor NOAEC or EC05 value should be used as the endpoint to evaluate direct effects to lichens.

        If more than one "quantitative" open literature endpoint is available for a particular taxonomic group, the lowest value should be used to derive the RQ for risk assessment purposes. In situations where "quantitative" endpoints are not available for aquatic-phase and terrestrial-phase amphibians, freshwater mollusks, and reptiles, RQs for these taxa should be derived based on endpoints for surrogate species including freshwater fish for aquatic-phase amphibians, freshwater invertebrates for freshwater mollusks, and birds for terrestrial-phase amphibians and reptiles.

        Where data from open literature are deemed to be of sufficient quality to permit their use quantitatively in an ecological risk assessment, the assessment must provide a relatively comprehensive review of the open literature study associated with the endpoint. Any open literature endpoints that are classified as "quantitative" and used to derive RQs in the risk assessment must be fully described in the effects section of the assessment, with particular emphasis on those open literature endpoints that result in lower values than those used in previous ecological risk assessments and/or new refined endpoints for amphibians, freshwater mollusks, or reptiles. In addition, the risk assessor should also include the OLRSs for all "quantitative" endpoints as an appendix to the risk assessment. The risk assessor must provide clear and transparent rationale for quantitatively using the open literature endpoint over guideline and GLP-compliant data. Depending on the measurement endpoint, the same evaluation criteria as those used in registrant-submitted guideline studies for similar endpoints should be used to gauge the utility of the open literature study. Ultimately, best professional judgment should be used to determine the appropriate use of an open literature study in a risk assessment.

        All open literature study endpoints that are classified as "quantitative" and deemed as appropriate for RQ derivation must be documented in the form of OLRS and have undergone two internal peer reviews (i.e., primary review by the risk assessor and secondary review by senior OPP staff); as such, the open literature review summary must contain at least two signatures. In addition, an MRID number must be obtained for the open literature study associated with the quantitative endpoint, and the OLRS must be submitted to the OPP EFED Tracking Team.

      • 4.2.2 Use of "Qualitative" Open Literature Data in Risk Assessment

        Although endpoints from the open literature that are classified as "qualitative" are not appropriate for quantitative use (i.e., RQ derivation), they should be discussed in the effects and risk description sections of the risk assessment as additional lines of evidence to support risk conclusions for listed and non-listed species. A clear rationale should be provided in the effects section that describes why the endpoints were not used quantitatively. These reasons might include limitations in the study design, lack of sufficient information to substantiate whether the conclusions/endpoints are accurate, and other uncertainties that confound the ability to discriminate a treatment-related effect. As previously stated, best professional judgment should be used to determine the appropriate use of an open literature study in risk assessment.

        It is up to the discretion of the risk assessor to decide whether to include OLRSs for those endpoints that are used qualitatively as an appendix to the risk assessment. At a minimum, a brief summary of the data, including the factors that precluded its quantitative use, should be described in the effects characterization of the analysis and risk description section of the risk characterization.

      • 4.2.3 Use of Mixtures and Sublethal Effects Data in Risk Assessment

        As discussed in Section 2, the screening process used to identify potentially useful open literature papers will capture studies on mixtures as well as studies with sublethal measurement endpoints that are more sensitive than guideline studies but not quantitatively linked to the Agency's assessment endpoints of survival, reproduction, and/or growth. While OLRSs are not required for these types of studies, a description of the available open literature data on mixtures and sublethal effects must be provided in the risk assessment.

        All documents must include a qualitative discussion of the available mixture data or include a statement that no data on mixtures including the chemical being evaluated are available. A brief summary of the available open literature data on mixtures should be provided in the scope subsection of the problem formulation (see suggested language below). In addition, a summary of the mixtures data should also be presented in the effects section of the risk assessment.

        Suggested language for mixtures to be included in the scope subsection of the problem formulation:

        "In general, the Agency does not routinely include, in its risk assessments, an evaluation of mixtures of active ingredients, including mixtures in the applicator's tank and/or environmental mixtures, because ecological effects toxicity data for these types of mixtures are typically not available. In the case of the product formulations of active ingredients (that is, a registered product containing more than one active ingredient), each active ingredient is subject to an individual risk assessment for regulatory decision regarding the active ingredient on a particular use site. However, if effects data are available for a formulated product containing more than one active ingredient, these data will be considered and may be used qualitatively or quantitatively, in accordance with the Agency's Overview Document and the Services' Evaluation Memorandum (USEPA, 2004; USFWS/NMFS/NOAA, 2004)."

        [The risk assessor should provide a brief discussion of the available data on mixtures. For example text on mixtures see: G:\Teams and Panels\Special Teams\CRLF-SFBAY\context\example text on mixtures and sublethal effects.doc]

        Open literature studies containing sublethal endpoints that are more sensitive than guideline studies but not quantitatively linked to survival, reproduction, and growth endpoints must also be described. While OPP does not base its risk conclusions and effects determinations on sublethal endpoints that are not quantitatively linked to survival, reproduction, or growth, it is important to summarize these types of endpoints in both the effects and risk description sections of the risk assessment. This information must be presented in the risk assessment, based on an agreement with the Services (i.e., USFWS and NMFS), who routinely include sublethal endpoints that are not quantitatively linked to survival, reproduction, and growth in their Biological Opinions.

  5. References

    Stephan, C.E., D.I. Mount, D.J. Hansen, J.R. Gentile, G.A. Chapman, and W.A. Brungs. 1985. Guidelines for Deriving Numerical Water Quality Criteria for Protection of Aquatic Organisms and Their Uses. U.S. Environmental Protection Agency. Office of Research and Development. Environmental Research Laboratories. PB85-227049.

    U.S. Environmental Protection Agency (USEPA). 1982. Pesticide Assessment Guidelines Subdivision E. Hazard Evaluation: Wildlife and Aquatic Organisms. Office of Pesticide and Toxic Substances. Washington DC. EPA 540/9-82-024.

    USEPA. 1994. Pesticide Reregistration Rejection Rate Analysis. Ecological Effects. Prevention, Pesticides, and Toxic Substances. EPA 738-R-94-035.

    USEPA. 1996. Ecological Effects Test Guidelines. OPPTS 850.1000. Special Considerations for Conducting Aquatic Laboratory Studies. Office of Prevention, Pesticides and Toxic Substances. EPA 712-C-96-113.

    USEPA.2004a. Interim Guidance of the Evaluation Criteria for Ecological Toxicity Data in the Open Literature. Phases I and II. Procedures for Identifying, Selecting, and Acquiring Toxicity Data Published in the Open Literature. Office of Pesticide Programs. July 16, 2004.

    USEPA. 2004b. Overview of the Ecological Risk Assessment Process in the Office of Pesticide Programs. USEPA Environmental Fate and Effects Division. Office of Pesticide Programs. Available at: Ecological Risk Assessment Process under the Endangered Species Act.

    USEPA. 2008. Memorandum on Guidance for Use of Dilution-Water (Negative) and Solvent Controls in Statistical Data Analysis for Guideline Aquatic Toxicity Studies. From Mary Frankenberry, Carolyn Hammer, and Keith Sappington to Donald Brady (EFED Division Director). September 25, 2008.

Attachments

  1. Attachment 1
    Phase I of the Interim Guidance (USEPA, 2004a): Literature Acceptance Criteria for Effects Data

    • I.1 Background on the ECOTOX Database

      Staff scientists in OPP will consider data from the open literature that the EPA Office of Research and Development's (ORD) Mid-Continental Ecology Division (MED) locate using the search strategy developed for EPA's Ecotoxicology database (ECOTOX). ECOTOX is a publicly available database summarizing the ecological effects of single chemicals to aquatic and terrestrial plants and animals (http://cfpub.epa.gov/ecotox/).

      • I.1.1 ECOTOX Eligibility Criteria

        Papers acquired using the ECOTOX literature searching and acquisition protocols must meet five minimum data requirements to be considered eligible for inclusion in the database. The paper is eligible for the ECOTOX database if it reports:

        1. toxic effects related to a single chemical exposure;

        2. toxic effects on an aquatic or terrestrial plant or animal species;

        3. a biological effect on live, whole organisms;

        4. a concurrent environmental chemical concentration/dose or application rate; and

        5. an explicit duration of exposure.

      • I.1.2 ECOTOX Literature Identification Procedures

        For the ECOTOX database, literature is identified through comprehensive and well-documented literature searches (See ECOTOX Database DATABASE OVERVIEW for standard operating procedures) either manually (e.g., skimming of reference sections of review/summary articles and skimming literature held in ORD/MED's library) or electronically (e.g., searches of electronic abstracting services such as STN, Cambridge Scientific Abstracts, Toxline). Quality Assurance/Quality Control (QA/QC) procedures for the ECOTOX database are outlined in the document Overview of the Ecological Risk Assessment Process in the Office of Pesticide Programs, U.S. Environmental Protection Agency (January 2004) and the support documents are itemized in Appendix I.

        • I.1.2.1 Manual Literature Searches

          For papers identified using a manual literature search method, citations for potentially applicable articles are entered into an electronic bibliographic file. Abstracts are not available in the electronic bibliographic file for these papers. As articles are received, papers that meet the minimum data requirements for ECOTOX are assigned an ECOTOX reference number and a copy of the paper is maintained in the ECOTOX file room. ORD/MED also maintains copies of some non-applicable papers that are useful to the ECOTOX database effort, and these are also assigned an ECOTOX reference number. These papers include review or summary articles that are not the primary source of data but are used in manual literature searches, and papers that document test method procedures. All papers that are received and identified as non-applicable, including any retained in the ECOTOX file room, have annotations added to the electronic bibliographic entry documenting the reason the paper was excluded from consideration for the ECOTOX database effort. Copies of non-applicable papers, other than methods and review articles, are discarded.

        • I.1.2.2 Electronic Literature Searches

          For papers identified using an electronic literature search method, the citation and abstract information are downloaded electronically into a bibliographic file. Each citation and abstract is reviewed to determine if the paper is potentially applicable to the ECOTOX database. When a paper is identified as not applicable to ECOTOX, the entry is annotated with the reason the study was excluded, and a copy of the paper is not acquired. The citation and abstract are moved to a separate electronic bibliographic file that stores citations for all papers identified via the ECOTOX electronic literature search method, which were never acquired. Citations for papers (excluding the abstract) that are identified as potentially applicable are moved to another electronic bibliographic file that tracks papers that have been ordered via the ORD/MED library or other sources, and the paper is acquired. As papers are received, the ECOTOX eligibility criteria are applied, and citations for any papers that are identified as not applicable to ECOTOX are transferred to another bibliographic file that tracks papers that were received and subsequently rejected. All citations for rejected studies are annotated with the reason for exclusion, and the paper is not retained. All papers that are received and identified as applicable are assigned an ECOTOX reference number and placed in the ECOTOX file room.

    • I.2 Identification of Relevant Literature

      The results that ORD/MED obtains through the ECOTOX literature search can be divided into two broad categories:

      • Papers identified as applicable to the ECOTOX database (i.e., those that meet the five minimum eligibility criteria); and

      • Papers located using the ECOTOX search strategy but rejected for entry into the ECOTOX database

      • I.2.1 Papers Eligible for the ECOTOX Database

        Within the category of papers identified as acceptable to the ECOTOX database effort, are two sub- categories:

        1. papers with data available in the online version of ECOTOX; and

        2. papers that are in the queue for data abstraction and inclusion in the ECOTOX database.

        All papers identified as applicable to the ECOTOX database, with the exception of the OPP database (ECOTOX Reference Number 344), will undergo an acceptability screen by ORD/MED to discriminate between papers that are potentially applicable to the OPP risk assessment (i.e., papers that meet all the criteria) from those papers that have a low probability of usefulness in the OPP risk assessment (i.e., papers that do not meet one or more of the criteria).

        • I.2.1.1 OPP Acceptability Criteria

          The OPP Technical Teams identified criteria that will assist in eliminating papers that have a low probability of usefulness in the hazard identification component of a screening level risk assessment. See Attachment I-A for the data form and guidance on determining the acceptability of an ecotoxicology study. Papers must meet all of the following Acceptability Criteria:

          • The paper reports toxicology information for a chemical of concern to OPP;

          • The article is published in English language;

          • The study is presented as a full article. Abstracts will not considered;

          • The paper is a publicly available document;

          • The paper is the primary source of the data;

          • The paper reports that treatment(s) were compared to an acceptable control;

          • The paper reports an explicit duration of exposure;

          • The paper reports a concurrent environmental chemical concentration/dose or application rate;

          • The paper reports the location of the study (e.g., laboratory vs. field);

          • The paper reports a biological effect on live, whole organisms;

          • The paper reports the species that was tested; and this species can be verified in a reliable source;

          • The paper reports effects associated with exposure to a single chemical.

        • I.2.1.2 Data Summary Tables

          Once the OPP acceptance criteria have been applied to all the papers meeting the ECOTOX eligibility requirements, ORD/MED will summarize the data associated with the potentially acceptable studies for use in the screening level risk assessment. For each paper that passes the Acceptance Criteria, ORD/MED will summarize the parameters listed below in separate fields within a Data Summary Spreadsheet, and provide the results in an electronic format to the appropriate staff person within OPP's Environmental Information Services Branch (EISB). See Attachment I-B for instructions on the completion of the Data Summary Table and an example table.

          • Chemical identification information

            • Chemical Abstract Services registry number
            • common name of chemical

          • Taxonomic information including:

            • phylum
            • class
            • order
            • family
            • genus
            • species
            • species common name

          • Effect information including:

            • major effect category (e.g., reproduction)
            • specific result measurement (e.g., number of viable eggs)
            • calculated endpoint (e.g., LC50)

          • Species habitat (aquatic and/or terrestrial)

          • Test location (laboratory vs. field)

          • Number of treatment levels

          • Concentration/dose/application rate associated with the observed effect response

            • Concentration units (e.g., µg/L)
            • Concentration type (e.g., active ingredient vs. formulated)
            • Percent purity

          • Media (freshwater vs. marine)

          • Duration of the study

            • Duration units (e.g., hours)

          • For avian and mammalian studies, identify the route of exposure (e.g., dietary, oral, etc.)

          • pH ranges

          • Hardness

          • Organic Carbon Concentrations

          • Remarks

          • ECOTOX reference number

        • I.2.1.3 Bibliographic Files

          The following procedures will be followed for all papers identified as acceptable to the ECOTOX database:

          • ORD/MED will provide OPP/EISB staff with a ProCite file that includes citations for all papers identified as acceptable to the ECOTOX database effort;

            • Each entry will include an ECOTOX reference number;

            • Each entry will include complete citation information;

            • Studies that pass the acceptance criteria will be marked as acceptable;

            • Each entry will identify the paper's coding status as it relates to the ECOTOX database (i.e., papers that are in the public version of the ECOTOX database and papers that are in the queue);

            • Each citation will identify whether the paper presents data for aquatic and/or terrestrial species;

            • Each citation will include a code identifying the chemical(s)

            For studies that did not pass the acceptance criteria, each entry will include a code that denotes the specific criterion/criteria that the paper did not meet.

      • I.2.2 Papers Excluded from the ECOTOX Database

        ORD/MED maintains electronic bibliographic files of all rejected papers that include codes documenting the reason the paper was rejected. See Attachment I-C for a list of exclusion criteria used within the scope of the ECOTOX database. The following procedures will be followed for these excluded studies:

        • ORD/MED will provide OPP with a ProCite database of citations and abstracts for papers that were rejected, and never acquired. The compilation will include the following:

          • Citations for papers that have been retained in the ECOTOX literature holdings, but that do not meet the eligibility requirements;

          • Citation for papers identified as potentially applicable to the ECOTOX effort, which were rejected once the full article was acquired;

          • Citation and abstract, when available, for publications excluded based on information provided in the literature search (i.e., without acquisition of the paper)

        • All records will include the following information:

          • The reason the paper was not considered applicable to the ECOTOX database;

          • All available citation information;

          • Abstracts when available (Note: These will only be available for electronic search results rejected prior to acquisition of the full publication.)

    • Attachment I-A
      Acceptability Criteria for Effects Data

      ECOTOX Reference No.: _________

      General Instructions:

      If more than one experimental design is used in the study, multiple Literature Acceptance Criteria Checklist forms may be required, but the acceptability of the paper is based on at least one experimental design meeting all the Acceptability Criteria.

      Literature Acceptance Criteria Checklist
      No. Criteria
      (Instructions)      		 Yes/No
      1

      The paper reports toxicology information for a chemical of concern to OPP.

      (Check chemical list for chemicals tested. If a chemical appears on the list answer Yes. If the chemical is not on the list answer No.)

      		  
      2

      The article is published in the English language.

      (The full article is published in the English language. Translations will not be conducted.)

      		  
      3

      The study is presented as a full article.

      (Abstracts from journal publications where the full article is published in non-English, the abstract is published in English and conference proceedings published as brief abstracts will not be considered.)

      		  
      4

      The paper is a publicly available document.

      (Publications that are not publicly available; i.e., internal memoranda, government reports that are not available from NTIS or other government sources will not be considered.)

      		  
      5

      The paper is the primary source of the data.

      (A document is considered a primary source if at least one of the investigators who conducted the toxicity test is an author, and the authors do not cite another publication as the original source of the data.)

      		  
      6

      The paper reports a calculated endpoint.

      (For the purposes of this evaluation, an endpoint is defined as the quantification of an observed effect obtained through statistics or other means of calculation for the expressed purpose of comparing equivalent effects (e.g., LC50, BCF, NOAEL). If within a single experiment, the authors report the same endpoint at multiple durations, only code the duration most relevant to the OPP SEPs (see Table below) and note the other endpoints in the remarks field. If NOEC/NOEL and LOEC/LOEL endpoints are not explicitly reported by the authors, reviewers should interpret endpoints based on levels of significance reported in the paper. If more than one measurement (e.g., juveniles per litter, juveniles per females, etc.) is observed for a particular effect (e.g., reproduction), then only the most sensitive measurement is coded, and the remaining measurements are noted in the remarks field.)

      		  
      7

      The paper reports that treatment(s) were compared to an acceptable control.

      (The control treatment must be comparable to the other treatments, and must be free of the chemical stressor. Appropriate controls include:

      • baseline or background control - parameters of actual or representative test species measured before and after administration of test chemical, though not as part of the same test scenario;
      • negative control - organisms maintained under conditions identical to exposed organisms except for the absence of the test substance;
      • positive controls - organisms maintained under conditions identical to the exposed organisms except the test substance is replaced with a substance known to elicit a consistent toxic response; and
      • solvent controls - organisms exposed to carrier or solvent that is used as a vehicle for administrating the test substance to exposed organisms.

      The number of treatments (other than controls) should be reported in the data summary table.)

      		  
      8

      The paper reports an explicit duration of exposure.

      (Authors must explicitly report the duration of the exposure related to the observed effect.

      Durations can include qualitative terms (e.g., at hatch, at harvest).)

      		  
      9

      The paper reports a concurrent environmental chemical concentration/dose or application rate.

      (Authors clearly report a concentration/dose or application rate associated with the observed effect response. If the study results are only available in a graphical format, add a comment to the remarks field.)

      		  
      10

      The paper reports the location of the study (e.g., laboratory vs. field).

      (Authors clearly state the locations of the study, either in a controlled laboratory setting or in the fields. Field studies can include natural or artificial settings (e.g., outdoor pen studies; enclosures).)

      		  
      11

      The paper reports a biological effect on live, whole organisms.

      (The authors clearly identify an observed effect response related to the exposure of a live organism to the chemical of concern. In vitro studies are not considered. Positive effects will be recorded, with a note in the remarks field flagging the response as such.)

      		  
      12

      The paper reports the species that was tested; and this species can be verified in a reliable source.

      (The authors clearly identify the test species and the organism's scientific name can be verified in a reliable reference. The preferred scientific name should be reported. If a specific genus/species is not reported, reviewers should code the species information at the lowest taxonomic level.)

      		  
      13

      The paper reports effects associated with a single chemical exposure.

      (Tests concerning synergism, additivity, potentiation, or antagonism, whether it involves two or more chemical stressors or a combination of chemical and non-chemical stressors are not acceptable studies under this criteria. Effluents, leachates, drilling muds, fly ashes, sediments, and sludges are not considered single chemicals. In addition, the single chemical cannot be introduced as a component of an effluent, etc.

      Formulated products, such as emulsifiable concentrates and wettable powders are considered single chemicals. Also, chemicals that are mixtures of structurally similar organic chemicals are considered single chemicals because the mixtures have similar biological, chemical, physical, and toxicological properties (e.g., toxaphene).)

      		  

    • Attachment I-B
      Completion of the Data Summary Table

      General Instructions

      All entries should be made in an Excel spreadsheet using version 9.0 (Excel 2000) or higher. Each sub-category within a general parameter must be data entered into a separate data field within the spreadsheet. Entries should be based on information provided by the authors, unless clarified in the instructions for each parameter.

      Instruction for Completion of the Data Summary Table
      No. Parameter Instructions
      1 Chemical Identification In separate data fields, enter the Chemical Abstract Services registry number and the chemical name as listed in Appendix III.
      2 Percent Purity Percent purity of compound tested as reported by the authors.
      3 Taxonomic Identification In separate data fields, enter the following taxonomic information: ECOTOX species number, phylum, class, order, family, genus, species, and the common name of the organism involved in the exposure.
      4 Habitat Identify whether the study is an aquatic or terrestrial study.
      5 Test Location In a separate data field, enter the test location using codes presented in the ECOTOX Code List.
      6 Media The type of exposure media reported using codes presented in the ECOTOX Code List.
      7 Route of Exposure Identify the route of exposure using codes presented in the ECOTOX Code List.
      8 Number of concentrations/doses Enter the number of treatment levels (other than controls) that were used in the study.
      9 Duration The exposure duration associated with the observed effect, as reported by the authors.
      10 Duration Units Units associated with the reported duration.
      11 Observed Effect and Calculated Endpoint

      In separate data fields, enter the major group effect, observed effect, and measurement codes presented in the ECOTOX Code List.

      In a separate data field enter the appropriate calculated endpoint using the codes presented in the ECOTOX Code List/a>.

      If more than one measurement (e.g., juveniles per litter, juveniles per female, etc.) is observed for a particular effect (e.g., reproduction), then only the most sensitive measurement is coded, and the remaining measurements are noted in the remarks field.

      If within a single experiment, the authors report the same endpoint at multiple durations, only code the duration most relevant to the OPP SEPs (see Table below), and note the other endpoints in the remarks field.
      12 Concentration Type Report if the chemical concentration is based on active ingredient or formulation. If chemical concentration is reported as measured, the concentration type should be coded as active. If chemical concentration is reported as nominal, concentration type should be coded as formulation. If the authors do not clearly state whether or not the chemical concentration was measured or whether the reported concentrations are based on formulated product vs. active ingredient, code as formulations, but make a comment in the remarks field that concentration type could not be determined from paper.
      13 Concentration, Dose, or Application Rate

      In a separate data field, enter the chemical concentration, dose, or application rate associated with the observed effect. If both a concentration and application rate are presented, only report the concentration. If a dose is presented and a concentration, only present the dose.

      In separate data fields enter the unit associated with the concentration/dose/application rate entry, the concentration type (F if formulated; A if active ingredient); and the percent purity of the compound as reported by the authors. If the authors clearly state that the compound is measured in the test matrix, enter the concentration type as "A".
      14 Remarks General comments on other endpoint durations, graphical data coding, concentration type, and positive effects.
      15 Reference Number In a separate field, enter the ECOTOX reference number for the paper.
      Example Data Summary Table
      CAS # Chemical Name Phylum Class Order Family Genus Species Common name Effect Meas Endp Media pH O.C Hardness Dur Dur Unit Conc Type Conc Conc Units Ref #
      298000 Methyl parathion Arthropoda Maxillopoda Calanoida Acartiidae Acartia tonsa Calanoid copepod BEH EQUL EC50 SW       96 h A 890 µg/L 742
      298000 Methyl parathion Arthropoda Insecta Diptera Culcidae Aedes nigromaculis Mosquito MOR MORT LC50 FW       24 h A 0.008 ppm 4431
      298000 Methyl parathion Chordata Actinoptergyii Siluriformes Ictaluridae Ameiurus melas Black bullhead MOR MORT LC50 FW       24 h F 7910 µg/L 6797
      298000 Methyl parathion Chordata Actinoptergyii Siluriformes Ictaluridae Ameiurus melas Black bullhead MOR MORT LC50 FW       96 h F 6640 µg/L 6797
      298000 Methyl parathion Chordata Actinoptergyii Cyprinodontiformes Poecillidae Gambusia affinis Western mosquitofish BEH THML NR FW       48 h A 5 µg/L 5149
      298000 Methyl parathion Chordata Actinoptergyii Cyprinodontiformes Poecillidae Gambusia affinis Western mosquitofish MOR MORT NR FW       24 h F 5000 µg/L 7762

    • Attachment I-C
      ECOTOX Exclusion Categories

      General Instructions:

      The following is a list of ECOTOX rejection codes, exclusion terms and definitions utilized under the ECOTOX database efforts. Each citation that is identified as not applicable to the ECOTOX database (i.e., NON-APPLICABLE entered in field # 37 of the ProCite bibliographic file) will have one or more of these codes entered in the KEYWORD field (number 45) of the ProCite bibliographic files to identify the reason the study was excluded.

      ECOTOX Exclusion Categories
      Code Used in ProCite File Description
      REVIEW All toxicity tests reported elsewhere. If the publication is applicable to one of the ECOTOX databases, the bibliography is skimmed and any applicable articles are ordered.
      METHODS Methods paper with no usable toxicity tests. Reports of methods of conducting tests, determination or purification of chemicals, etc. Methods publications are selected to be ordered for the ECOTOX toxicology methods information file.
      PUBL AS Author publishes data in multiple formats. For instance, an author publishes the same study results in a Ph.D. thesis and in a peer-reviewed journal. In order to reduce redundancy of data in the ECOTOX database, only one document is encoded; with preference given to peer-reviewed publications.
      NON-ENGLISH or FORE Paper is published in a foreign language.
      Modeling Paper reports models only, no new organism exposure data. Modeling studies may report original toxicity tests performed as comparisons or as a basis for extrapolation; order the paper if it is not clear from the abstract.
      Air pollution Air pollution and emissions, no specific chemical.
      Other ambient conditions Effects on organisms from changes in conditions other than addition of chemicals, including radioactivity, ultraviolet light (UV), temperature, pH, salinity, dissolved oxygen (DO), or other water, air, or soil parameters.
      Biological Toxicant Biological toxicants; includes venoms, fungal toxins, Bacillus thuringiensis, other plant, animal, or microbial extracts or toxins.
      Drug Testing for drug effects and side-effects specific to human health.
      Effluent Studies reporting effects associated with exposure to effluent, sewage, or polluted runoff.
      Mixture Mixture study without data on single chemical exposures.
      Nutrient Nutrient studies--in situ chemicals tested as nutrients.
      Oil Studies with oil and petroleum products as the only test chemical.
      No Species No organism present or tested, unable to verify a species, or exposure of dead organism.
      In Vitro In vitro studies, including exposure of cell cultures and excised tissues.
      Bacteria Bacteria as test organism, including Microtox tests, or other microbial organisms.
      Yeast Yeast as the test organism.
      No Tox Data Publications which are not toxicology studies; no toxicity data are presented in the paper.
      Human Health Human health effects; studies with human subjects or with animal subjects as surrogates for human health risk assessment.
      No Conc No usable dose or concentration reported; identified after examination of full paper. Includes lead-shot studies which lack dose information or give only number of pellets. Also includes papers with concentrations reported only in log units.
      Sediment Conc Chemical concentration reported in sediment. Sediment studies are coded if a water concentration of the added chemical is also reported; order the publication if unclear from the abstract.
      No Duration Duration is not reported, identified after examination of full paper.
      Incident Report of incidents--reports of animal deaths by poison, etc. Lacks usable concentration or duration or both.
      Survey Survey studies--measuring amounts of chemical present, but no usable quantification of exposure. Lacks either usable concentration or duration or both.
      Fate Studies reporting only what happens to the chemical in abiotic matrices
      Food Studies Food studies, no chemical and effects information are reported

  2. Attachment 2
    Instructions for Sorting and Formatting the ECOTOX Excel Spreadsheet for Inclusion as an Appendix to the Risk Assessment

    Step 1- Make a Duplicate Worksheet with All the Data in it and Sort Data by Effect

    • Your excel spreadsheet comes with a single worksheet called "dynamic1". The name of the worksheet is on the bottom left of your screen on a tab.

    • Put the cursor over the name of the worksheet - i.e., over the words "dynamic1" and right click.

    • A list appears, highlight "Insert..." and click.

    • Choose "worksheet" and click ok. This will insert a worksheet called "sheet 1".

    • Go back into the original worksheet, "dynamic 1", and press the "control" button and the "a" button together - this will select everything in the worksheet.

    • When everything is highlighted blue to denote that everything has been selected, press the "control" button together with the "c" button - this will copy all the information.

    • Click on "sheet 1", which is the blank worksheet that you just created. Put your cursor over cell A3 and click on cell A3. Now press the "control" button and the "v" button at the same time. This will paste all the information you just copied. Basically, you just made a duplicate worksheet so you won't lose any information if you need it later.

    • Sort by Effect: On "sheet 1" (the worksheet you just created), while all of the data are still highlighted, go to "Data" on the file menu; select "Sort"; in the "Sort by" window, select "Effect", "Ascending", and then click "OK" (your data are now sorted by effect).

    • Click on cell B1, and type the title of the appendix (e.g., APPENDIX G: Accepted ECOTOX Data Table).

    • Click on cell B2 and type the following: The code list for ECOTOX can be found at: https://cfpub.epa.gov/ecotox/pdf/codeappendix.pdf.

    Step 2. Delete Columns in the New Worksheet

    • Open the excel file that contains your Ecotox Data.

    • Save the file with a new name so you don't ruin the original file.

    • Click on "File", then "Save As" - you just have to select the location where you want to save the document, its format (in "Save as type" select "Microsoft Office Excel Workbook" and click "save"), and give it a file name.

    • To delete a column, put your cursor over the column heading (e.g., the "A" or "B" on top of the column) and right click, which will highlight the column, then select "delete":

    • Delete all columns except the following:

      Keep:
      (Ref#)
      (% Purity)
      (Conc Units Preferred)
      (Conc #2 Purity Adjusted in Preferred Unit Mean)
      (Conc #1 Purity Adjusted in Preferred Unit Mean)
      (Dur Preferred Mean)
      (Dur Unit Preferred)
      (Genus through Endpt2; 8 columns)
      (Chemical Name)

    All other columns are to be deleted.

    • Click on "file" in the upper left part of the page.

    • Click on "page setup".

    • A box will appear with four tabs (page, margins, header/footer, and sheet).

    • On the "page" tab, click on "Landscape" button if it is not already highlighted in Green.

    Step 3. Set up the page breaks

    • Click on "View" in the upper left part of the page.

    • Click on "page break preview"; click "OK" in the "Welcome to page break preview".

    • - Blue lines will show up -

    • Click on the first vertical dashed blue line and drag it to just past the last column with data (column P or so).

    • If there is another vertical line, then repeat this step until there are no dashed vertical lines between Column A and the end of the sheet.

    • There should be a giant "Page 1" on your screen. There should be no page number next to the page 1 - there should only be increasing page numbers as you move down the sheet, not as you move across to the right or left.

    Step 4. Set up the page settings

    • Click on the "sheet" tab in the page setup menu.

    • Next to the "Rows to repeat at top" input click on the square that is gray, red, and blue.

    • Highlight ("click on" row 3) then press the gray box again.

    • Under "Page Order" also under the "sheet" tab, click on the "over, then down" circle, so it turns green (a picture of a green "Z" looking arrow will show up).

    • Click OK.

    Step 5. PDF the sheet (okay to print without tags)

    This can be accomplished by either:

    • clicking on the "convert to pdf" button (red white button on/near the top right corner of the worksheet);

    • Click on Adobe PDF in the menu on the top of the screen then choosing "convert to adobe pdf";

    OR

    • Click on File in the menu bar, then select print - then choose "adobe PDF" as the printer.

    • In all cases, you will need to choose a file name and a location to save your file to.

  3. Attachment 3
    Blank Summary Table of Lowest Toxicity Values from FIFRA Guideline Studies

    Pesticide Name:

    PC Code:

    Lowest Toxicity Values from FIFRA Guideline Studies
    Kingdom Habitat Taxon Effect_Type Endpoint Value Unit
    Animal Aquatic Freshwater fish Acute LC50   mg/L
    Freshwater invertebrate EC50   mg/L
    Marine/estuarine fish LC50   mg/L
    Marine/estuarine crustacean EC50   mg/L
    Marine/estuarine mollusk EC50   mg/L
    Freshwater fish Chronic NOAEC   mg/L
    Freshwater invertebrate NOAEC   mg/L
    Marine/estuarine fish NOAEC   mg/L
    Marine/estuarine crustacean NOAEC   mg/L
    Marine/estuarine mollusk NOAEC   mg/L
    Terrestrial Avian Subacute dietary LC50   mg/kg-diet
    Avian Acute oral LD50   mg/kg bw
    Rodent Acute oral LD50   mg/kg bw
    Honeybee Acute contact LD50   µg/bee
    Avian Chronic NOAEC   mg/kg-diet
    Rodent Chronic NOAEC
    NOAEL    		   mg/kg-diet
    mg/kg-bw
    Plant Aquatic Vascular aquatic plant
    (non-listed)    		 Acute EC50   mg/L
    Vascular aquatic plant
    (listed)    		 NOAEC or EC05   mg/L
    Nonvascular aquatic plant
    (non-listed)    		 EC50   mg/L
    Nonvascular aquatic plant
    (listed)    		 NOAEC or EC05   mg/L
    Terrestrial Terrestrial monocot
    (non-listed)    		 EC25   lbs/A
    Terrestrial monocot
    (listed)    		 NOAEC or EC05   lbs/A
    Terrestrial dicot
    (non-listed)    		 EC25   lbs/A
    Terrestrial dicot
    (listed)    		 NOAEC or EC05   lbs/A

  4. Attachment 4
    Explanation of OPP Acceptability Criteria and Rejection Codes for ECOTOX Data

    Studies located and coded into ECOTOX must meet acceptability criteria, as established in the Interim Guidance of the Evaluation Criteria for Ecological Toxicity Data in the Open Literature, Phase I and II, Office of Pesticide Programs, U.S. Environmental Protection Agency, July 16, 2004. Studies that do not meet these criteria are designated in the bibliography as "Accepted for ECOTOX but not OPP." The intent of the acceptability criteria is to ensure data quality and verifiability. The criteria parallel criteria used in evaluating registrant-submitted studies. Specific criteria are listed below, along with the corresponding rejection code.

    • The paper does not report toxicology information for a chemical of concern to OPP; (Rejection Code. NO COC)

    • The article is not published in English language; (Rejection Code: NO FOREIGN)

    • The study is not presented as a full article. Abstracts will not be considered; (Rejection Code: NO ABSTRACT)

    • The paper is not publicly available document; (Rejection Code: NO NOT PUBLIC (typically not used, as any paper acquired from the ECOTOX holding or through the literature search is considered public)

    • The paper is not the primary source of the data; (Rejection Code: NO REVIEW)

    • The paper does not report that treatment(s) were compared to an acceptable control; (Rejection Code: NO CONTROL)

    • The paper does not report an explicit duration of exposure; (Rejection Code: NO DURATION)

    • The paper does not report a concurrent environmental chemical concentration/dose or application rate; (Rejection Code: NO CONC)

    • The paper does not report the location of the study (e.g., laboratory vs. field); (Rejection Code: NO LOCATION)

    • The paper does not report a biological effect on live, whole organisms; (Rejection Code: NO IN-VITRO)

    • The paper does not report the species that was tested; and this species can be verified in a reliable source; (Rejection Code: NO SPECIES)

    • The paper does not report effects associated with exposure to a single chemical. (Rejection Code: NO MIXTURE). It should be noted that all papers including data on pesticide mixtures are considered.

    Additionally, efficacy studies on target species and toxicological data on a subset of target species (i.e., plants and terrestrial insects excluding bees, butterflies, beetles, and non-insect invertebrates such as soil arthopods and worms) are excluded and coded as NO TARGET.

    Data that originated from the OPP Pesticide Ecotoxicity Database is coded as NO EFED. These data are already available to the chemical team.

  5. Attachment 5
    Guidelines for Invalidation of Open Literature Studies

    The following list of invalidation guidelines should be considered in the review of open literature papers that pass the screening steps described in Section 2. While an effort was made to list the majority of factors that would invalidate an open literature study, it should be noted that the following list is not exhaustive. In addition to invalidation guidelines, the subsection on honey bee studies also includes factors that should be considered in the review of open literature studies on pollinators.

    General

    1. Studies that do not report a control and associated data are invalid. In addition, the controls must be run concurrently with treatments during the study.

    2. Control mortality is high (see the appropriate comparable guideline study SEP and/or 850 guideline) and/or test species show signs of stress/disease. In general, control mortality should not exceed 10% for most test species in acute studies. For honey bees, control mortality should not exceed 20% in laboratory and/or field studies.

    3. Although tests using wild-caught animals with unknown previous exposure histories are acceptable provided their source is not in agricultural areas where prior exposure to pesticides is likely, test organisms that are reported as having been previously exposed to the test material or other contaminants are invalid.

    4. In general, if the study reports the presence of test material in the control and measured concentrations of the test material in the control exceed the LOD, the study is invalid because the ability of the study to discriminate a treatment effect may be compromised. Rare exceptions may occur for naturally occurring test materials (e.g., copper, various salts) and in situations where the presence of exceptionally low concentrations of the test material in controls can be discounted given acceptable performance in the controls and in treatment groups with concentrations orders of magnitude higher than the control.

    5. If the study does not provide proper identification of the test material in terms of the percentage of active ingredient, trade name, and/or TGAI, the study is invalid. The reviewer should also ensure that the tested product and/or formulation are currently registered for use and that the endpoints have been adjusted to account for the percentage a.i., if appropriate.

    6. If reported measured concentrations deviate more than 20% (i.e., variability between treatment group measured concentrations is sufficiently high to render treatment group measured concentrations statistically indistinguishable, and/or overlapping concentrations occur between different treatments), the test may be classified as invalid. The following factors should be considered when reviewing a study with high variability between treatment group measured concentrations:

      1. the treatment group with high variability is not an endpoint of concern (i.e., different than the NOAEC or LOAEC value);

      2. the dose-response is strong despite the +/- 20% variance;

      3. the frequency and duration of occurrence (i.e., variance at a low frequency vs. all the time);

      4. the study authors have provided justification for variability in measured concentrations and identified all measures taken to mitigate the problem; and

      5. the duration of the study (i.e., < 20% variability is rarely achieved in fish full life cycle studies).

    7. Concentrations of the test material are not adequately measured (i.e., only nominal test concentrations are available) for chronic studies, field studies, and acute studies with materials that are not stable and/or have low solubility.

    Aquatic Studies

    • 8. If reported, aquatic studies that use distilled or deionized water used as dilution water without reconstitution (addition of appropriate salts) are invalid.

    • 9. If reported, the concentration of organic solvent in test solution should not exceed 0.1 mL/L for acute and/or chronic invertebrate studies or acute fish flow-through studies; the solvent should not exceed 0.5 mL/L for acute fish static or static-renewal studies (see the appropriate comparable guideline study SEP and/or 850 guideline for additional solvent criteria; note: there may be some flexibility in this criteria if controls appear to be functioning normally). Dimethyl sulfoxide is not considered an appropriate co-solvent.

    • 10. Generally, for water column tests, mean dissolved oxygen concentrations should not drop below 60% saturation for prolonged periods (if reported), unless justification can be provided that dissolved oxygen suppression does not interfere with the interpretation of the study results.

    • 11. Except for tests using saltwater annelids and mysids, results of acute tests during which test organisms were fed should not be used, unless data indicate that food did not affect the toxicity of the test material and/or the test material has a low Kow value (< 2). For compounds with a log Kow less than two, the presence of food is assumed not likely to significantly alter the dissolved concentration or bioavailability of the test material. For pesticides with log Kow values between 5 and 7, laboratory toxicity data should be carefully reviewed to ensure that feeding regimes are eliminated to minimize any effects from interaction of the pesticide with food particles (e.g., reduction of test solution concentration as a result of partitioning into the food particles, or introduction of a dietary exposure route if animals ingest food that has sorbed to the pesticide).

    • 12. Results of acute tests conducted in dilution water with total organic carbon or particulate matter exceeding 5 mg/L should not be used, unless a relationship between acute toxicity and organic carbon or particulate matter has been established and/or data show that organic carbon, particulate matter, etc. do not affect exposure/toxicity.

    • 13. If reported for aquatic studies, biological loading must be suitable for the test species and container size and should not compromise water quality during the study (see the appropriate comparable guideline study SEP and/or 850 guideline for specific biological loading criteria).

    • 14. For aquatic studies, tests conducted in plastic test chambers (test vessels constructed from materials other than glass) without measurement of test material should be invalid unless it can be confirmed that the test material is stable and/or has a low Kow value (< 2). It should be noted in the OLRS that tests conducted in plastic test chambers are problematic because there is the potential for plasticizers to leach into the dilution water.

    • 15. Issues related to significant differences between solvent and negative controls (see policy memo at: G:\Teams and Panels\Tech Teams\Aquatic Biology(ABTT)\Solv Neg Control Guidance Memo-signed.pdf) should be considered when classifying a study as invalid.

    • 16. If the reported water temperature is outside of the range of values that are acceptable for the given test species (see the appropriate comparable guideline study SEP and/or 850 guideline for species-specific recommended test temperatures), the study is invalid.

    • 17. For aquatic test systems containing sediment, lack of reported information on the organic carbon content and composition of the sediment may result in a study classification of "invalid", depending on the chemical properties of the test substance.

    • 18. For acute oyster shell deposition tests, a minimum of 2 mm of shell growth should be observed in the control oysters.

    • 19. In aquatic plant studies, control plants should be maintained under the same test conditions as plants exposed to the test material.

    • 20. The population of test plants and/or replicates should be of sufficient size to characterize the effects with an acceptable degree of certainty.

    • 21. Test treatments should be randomly assigned to individual test chambers or the group of test chambers constituting a replicate, and the test chambers or replicates should be randomly assigned to locations. In addition, all test vessels and closures should be identical.

    • 22. A minimum of five test concentrations should be used in the definitive test to bracket the endpoint of concern for studies with aquatic vascular plants, algae, and cyanobacteria.

    • 23. Temperature and light intensity should be measured as specified during the test for studies with aquatic plants, algae, and cyanobacteria.

    • 24. The doubling time of the number of fronds in the control should not exceed 2.5 days for studies with Lemna sp.

    • 25. The lowest test concentration level for studies with Lemna sp. should be less than the 7-day yield and average specific growth rate IC50 values based on number of fronds and frond area (dry weight or frond area). For studies with algae, the lowest test concentration level should be less than the 96-hour yield, average specific growth rate, and area under the growth curve IC50 values based on cell density. For studies with cyanobacteria, the lowest test concentration level should be less than the 96-hour IC50 values and at or below the NOEC values for yield, average specific growth rate, and area under the growth curve based on cell density.

    • 26. For studies concerning algae, during the 96-hour test period, cell counts in the controls should increase by a factor of at least 100 times for P. subcapitata and a factor of at least 30 times for S. costatum (i.e., logarithmic growth in the controls should be reached during the test).

    The following types of aquatic open literature studies should be classified as "invalid", unless they are the most sensitive endpoint identified for a particular taxa: The rationale for including data from these types of studies is because the reported nominal value from an open literature study that is the most sensitive or lowest endpoint provides a margin of safety relative to the higher, less sensitive endpoint, regardless of the lack of information on measured concentrations over the duration of the study. Inclusion of measured concentrations from these studies would only result in a lower endpoint than the endpoint based on the nominal concentration.

    • 27. The test material has low solubility as evidenced by precipitates or films in/on the water without centrifuging and/or filtering of the water samples prior to analysis.

    • 28. Aerated treatment units where the compound is likely to be volatile (without measured concentrations).

    Avian Studies

    • 29. Regurgitation during the study (unless a reasonable effort to resolve the problem of regurgitation was made during the study).

    • 30. In an acute oral toxicity study, the length of time birds were fasted prior to dosing is not reported.

    • 31. In an acute or sub-acute study, the birds laid eggs.

    • 32. In a reproduction study, there are ≥ 13% cracked eggs in control birds.

    • 33. In a reproduction study, the productivity in the control groups does not average 12 (bobwhite quail) or 10 (mallard duck) 14-day old survivors per pen over a 10-week period.

    • 34. In a reproduction study, the average egg shell thickness is < 0.19 mm for bobwhite quail and < 0.34 mm for mallard duck.

    • 35. Birds are not in good health (e.g., are deformed, abnormal, sick, or injured) at the start of the study.

    • 36. Excessive loading rates (see OPPTS 850.2100, 850.2200, and 850.2300 Guidelines).

    • 37. Poor feed consumption in controls (see OPPTS 850.2200 and 850.2300 Guidelines).

    Terrestrial Invertebrate/Honey Bee Studies

    • 38. Disease control efforts not adequately described.

      Where existing colonies are used in a study [as opposed to new worker/queen bees (aka "package" bees) with new hive construction], disease may be a recurrent problem. Disease can include parasites such as mites (trachael and varroa), parasites (Nosema spp.) and fungal and bacterial diseases. The study should report general bee health and whether any diseases are being treated and if so, what is being used to treat the disease. Prior disease treatments such as coumaphos, fluvalinate and amitraz should also be reported if used. Failure to properly identify the health of the colony and disease control measures can invalidate a study. Where disease control measures have been applied, control bees must also account for these measures; otherwise, the study is invalid.

    • 39. Bee cross-over.

      Colonies placed in close proximity to one another may result in bees from one hive crossing into another from a different treatment. Where this potential exists, the study should report the results of a mark and recapture to determine the extent to which cross-over is occurring.

    • 40. Unsuitable replication.

      Sufficient replication must be available to adequately test for treatment effects. Although not required, a positive control can test whether the test system is capable of detecting effects. Insufficient replication can render the test of little value.

    • 41. Nonrandom distribution of test organisms.

      Package bees should ideally be from sister queens (queens produced from the same queen [genetics]) and should be randomly assigned to treatment units. Failure to do so can invalidate a study.

    • 42. Excessive bee mortality in controls.

      If reported, mortality measured using a cage outside the hive entrance (referred to as dead zone dead bee [DZDB] traps) should be calibrated and mortality reported. DZDB traps should be checked daily to ensure accurate estimates of mortality. Excessive mortality in controls (> 20%) can serve as a rationale for invalidating the study.

    • 43. Control colony not "queen right".

      Determinations should be made on the status of the queen. If the colony senses that a queen is failing, they will attempt to raise new queens in a process referred to as supercedure. Signs of supercedure (queen-rearing cells on the margin of larvae [brood]-rearing cells) can be used as an indication that the queen is not functioning properly. Number of new brood and presence/absence of the queen (colony with a queen is referred to as "queen right") should be reported as a measure of queen activity. Colonies should be monitored for the number of drone cells; male (drone) queens in controls suggests that the colony is under stress and is likely to collapse. While the loss of a queen is not unusual, unless a queen can be quickly restored, the colony will likely fail.

    • 44. Availability of alternate nectar/pollen sources.

      The proximity of alternate sources of pollen/nectar beyond the treated site should be carefully considered. Bees can forage up to 5 miles, and while they tend to focus on areas where pollen/nectar are abundant, readily accessible alternate forage areas should be minimized to prevent off-site foraging. Pollen can be analyzed to determine its origin (plant species).

    • 45. Improper documentation of exposure.

      Free foraging bee studies using a treated crop should properly document exposure. This can be done by measuring residues in pollen using pollen traps at the hive entrance and through residues in nectar (collected from hive frames or from the honey stomach of bees). Ideally, pollen should be analyzed to demonstrate that it is from the treated plant and contains residues of the test material. In tunnel/cage feeding studies, the treatment solution and/or treated honey/pollen (aka bee bread) cake should be subject to chemical residues analysis for the test material and its degradates. Also, broad spectrum residue analyses should be included for free foraging bees to document previous/concurrent exposures. Failure to properly document exposure can render the study invalid. Because wax can serve as a depot for lipophilic compounds, residues should also be reported for the wax as well.

    • 46. Sublethal effects not reported.

      For both acute and chronic studies, investigators are required to report sublethal (e.g., bee behavior, incidence of disease, proboscis extension reflex, visual discrimination tests) effects. Minimally, the study should report that no sublethal effects were observed during the study period. Any effects that did occur must be reported. Studies failing to document the presence/absence of sublethal effects should be classified as "qualitative" at best.

    • 47. Use of immature hives.

      Bee studies relying on package bees should minimally be afforded 6 - 8 weeks to acclimate. This acclimation period will allow several cohorts to develop and will ensure that forage bees are available for the study and will equalize colony strength.

    • 48. Adequate diet.

      Bees fed on pollen alone over a protracted study can exhibit signs of malnutrition as bees need both pollen [from a variety of sources] and nectar to survive. Ideally, bees should be fed a protein supplement that is more likely to be nutritionally balanced rather than to rely on one particular pollen source. Poor diet can lead to poor colony performance and will limit the study's ability to detect treatment effects. Ideally, sucrose solution and not honey should be used; if the latter is used, then a broad spectrum residue analysis should be conducted on the honey to document contaminants that may be present.

    • 49. Duration of study.

      Prolonged cage studies can substantially stress bees that are confined to the cage. This is particularly true for bees that have been accustomed to free foraging. Mortality of caged bees can be reduced by first acclimating the bees to the cage/tunnel; however, colony performance measures must be reported to gauge whether the colony is stressed by confinement. Failure to account for the duration of the study on colony performance can invalidate a study.

    • 50. For more information on bee biology and the terminology used in this section, refer to:

      http://pubs.cas.psu.edu/FreePubs/pdfs/agrs93.pdf

    Terrestrial Plant Studies

    • 51. Many of the same issues for animal studies may be applied to terrestrial plant studies. For example, contamination of the controls, inadequate measures of exposure or excessive control mortality are reasons to consider a study invalid.

    • 52. Although a complete randomized block design is preferred, the reviewer should bear in mind that plant studies, particularly field studies, may not be conducive to complete randomization. Reviewer discretion should be used.

    • 53. All studies should be conducted on typical end-use product (TEP). Use of TGAI may not represent the true phytotoxicity of a given chemical due to inherent plant defense mechanisms (e.g., waxy coatings, hair-like structures (trichomes), scales, etc.) on leaf surface.

    • 54. Validity elements detailed in the current 850.4225 (seedling emergence) and 850.4250 (vegetative vigor) should be considered when reviewing terrestrial plant studies identified in open literature.

  6. Attachment 6
    Open Literature Review Summary

    Chemical Name:

    CAS No:

    PC Code:

    ECOTOX Record Number and Citation:

    Purpose of Review (Note: DP Barcode required for Quantitative studies):

    Date of Review:

    Summary of Study Findings:

    Description of Use in Document (QUAL, QUAN, INV):

    Rationale for Use:

    Limitations of Study:

    Primary Reviewer:

    Secondary Reviewer (required if study results are used quantitatively):

  7. Attachment 7
    EFED Policy Memo on Reviews of Scientific Literature from Steven Bradbury to EFED Staff
    (dated March 20, 2007)

    MEMORANDUM

    SUBJECT: Reviews of Scientific Literature

    FROM: Steven Bradbury, Director /original signed by S. Bradbury, March 20, 2007/
    Environmental Fate and Effects Division

    TO: EFED All Employees

    In order to ensure an efficient and consistent process for documenting review of the ECOTOX open literature, EFED staff should complete the attached open literature summary review form for all open literature studies that are used in ecological risk assessments and endangered species effects determinations. The attached review form was originally included in the July 2004 Interim Guidance of the Evaluation Criteria for Ecological Toxicity Data in the Open Literature (USEPA, 2004a), and has been slightly modified to include tracking of the CAS No., purpose of the review, and the name of the secondary reviewer.

    Secondary review of open literature summaries are necessary when the data are used quantitatively (i.e., to derive RQs). Review summaries of open literature data that are used quantitatively should include all available information that would normally be included as part of the current guideline/non-guideline DER templates. Review summaries of open literature data that are used qualitatively or determined to be invalid must also be documented using the attached form, and must include a description of the study limitations which preclude its quantitative use. Further guidance on screening open literature in an efficient and consistent manner for use in ecological risk assessments is provided in the Open Literature Guidance Document (USEPA, 2004a) and the Overview Document (USEPA, 2004b).

    Completed open literature review summary forms should be saved with corresponding electronic file names that begin with the letter "e" followed by the ECOTOX record number. For example, the file name of the study review for ECOTOX paper 81457 would be "e81457". Electronic files of completed open literature study reviews should be emailed to Cara Dzubow (and cc'd to Jerry Johnston). Completed open literature study reviews will be converted into ".pdf" files and copied to the SANS drive, where they will be included as the first page in the study file, followed by the actual study.

    Completing and submitting study reviews for open literature is important to ensure we keep our records current and complete. Also, the ability to see that a document has been reviewed, by whom and the conclusions reached in that review has the potential to save duplication of work and significant resources for the division.

    Attachment

    Questions about the review form and guidance for reviewing open literature can be directed to the Red-legged Frog Steering Committee.

    U.S. EPA. 2004. Overview of the Ecological Risk Assessment Process in the Office of Pesticide Programs. Office of Prevention, Pesticides, and Toxic Substances. Office of Pesticide Programs. Washington, D.C. January 23, 2004

    U.S. EPA. 2004a. Interim Guidance on the Evaluation Criteria for Ecological Toxicity Data in the Open Literature: Phases I and II: Procedures for Identifying, Selecting, and Acquiring Toxicity Data Published in the Open Literature For Use in Ecological Risk Assessments. Office of Pesticide Programs. Washington, D.C. July 16, 2004.

Contact Us to ask a question, provide feedback, or report a problem.
Last updated on October 31, 2024