Advancing Systematic Review Workshop (December 2015)
The purpose of the workshop is to examine developments and applications of methods for the identification and evaluation of different types of evidence (epidemiology, animal toxicology, and mechanistic) for use in chemical assessments.
EPA is developing and implementing a systematic review process within the IRIS Program to increase the transparency and clarity related to identifying and evaluating evidence and drawing conclusions in ongoing and future assessments. This workshop will serve as a follow-up to two previously held systematic review workshops held in August 2013 and October 2014. The August 2013 workshop examined the state-of-the-science of systematic review methodology, focusing on recent developments as the methodology has expanded outside the arena of evidence based medicine. The October 2014 workshop focused on recommendations in the May 2014 National Research Council report related to systematic review and evaluation of the risk of bias, integration of evidence streams, and combining quantitative results from multiple studies.
The December 2015 workshop examined developments and applications of methods for the identification and evaluation of different types of evidence (epidemiology, animal toxicology, and mechanistic). Two sessions were held: (1) From Theory to Practice: Lessons Learned From the Evaluation of Studies of Chemical Exposure, and (2) Systematic Review Relating to Mechanistic Data: What is Really Needed, and How can it be Efficiently Applied? This workshop built on some of the sessions from the previous workshops and further addressed the practical application of systematic review methodologies to specific elements of IRIS assessments.
The December 2015 workshop includes presentations and discussions by scientific experts in areas pertaining to systematic review. The workshop was structured to include an opportunity for comments, questions, and engagement from stakeholders and members of the public.
To promote transparency, speakers were requested to disclose: (1) The nature of any financial relationships they may have with any organization(s) or entities having an interest in issues under discussion (e.g., consulting agreements, expert witness support, or research funding); and (2) The extent to which their comments were reviewed by an interested party.
The workshop was held on December 16 - 17, 2015.
The meeting was held in the EPA Conference Center at 2777 South Crystal Drive, Arlington, Virginia 22202. The meeting was also available by webinar.
The purpose of the workshop was to examine recent developments and applications of methods for the identification and evaluation of different types of evidence (epidemiology, animal toxicology, and mechanistic). Two sessions were held: (1) From Theory to Practice: Lessons Learned From the Evaluation of Studies of Chemical Exposure, and (2) Systematic Review Relating to Mechanistic Data: What is Really Needed, and How can it be Efficiently Applied?
Session 1: From Theory to Practice: Lessons Learned From the Evaluation of Studies of Chemical Exposure
Considerable efforts have been undertaken to develop and refine systematic review methods that address the scope, purpose, and designs of studies examining potential health effects or toxicities from exposure to environmental chemicals. Much of this work has built upon the methods and resources of the Cochrane Collaboration, such as recently-released methods designed specifically for the assessment of risk of bias in non-randomized studies of interventions (ACROBAT-NRSI). This session will use the ACROBAT-NRSI as a starting point to examine developments in methods used to evaluate studies. Discussion topics:
- In what ways do the differences between evaluating “interventions” and evaluating “exposures” necessitate different approaches to assessing bias?
- What insights can be drawn from the experience of various groups using or adapting ACROBAT-NRSI?
- Does "risk of bias", as currently conceptualized and used, adequately encompass the breadth of issues needed for evaluation of studies of chemical exposures in observational epidemiology and in animal toxicology studies? If not, what approaches can be used to appropriately expand the focus?
Session 2: Systematic Review Relating to Mechanistic Data: What is Really Needed, and How can it be Efficiently Applied?
Evaluations of the potential mechanisms underlying the toxic effects of chemicals can lend support to determinations of hazard in humans. Approaches to the comprehensive and systematic identification and evaluation of mechanistic data that ensure efficiency and transparency in this process are currently being considered. This is a particular challenge for in vitro studies, given their heterogeneity and, in many cases, the large number of studies/assays. This session will examine approaches to sorting, screening, and evaluating mechanistic evidence, drawing upon use of key characteristics of carcinogens as an organizing principle. Discussion topics:
- What methods can be used to prioritize the relevant mechanistic studies that will most directly address questions on mechanistic events and pathways, i.e., mode of action?
- What is the utility of evaluating in vitro studies for bias and what approaches are available?
- Can the model of using key characteristics of carcinogens as an organizing principle be used for mechanistic evidence for other hazards (e.g., neurotoxicity, reproductive and developmental toxicity)?
- Are there frameworks or approaches such as adverse outcome pathways (AOPs) that may be useful for prioritizing the most relevant mechanistic studies for addressing questions on mechanistic events and pathways, i.e., mode of action for cancer and noncancer effects?
Comments on the workshop may be submitted to EPA_Sys-Review@icfi.com.
- (3 pp, 452 K, About PDF).
Session 1: Presentations
- Session 1: A Tool for Evaluating Risk of Bias in Non-Randomized Studies of Interventions (ROBINS-I): Development and Applications (PDF) (36 pp, 388 K, About PDF)
- Session 1: Applicability of ROBINS-I to Studies of Environmental Exposures (PDF) (23 pp, 252 K, About PDF)
- Session 1: Risk of Bias and Causality Assessments (PDF) (17pp, 314 K, About PDF)
- Session 1: Adaptation of Risk of Bias Assessment for Environmental Exposures: The IRIS Experience (PDF) (12 pp, 636 K, About PDF)
- Session 1: Methods Development for Bias Assessment of Environmental Exposures for Male Reproductive Outcomes (PDF) (10 pp, 119 K, About PDF)
- Session 1: Methods Development for Measurement of Environmental Exposures of Diabetes and Diabetes-Related Outcomes (PDF) (11 pp, 334 K, About PDF)
- Session 1: Methods Development for Assessment of Environmental Exposures for Pregnancy Outcomes (PDF) (9 pp, 43 K, About PDF)
- Session 1: Developing and Testing Study Evaluation Methods (PDF) (10 pp, 37 K, About PDF)
- Session 1: Lessons Learned for the Navigation Guide: Risk of Bias Tools for Exposure Metrics in Epidemiology Studies (PDF) (30 pp, 1 MB, About PDF)
- Session 1: The Navigation Guide Systematic Review Methodology: Evaluating Neurodevelopmental Outcomes (PDF) (32 pp, 2 MB, About PDF)
- Session 1: Lessons Learned in Protocol Development and Implementation of Multi-Endpoint Systematic Reviews (PDF) (37 pp, 2 MB, About PDF)
Session 2: Presentations
- Session 2: Advancing Systematic Review Relating to Mechanistic Data (PDF) (19 pp, 878 K, About PDF)
- Session 2: Approaches for Considering Mechanistic Information in Systematic Reviews (PDF) (12 pp, 256 K, About PDF)
- Session 2: Extending a Risk-of-Bias Approach to Address in Vitro Studies (PDF) (14 pp, 538 K, About PDF)
- Session 2: Improving Systematic Review and Usability of NexGen/High Throughput Data in Studies of Chemical Toxicity Using Aops (PDF) (35 pp, 2 MB, About PDF)
- Session 2: Key Characteristics of Carcinogens as a Basis for Organizing Data on Mechanisms of Carcinogenesis (PDF) (17 pp, 974 K, About PDF)
- Session 2: Systematic Identification of the Mechanistic Evidence for Cancer Hazard Assessment: Experience of the IARC Monographs Programme (PDF) (25 pp, 2 MB, About PDF)
- Session 2: Epigenetic Alterations Induced By Genotoxic Occupational and Environmental Human Chemical Carcinogens: A Systematic Literature Review (PDF) (20 pp, 5 MB, About PDF)
- Session 2: Adverse Outcome Pathway as An Integrating Framework for Systematic Review (PDF) (22 pp, 2 MB, About PDF)
- Session 2: Examination of Human Relevance of Anti-Androgenic Effects Observed Following Exposure to Dibutyl Phthalate (PDF) (23 pp, 1 MB, About PDF)
- Session 2: Noncancer MOA Decision Points: Examples from the Draft Formaldehyde Inhalation Assessment of Respiratory Tract Effects (PDF) (22 pp, 2 MB, About PDF)
- Session 2: Evidence-Based Methodologies in Toxicology: Application to Test Methods Comparison (PDF) (38 pp, 15 MB, About PDF)