Study Design for Evaluating of Impacts to Underground Sources of Drinking Water by Hydraulic Fracturing of Coalbed Methane Reservoirs
1.1 Study Background
Coalbed gas production through wells began in the 1970's as a safety measure in underground coal mines to reduce the explosion hazard posed by methane (Elder and Deul, 1974). In 1980, the U.S. Congress enacted a tax credit for non-conventional fuels production, including coalbed methane, as part of the Crude Oil Windfall Profit Act. In 1984, there were fewer than 100 coalbed wells in the U.S. By 1990, however, with the anticipated expiration of the tax credit almost 8,000 coalbed wells had been drilled nationwide (Pashin and Hinkle, 1997). In 1996, coalbed methane production in 12 states totaled about 1,252 billion cubic feet, accounting for approximately seven percent of U.S. gas production (U.S. Department of Energy, 1999). According to the U.S. Department of Energy, natural gas demand is expected to increase at least 45% in the next 20 years (U.S. Department of Energy, 1999a). The rate of coalbed methane production is also expected to increase in response to the growing demand.
Although the use of natural gas has many environmental benefits over traditional energy sources, concerns have been raised regarding the environmental impacts of coalbed methane production. Coalbed methane production in certain areas has led to alleged ground water depletion and produced water discharge issues. Citizens, state agencies, producers, and the regional EPA offices in those areas are working in concert to understand and mitigate potential problems. Separate from ground water depletion and water discharge issues are allegations that hydraulic fracturing of coalbed methane wells has affected the quality of ground water. State oil and gas agencies receiving citizen complaints have stated that, based on their investigations, hydraulic fracturing did not contribute to water quality degradation.
The State of Alabama recently adopted hydraulic fracturing regulations into its Underground Injection Control (UIC) program in response to an 11th Circuit Court of Appeals (hereafter, "the Court") decision (LEAF v. EPA, 118F.3d 1467). Prior to the Court's decision, EPA had not considered hydraulic fracturing as underground injection because it did not regard production well stimulation as an activity subject to regulation under SDWA's UIC program. However, the Court held that the injection of fluids for the purpose of hydraulic fracturing constitutes underground injection as defined in the Safe Drinking Water Act (SDWA); that all underground injection must be regulated; and that hydraulic fracturing of coalbed methane wells in Alabama was not regulated under Alabama's UIC program. Stakeholders are interested as to whether or not and to what extent hydraulic fracturing in coalbed methane could be a national issue.
Given that coalbed methane production will likely increase and given that EPA has received reports from citizens in different states alleging hydraulic fracturing resulted in ground water degradation, EPA believes further investigation is necessary to evaluate the potential risks.
The purpose of the fact-finding study described herein is to assist EPA in determining if hydraulic fracturing of coalbed methane wells poses a threat to underground sources of drinking water. EPA intends to complete this study before making regulatory or policy decisions regarding hydraulic fracturing.
In July, 2000 EPA published a Federal Register notice (Volume 65, Number 143, [Page 45774-45775]) requesting comment on a conceptual study design in order to receive stakeholder input on how an EPA study should be structured. EPA received more than 80 sets of comments from industry, state oil and gas agencies, environmental groups and individual citizens. A summary of comments can be viewed on EPA's Web site. Hard copies can be ordered by contacting EPA's Water Resource Center by phone at (202) 260-7786, or by e-mail to firstname.lastname@example.org or by conventional mail to EPA Water Resource Center, RC-4100, Ariel Rios Building, 1200 Pennsylvania Avenue, N.W., Washington, D.C. 20460.
1.2 What is Hydraulic Fracturing?
Hydraulic fracturing is the initiation and propagation of a fracture in a rock matrix by means of hydraulic pressure. The hydraulic fracturing process uses very high pressures to initiate a fracture at a specific depth in the coal formation. The length and height of this fracture is dependent upon a number of geological properties of the rock in which the fracture is initiated and the rock types that bound that interval. Due to the loading forces contributed by the rock above the injection zone, hydraulic fractures that are initiated at depths greater than 1,500 feet below the surface are typically vertical fractures. Hydraulic fracturing operations initiated at shallower depths may create horizontal fractures under ideal situations. Rather than a branched network of major and minor fractures, hydraulic fracturing creates fractures with vertical, two-dimensional fan-shaped, planar features, having two "lobes" centered on the wellbore perforations (Diamond, 1987; Morales et al, 1990).
One or more stimulation fluids is used in each hydraulic fracturing event. The composition of these fluids include water, hydrochloric acid, "slick" water (which includes a viscosity reducer), gels, and nitrogen foam (Palmer et al, 1993). Included with most stimulation fluids are "proppants" which are sand grains or glass, plastic or ceramic beads that serve to hold the hydraulic fracture open after the initiating pressure is released. The most common proppant used in coalbed methane treatments is sand. The amount of proppant injected for each fracture treatment ranges from 10,000 to 120,000 pounds (Holditch et al, 1989; Palmer et al, 1991a, 1993). Fracture widths in the formation vary from 0.5 inches to closed (i.e., no proppant used), depending on the distance from the wellbore and the efficiency of the proppant displacement into the fracture (Palmer and Sparks, 1990; Palmer et al, 1993; Steidl, 1993).
1.3 Summary of EPA's Study Approach
EPA will perform an investigation of the potential impacts on underground sources of drinking water (USDWs) from hydraulic fracturing of coalbed methane wells. Given the enormous variation in geology among and within coalbed basins in the U.S., any evaluation of potential impacts at a national level will necessarily be broad and general in scope. In order to best utilize resources in investigating this issue, EPA will conduct the study in three phases, narrowing its focus from general to more specific as needed. At the completion of each phase, EPA will decide if findings warrant continuation into the next phase
Phase I of the study will be a limited-scope assessment designed to enable the Agency to determine if hydraulic fracturing of coalbed methane wells clearly poses little or no threat to USDWs, or if the practice may pose a threat and require further investigation. In Phase I, EPA will
- Gather existing information to review hydraulic fracturing processes, practices, and settings;
- Request public comment to identify incidents that have not been reported to EPA; and,
- Review reported incidents of ground water contamination and any follow-up actions or investigations by other parties (State or local agencies, industry, academia, etc.). Phase I will include a review of the results of the GSA study described above. Once the information gathered through the Phase I effort is analyzed, EPA will make a determination regarding whether further investigation is needed.
The decision to continue with Phase II will be based on the results of the Phase I study and consideration of other priorities within the Office of Ground Water and Drinking Water. EPA will not continue into Phase II of the study if the investigation finds no hazardous constituents are used in fracturing fluids, hydraulic fracturing does not increase the hydraulic communication between previously isolated formations, and reported incidents of water quality degradation can be attributed to other, more plausible causes.
If OGWDW moves to Phase II, it would likely include site investigation(s) of proposed physical or chemical mechanisms that potentially lead to USDW endangerment and an estimate of national risk. If OGWDW believes Phase II findings indicate significant risk exists, EPA would move to Phase III of the study. Phase III would consist of an evaluation of existing regulations and a policy determination. EPA will not draft a detailed approach for conducting Phases II and III unless and until the Agency makes decisions to continue the study into each subsequent phase.
EPA will take several steps to involve the public during the course of the study. Those steps include publishing Federal Register Notices requesting comments on study plans and draft work products; providing periodic updates for stakeholders in the form of written communication; and maintaining a web site where stakeholders can view the project documents and provide information to EPA.
In addition to efforts to review hydraulic fracturing at a national level, EPA has jointly funded a site-specific study, which will be conducted by the Geological Survey of Alabama (GSA). This study will attempt to address a concern that is central to contamination and drawdown issues - the degree to which flow is confined within coal beds in coalbed methane fields (GSA, June, 2000). Additional information is available on the GSA's web site at http://www.gsa.state.al.us/gsa/3DFracpage/3Dfracstudy.htm.
2.0 PHASE I STUDY OBJECTIVES - MECHANISMS THAT MAY IMPACT USDWs
The first step in investigating the potential for hydraulic fracturing to endanger USDWs is to define mechanisms by which endangerment could occur. EPA defined two hypothetical mechanisms by which hydraulic fracturing of coalbed methane wells could potentially impact USDWs:
- The intentional direct injection of fracturing fluids into a USDW; and,
- Creation of a hydraulic communication between the target coalbed formation and adjacent USDWs.
The objective of the Phase I study is to consider these two mechanisms, based on existing literature and data, when evaluating whether hydraulic fracturing endangers USDWs.
Fluids can be injected into a USDW (mechanism #1 above) directly or indirectly depending on the location of the coalbed relative to a USDW. In many coalbed-methane producing regions, the target coalbeds occur within USDWs, and the fracturing process injects stimulation fluids directly into the USDWs (see Figure 1). In other production regions, target coalbeds are adjacent to the USDWs that exist either higher or lower in the geologic section.
Once the fluids are injected, local geologic conditions may interfere with their complete recovery. This may result in fracturing fluids being "stranded" in a USDW. If hazardous materials are included in the stimulation fluids, they could potentially endanger a USDW and any drinking water supplies relying on the USDW.
The second hypothetical mechanism is the creation of hydraulic communication, or a pathway, between the target coalbed and an adjacent USDW that may cause associated water quality and quantity impacts within the USDW, separate from the injection of fluids as described above. Hydraulic fracturing in coalbed methane formations typically creates fractures that are taller than they are wide (Morales et al, 1990; Zuber, 1990; Holditch et al, 1989; Palmer et al, 1991, 1991a, 1993), and the potential exists for fractures to extend through the stratigraphic layers that separate coalbeds and USDWs. Some stakeholders have theorized that either stimulation fluids containing hazardous materials or more saline ground water may enter a USDW through newly-formed pathways created from hydraulic fracturing (see Figure 2) and water quality may be affected from this hydraulic communication between coalbeds and adjacent USDWs.
It will be difficult to properly evaluate the potential for hydraulic
fracturing to create hydraulic communication between coalbeds and adjacent
stratigraphic layers as a Phase I study element. In order to make a defensible
determination that hydraulic fracturing has impacted an aquifer, it would
be necessary to have site-specific data concerning:
- Water quantity and quality conditions in a USDW (or a well) both before and after a fracturing event;
- Location, dimensions, and conductivity of fractures created during the coalbed stimulation event;
- Measured changes in ground water flow between the USDW and coalbeds or other aquifers; and
- Impacts of other, unrelated, hydrologic and water quality processes that could also be affecting the USDW.
The extent to which these data are available to properly support an analysis of national scope is unknown, but the effort necessary to collect and evaluate them fully is beyond the scope of Phase I. EPA believes the first mechanism described in this section, fluid injection directly into a USDW, presents the more plausible scenario of the two endangerment scenarios presented. However, the data that are collected as part of Phase I will be analyzed to determine if information exists to draw conclusions regarding the impacts of creating a hydraulic connection between a coalbed and a USDW, and if so, this information will be included in the Phase I report.
3.0 STUDY COMPONENTS
EPA will research the topic areas described below to evaluate the impacts of hydraulic fracturing on USDWs. Information will be collected regarding the geology and hydrogeology of the coalbed methane production regions (Section 3.1), the processes used to hydraulically fracture coalbed methane production wells (Section 3.2), the fluids used in the fracturing process (Section 3.3) and the techniques that may be used to monitor the effects of fracturing (Section 3.4). EPA will also evaluate water supply incidents possibly related to hydraulic fracturing of coalbed methane production wells (Section 3.5). EPA will rely on currently available literature and data as the primary source of information for Phase I efforts.
3.1 Hydrogeology of Coalbed Methane Production Regions
The study will evaluate, on a regional basis, the hydrogeologic setting of each coalbed methane production region. Local hydrogeologic conditions vary in each coalbed methane production region in the country, and, as a result, the potential impacts of the hydraulic fracturing process may also vary.
The basic geology of each region will be evaluated, including the location and extent of the coalbed formations. The study will also identify the depth and hydrologic relationships between USDWs and coalbed methane wells by identifying where coalbed formations are located within or directly adjacent to currently mapped USDWs. This information will be used to evaluate if hydraulic fracturing of methane production wells could result in the injection of fluids into USDWs as described in Section 2.0 above.
3.2 Processes Used to Hydraulically Fracture Coalbed Methane Formations
Information will be obtained to describe the basic processes used in coalbed methane production, including well construction information, water and methane withdrawal data. The ways in which methane production practices vary by region will be evaluated and described.
If coalbed formations are identified in proximity to mapped USDWs as
determined in the hydrogeologic analysis (Section 3.1), the potential
to create a hydraulic communication between the target coalbeds into the
USDW will be evaluated considering:
- The depth of separation between coalbed formations and USDWs;
- The petro-physical characteristics and susceptibility (e.g., the in-situ vertical stress distribution) within intervening rock layers to fracturing; and
- The general characteristics and dimensions of fracturing techniques utilized in the particular region or field from readily available data.
3.3 Hydraulic Fracturing Fluids and Proppants
EPA will collect the types of information listed below to address the
potential effects of injecting fracturing fluids directly into a USDW:
- The composition of hydraulic fracturing fluids, including potentially hazardous materials;
- The types of proppant materials injected with the hydraulic fracturing fluids;
- Quantities of materials injected;
- Data on the extent of fractures, specifically in coalbed formations; and
- Recovery of fluids during methane production following a fracturing event, and the corresponding loss of stranded fluids in the rock matrix.
Because practices may vary from one region to another based on the nature of the coal seams being produced, the study will describe the variation in fluids used for hydraulic fracturing throughout the country.
3.4 Techniques Used to Monitor and Model the Hydraulic Fracturing Process
The project team will document how the hydraulic fracturing process and its effects are modeled and monitored. The study will evaluate what types of instruments are used to document a fracturing event, and what types of models are used to predict the extent of fracturing. A review of modeling and monitoring information will help document how accurately the extent of fracturing in coalbed formations can be predicted, and how useful these data may be in evaluating the issues associated with this study.
3.5 Reports of Alleged Contamination Cases
In 1998, the Ground Water Protection Council conducted a survey of state oil and gas agencies to determine if those agencies were aware of any cases in which hydraulic fracturing (CBM) adversely impacted water wells. Of those 25 states that responded to the survey, thirteen state oil and gas agencies reported coalbed methane wells. Each of those thirteen State agencies reported they were not aware of any case in which hydraulic fracturing impacted water wells (Ground Water Protection Council, 1998). In an effort to be thorough, EPA believes it should offer other drinking water agencies and the public at large an opportunity to provide information to EPA on any impacts to ground water believed to be associated with hydraulic fracturing. EPA intends to offer this opportunity through a request for public comment.
Reports of alleged ground water impacts due to hydraulic fracturing
of coalbed methane wells that are received through the request for public
comment will be reviewed in conjunction with the appropriate State agency.
EPA will evaluate reports based on State agency responses and available
data to determine if sufficient information exists to understand the source
of water quality issues, or if additional data should be collected. The
most pertinent information to evaluate in these reviews will be:
- basic construction details of allegedly impacted wells,
- the timing of the alleged impact relative to fracturing events at nearby methane production wells;
- nature of the alleged water quality problems;
- baseline and subsequent water quality data; previous industrial activity in the area of concern; and
- climate data.
To the extent possible, based on the available incident data, EPA will evaluate where or when USDW degradation has occurred specifically from hydraulic fracturing of coalbed methane production wells.
4.0 DATA COLLECTION
The study of potential impacts associated with hydraulic fracturing will be based on existing information on hydraulic fracturing as utilized in coalbed methane production. The ability to evaluate the study objectives is based on the amount of information available from existing literature sources, academic institutions, industry organizations, federal, state and local agencies, and individuals' reports.
Data collection for the study will follow a four-tiered approach. First, EPA will review existing industry, academic, and government agency resources and collect pertinent information. Second, contacts will be made with State and federal agencies involved in coalbed methane production. These will include the U.S. Geological Survey, the U.S. Department of Energy, State water supply agencies, State oil and gas boards, and the Groundwater Protection Council. Third, EPA will contact State agencies to discuss the history and nature of complaints received from citizens and any follow-up conducted in relation to those complaints. Fourth, to the extent possible within this project, visits will be made to targeted coalbed methane production areas to collect site-specific data on methane production and hydraulic fracturing practices. Depending on the information available from State agencies, EPA may review locations and construction details of subject water wells, timing and locations of fracturing events, and other data detailing conditions that may have contributed to problems in water wells.
Quality Assurance (QA) procedures for data collection are included in Appendix A, Section 3.1. Section 3.2 of the QA protocol addresses procedures for data logging.
5.0 DATA ANALYSIS
Once data have been collected, EPA will evaluate the information to determine if hydraulic fracturing of coalbed methane wells clearly poses little or no threat to USDWs, or if the practice may pose a threat and require further investigation. EPA will decide if Phase II is warranted based on Phase I results, resources available to EPA to conduct further investigation, priorities of the Agency at that time, and whether the scope of the issue is national or localized.
6.0 STAKEHOLDER INVOLVEMENT
Maximizing stakeholder input to the study process is a priority of EPA.
To that end, efforts will be made to identify all stakeholders to the
greatest extent possible. Stakeholder input has been solicited on this
study methodology and will be solicited on the draft summary report. Postings
on relevant web sites, mailings, public meetings, public notices, and
other means, will be used to involve industry, environmental groups, citizen
groups, academia, drinking water agencies, and other governmental and
non-governmental agencies. EPA will obtain input from experts (industry,
academia, and government agencies) on the issues of coalbed methane production
and hydraulic fracturing, in order to peer-review data sources, analytical
approaches, and conclusions. The review process will include:
- EPA internal review conducted by Offices outside the Office of Ground Water and Drinking Water;
- Inter-agency review requested from Department of Energy, U. S. Geologic Survey, Bureau of Land Management, and appropriate state agencies;
- Independent scientific peer review conducted by an EPA-convened panel; and
- Stakeholder review.
Further discussion of the quality assurance aspects of the review process is included in Section 4.0 of the QA protocol. This review process will result in an official record of comments on study methods and results, and a determination of whether the study conclusions can be supported by the data.
Reporting of study methods, data sources, data analysis, and conclusions will be made in a clear, accurate, and complete manner. The body of technical information considered during the study will be recorded and available for review by stakeholders. In addition, the manner for evaluating, analyzing, and drawing conclusions from the collected data will be communicated in order that reviewers may observe and validate methods. Limitations on where a conclusion may apply, and alternative interpretations of data, will be documented. Overall, the conclusions that address the objectives of the study will be supported by the results of peer-reviewed data evaluation and analysis. EPA will post materials to the internet periodically to inform stakeholders of progress on the study. The stakeholders will have an additional opportunity for input when the draft study is released. Notice of availability will be made through mailings to individual stakeholders.
8.0 SCHEDULE AND DELIVERABLES
EPA estimates the study as described herein will be completed by February 2002. EPA expects to complete the literature search by April 2001; a request for public comment in May, 2001; and follow-up of reported incidents in August, 2001. A draft report is expected to be completed and available for comment three months after data collection.
Deliverables for Phase I of the study will include a study methodology and draft and final reports.
Diamond, W.P. 1987. Characterization of fracture geometry and roof penetrations associated with stimulation treatments in coalbeds, Proceedings of the 1987 Coalbed Methane Symposium, U. of Alabama, p. 243.
Elder, C. and M. Deul. 1974. Degasification of the Mary Lee coalbed near Oak Grove, Jefferson County, Alabama, by vertical borehole in advance of mining, U.S. Bur. Of Mines Report 7968.
Holditch, S., J. Ely and R. Carter. 1989. Development of a coal seam fracture design manual, Proceedings 1989 Coalbed Methane Symposium, Tuscaloosa, Alabama, pp. 299-320.
Hansen, C. 2000. Interstate Oil and Gas Conservation Commission August 15, 2000 Comments in Response to FRL6839-8, Oklahoma City, OK, pp. 1-3.
Geological Survey of Alabama. 2000. Grant Application for 3-D Characterization of Natural and Induced Fractures in Coalbed Methane Reservoirs of the Black Warrior Basin in Alabama, Tuscaloosa, AL.
Ground Water Protection Council. 1998. Survey Results on Inventory and Extent of Hydraulic Fracturing in Coalbed Methane Wells in the Producing States, Oklahoma, City, OK.
Morales, R., J. McLennan, A. Jones and R. Schraufnagel. 1990. Classification of treating pressures in coal fracturing, Proceedings of the 31st U.S. Symposium on Rock Mechanics, 31, pp. 687-694.
Palmer, I., N. King and D. Sparks. 1993. The character of coal fracture treatments in the Oak Grove Field, Black Warrior Basin, In Situ, J. of Coal Research, v. 17(3), pp. 273-309.
Palmer, I., N. King and D. Sparks. 1991. The character of coal fracture treatments in Oak Grove Field, Black Warrior Basin, SPE Paper 22914, Proceedings, 1991 Society of Petroleum Engineering Annual Technical Conference and Exhibition, pp. 277-286.
Palmer, I., R. Fryar, K. Tumino and R. Puri. 1991a. Comparison between gel-fracture and water-fracture stimulations in the Black Warrior Basin, Proceedings 1991 Coalbed Methane Symposium, University of Alabama, pp. 233-242.
Palmer, I. and D. Sparks. 1990. Measurement of induced fractures by downhole TV camera in coalbeds of the Black Warrior Basin, Soc. Of Petroleum Engineers Paper 20660, Proceedings, 1990 Soc. Of Petroleum Engineers Annual Technical Conference and Exhibition, pp. 445-458.
Pashin, J. and F. Hinkle. 1997. Coalbed methane in Alabama, Geol. Surv. Of Alabama, Circular 192, 71pp.
Steidl, P. 1993. Evaluation of induced fractures intercepted by mining, Proceedings 1993 Coalbed Methane Symposium, University of Alabama, pp.675-686.
U.S. Department of Energy. 1999. U.S. Crude Oil, Natural Gas, Natural Gas Liquids Reserves 1999 Annual Report, p. 34.
U.S. Department of Energy. 1999a. Environmental Benefits of Advanced Oil and Gas Exploration and Production Technology, Office of Fossil Energy, p 8.
Zuber, M., V. Kuuskraa and W. Sawyer. 1990. Optimizing well spacing and hydraulic fracture design for economic recovery of coalbed methane, SPE Formation Evaluation, v.5(1):98-102.
Zuber, M. 1997. Production characteristics and reservoir analysis of coalbed methane reservoirs, International Journal of Coal Geology (Appalachian Coalbed Methane), v.38(1-2).
QUALITY ASSURANCE PROTOCOL
The current strategic plan of the U.S. Environmental Protection Agency (EPA), published in 1997 in response to the Government Performance and Results Act, acknowledges that environmental protection efforts need to be "based on the best available scientific information," and "sound science." Although this goal sounds straightforward, achieving it in practice is complicated and challenging. However, the credibility of the resulting policy decision depends, to a large extent, on the strength of the scientific evidence on which it is based. This Quality Assurance Process for data collection describes the procedures to be used for a systematic and well-documented approach to realizing this goal for the Hydraulic Fracturing Study Design and Methodology.
It is imperative that this quality assurance process produces a set
of data and scientific findings that are sound and conclusions that are
supported by the data. "Sound science can be described as organized investigations
and observations conducted by qualified personnel using documented methods
and leading to verifiable results and conclusions (SETAC, 1999)." In order
to ensure that findings are sound, the following quality assurance questions
will be addressed for all sources of data:
- What was the purpose of the study?
- Whose data is it?
- What is their source?
- Are the data reliable?
- Is the interpretation biased?
This quality assurance process establishes a set of guidelines and general approaches to assess available data and information in a clear, consistent, and explicit manner. Data collection and review according to this process will make conclusions more transparent and thus more readily understood and communicable to stakeholders. In this study, the quality assurance process will proceed even when the data and scientific information is insufficient. Areas lacking data will be recorded and discussion will ensue on the role of scientific uncertainty in the quality assurance process.
The objectives of the systematic expert review of data and information
are transparency, avoidance of bias, validity, replicability, and comprehensiveness.
Following a protocol for review of data and information, by everyone involved
in data collection and evaluation, can ensure a common understanding of
the task and adherence to a systematic approach. The components of this
quality assurance protocol are as follows:
- Specification of the hypotheses to be addressed;
- Justification of the expertise represented in the expert investigators team;
- Specification of the methods to be used for identification of relevant studies, assessment of evidence of the individual studies, and interpretation of the entire body of available evidence (WHO, 2000);
- Review process; and
- Communication of findings.
Revisions of the protocol may be necessary as new aspects of the task emerge during the study development process. The components of the protocol are addressed in further detail below.
1.0 SPECIFICATION OF THE HYPOTHESES/OBJECTIVES
Specification of the hypotheses to be addressed is included in Section 2.0 of the Study Methodology. This quality assurance process will ensure that all participants in the study understand the scope and purpose of the study. Definition of the scope and purpose of the study in Section 2.0 of the Study Methodology is the basis for the choices made concerning data collection and methods for evaluation.
2.0 QUALIFIED INVESTIGATORS
To provide authoritative assessments of data and information, it is
important to rely on expert investigators to evaluate the evidence, draw
conclusions on the existence of actual and/or potential hazard, and estimate
the magnitude of the associated risk. The team of expert investigators,
which will evaluate the evidence associated with this study, possesses
the following qualifications:
- Formal training and/or on-the-job experience;
- Knowledge of the subject and the body of technical information pertaining to it;
- Experience in scientific review of technical data and information;
- Ability to use descriptive and analytical tools appropriately;
- Ability to design studies to test hypotheses;
- Experience in risk assessment of environmental factors;
- Ability to communicate results accurately to decision-makers and stakeholders;
- Experience coordinating multiple tasks and disciplines to ensure timely and accurate delivery of study components; and
- Representation of a broad range of abilities.
3.0 DOCUMENTED METHODS
Processes and methods used to collect and evaluate the data and information must be clear, explicit, and based on valid practice. It is important to adhere to a rigorous and thorough approach to the processes of data collection (Section 3.1), data logging (Section 3.2), evaluation of data (Section 3.3), data analysis (Section 3.4), and use of data in drawing conclusions (Section 3.5).
3.1 Data Collection
Comprehensive identification of relevant studies will be achieved by
use of the following strategy:
- Involvement of qualified researchers and trained investigators;
- Adherence to an explicit search plan including identification of key words;
- Considered effort to include all available studies;
- Search of bibliographic databases and journals;
- Inclusion of abstracts and unpublished data;
- Contact with authors of published data for further information;
- Contact with applicable agencies and associations;
- Contact with industry representatives;
- Contact with environmental group representatives;
- Other stakeholder contacts; and
3.2 Data Logging
Verification of data and information will be achieved by adherence to a rigid data collection protocol involving explicit written documentation of where, when, and from whom particular data and information is obtained, including information obtained from personal or telephone contacts. All sources of data and information, both verbal and written will be recorded in ink in bound notebooks.
All reports, records, etc., potentially providing data for the study, will initially be included in the inventory (data log) of relevant information. Strict maintenance of the data log will address the necessity for producing a record of the comprehensiveness of the study base of information.
3.3 Data Evaluation
Subsequent to the data logging process, those reports potentially providing
useful information will undergo a selection process to evaluate quality
of the information and usefulness to the study. Systematic evaluation
of the validity of individual studies, data, and information will therefore
include assessment of the following:
- Source of the data and information;
- Qualitative review of the literature;
- Qualitative review of data and information collected;
- Scientific strength of the data and information;
- Geographical, geological, geochemical, spatial, and temporal relevance;
- Relevance to determining baseline conditions;
- Validity of extrapolation to the scope of the Study;
- Characteristics of associations, plausibility, alternative explanations;
- Consistency and specificity of the results;
- Scientific uncertainties, limitations, and confounding variables; and
A scale or rating of the data and information with respect to a level of proof required to support conclusions is specifically not proposed as part of this quality assurance process. Establishing a specific level of scientific evidence required to justify a subsequent conclusion would generate significant controversy. Instead, expert judgment will be used to evaluate and weigh available data and information.
3.4 Data Analysis
A variety of technical methods and tools will be utilized to sort through
the pertinent information and decipher the meaning of the data. These
data analysis methods may include:
- Quantitative review of selected data and information collected;
- Tabulating valid data and information;
- Constructing geologic cross sections;
- Evaluating current and historical site operations;
- Examining adjacent land uses;
- Review of consistencies between studies and information;
- Review of sources of discrepancies between studies and information;
- Evaluation of heterogeneity among studies;
- Development of inclusion and exclusion criteria for data and information used for hypothesis testing; and,
All assumptions will be explicitly documented, the basis for use of any models explained, lack of evidence noted, and scientific uncertainties described as precisely as possible.
3.5 Drawing Conclusions
Drawing conclusions from evaluated and appropriately analyzed and summarized
data and information will involve expert judgment as to whether observations
are consistent with the study hypotheses/objectives (presented in Section
2.0 of the Study Methodology) or whether some alternative is suggested.
The expert investigators will draw upon all evaluated and appropriately
summarized data and information, however, no checklist or formula will
be applied to arrive at conclusions. Instead, critical scientific reasoning
and judgment will be used to draw conclusions. The process of scientific
reasoning and judgment will be made explicit by describing and documenting
how expert investigators:
- Assessed completeness of data and information;
- Accounted for lack of evidence and limitations, and impacts on the conclusions;
- Qualified and quantified the degree of scientific uncertainty;
- Assessed and accounted for bias in original data and/or information;
- Weighed particular information;
- Used applicable guidelines and rationales;
- Used any ranges of estimates to arrive at conclusions, where appropriate;
- Incorporated assumptions into assessments and accounted for the implications of those assumptions in their conclusions; and
Conclusions will be drawn within the boundaries of the data and the scope of the study. Lack or absence of evidence will be addressed. The relative strength or weakness of available information to support conclusions, limitations on where a conclusion may apply, and alternative interpretations of data, will be recognized and discussed. Any qualification on the use of the data and factors that contribute to uncertainty will be conveyed.
4.0 REVIEW PROCESS
The quality assurance review process will provide a means to examine
if the results and conclusions are verifiable. The review process will
result in a determination of whether the conclusions are directly supported
by the data or evidence and can be independently validated by others.
This quality assurance review process will be hierarchical and will include
four review levels:
- EPA internal review,
- Inter-agency review,
- Independent scientific peer review, and
- Stakeholder review.
Internal review will be accomplished by using an in-house staff member, skilled in quality assurance and other EPA offices involved with coalbed methane or hydraulic fracturing. Other federal agencies will be asked to review products including the USGS, the BLM and the DOE. Those agencies may also participate as partners with EPA in conducting the study. Thus, this level of peer-review will not constitute an independent peer review. EPA will assemble a peer review panel consisting of experts in hydraulic fracturing or associated subjects. Finally, EPA will accept comments from stakeholders. Work products will be available on the internet, or stakeholder may request hard copies by contacting EPA via electronic mail, telephone, or in writing.
Supporting documentation and data will be easily available for in-depth consideration by other reviewers. Candid and critical comments will be sought to assist the investigators in making the study as sound as possible and to ensure that the study meets Agency standards for objectivity, evidence, and responsiveness to the study charge. Reviewer comments and objections will be preserved and made a part of the record for the study. Reasons for proceeding or not proceeding with the study will be clearly explained.
5.0 COMMUNICATION OF FINDINGS
This quality assurance process will be reflected in the communication
of scientific findings in a clear, accurate, and complete manner to interested
parties. Investigators will communicate to Agency and stakeholders:
- The body of technical information that was considered;
- The manner for evaluating, and drawing conclusions from, collected data and information; and
- Conclusions that address the hypotheses/objectives and are supported by the results of data evaluation and analysis.
The use of presentation tools such as charts, diagrams, and computer-generated displays will be based on sufficient, valid, and defensible data.
Breslin, K., Airing on the Side of Caution or Pulling Standards Out of Thin Air? Environmental Health Perspectives, 108(4), April 2000. http://ehpnet1.niehs.nih.gov/members/2000/108-4/spheres.html
Clay, R., Still Moving Toward Environmental Justice, Environmental Health Perspectives, 107(6), June 1999. http://ehpnet1.niehs.nih.gov/members/1999/107-6/spheres.html
Evaluation and Use of Epidemiological Evidence for Environmental Health Risk Assessment: WHO Guideline Document. Environmental Health Perspectives, 108(10), October 2000. World Health Organization European Centre for Environment and Health, Bilthoven Division, A. van Leeuwenhoeklaan 9, Bilthoven, The Netherlands. http://ehpnet1.niehs.nih.gov/members/2000/108p997-1002kryzanowski/abstracts.html.
Manuel, J., Truth in Numbers, Environmental Health Perspectives, 108(8), August 2000. http://ehpnet1.niehs.nih.gov/docs/2000/108-8/niehsnews.html
Sound Science Technical Issue Paper. Society of Environmental Toxicology and Chemistry (SETAC), SETAC Press, Pensacola, FL, 1999. http://www.setac.org/sstip.html
Strengthening Science at the U.S. Environmental Protection Agency: Research-Management and Peer-Review Practices. National Academy Press, Washington, D.C. (2000). http://www.nap.edu/catalog/9882.html