Department of Defense BEST PRACTICES FOR DATA QUALITY OVERSIGHT OF ENVIRONMENTAL SAMPLING AND TESTING ACTIVITIES - INTRODUCTION

INTRODUCTION

This report documents several DoD best practices for assuring that data of known and documented quality are obtained during environmental investigations and that logical decisions based on quality data drive remedy selections. This report was developed by the DoD Environmental Data Quality Workgroup (EDQW) which is tasked to develop and coordinate environmental sampling and testing policy. The report was prepared in partial response to a request by the U. S. Environmental Protection Agency, Office of Solid Waste and Emergency Response (OSWER), Federal Facilities Reuse Office (FFRRO), dated July 2, 1997, "to define those processes that contribute to uniform data collection and analysis, reporting, and interpretation thus improving the quality of the data, saving time, or reducing program costs." Additionally, this report addresses issues raised in the February 21, 1997 DoD Inspector General Report No. 97-098 and provides a framework for finalizing the EDQW Strategy for improving DoD environmental sampling and testing activities.

BACKGROUND

Prompted by a multi-million dollar laboratory fraud issue, EPA Region 9's laboratory program was audited by the EPA Office of the Inspector General (OIG) in 1995 (Laboratory Data Quality at Federal Facility Superfund Sites, ElSKB6-09-0041-7100132, 20 March 1997). This audit led to 1997 audits of all EPA regions. In 1997, the DoD IG also performed an audit of environmental laboratory services, focusing primarily on contracted services (DoD IG Audit Report on Laboratory Support Services for Environmental Testing, Report No. 97-098, 21 February 97). The DoD audit looked at both compliance and cleanup programs. Also in response to laboratory fraud issues, the California Military Environmental Coordinating Committee (CMECC) issued a report in March 1997: Best Practices for the Detection and Deterrence of Laboratory Fraud. These reports were used as resources by the EDQW to identify and prioritize this compilation of best practices.

OBJECTIVE

Best Practices identified by the DoD fall into several broad categories and cover a range of activities. Some are current practice among the components, while others can be easily implemented.Some will require additional work to implement DoD-wide. The categories and Best Practices discussed in the report include:

• Using Data Quality Objectives (DQOs)

  • Use a Systematic Planning Process for Data Collection Activities

  • Involve Regulators

• Improving Policy, Guidance, and Documentation

  • Develop DoD Policy and Guidance Documents

  • Implement IS0 Guide 25

  • Implement IS0 Guide 58

  • Implement ANSVASQC E4

• Improving Laboratory Oversight Practices

  • Perform Laboratory Audits

  • Include Proficiency Testing Samples

  • Require Standard Electronic Data Deliverables

  • Validate Data

  • Institute the National Environmental Laboratory Accreditation Program

• Improving Management and Contracting Processes

  • Share Laboratory Performance Data

  • Use Standard Performance Based Laboratory QA/QC Contracts

  • Maintain DoD Core Capability In Environmental Data Analysis

  • Use a Quality Assurance Officer

For each best practice, brief discussions are provided about the implementation status, the objective, and recommendations to further improve the practice. Each best practice is assessed for its effect on quality, schedule, and cost. The rating system used is:

1. Definite demonstrated improvement. Improvement is quantified or quantifiable.

2. Probable improvement. May not be immediately quantifiable.

3. Neutral.

4. Definitely will not improve.

The improvement in data quality that would result from implementation of each best practice is measured relative to the quality of data obtained using existing DoD procedures. The general status quo used as a basis for comparison may not be reflective of the standard operating procedure of a particular component or branch of service within a component relating to a specific suggested best practice.

The Best Practices described herein were selected from a comprehensive list of recommendations and best practices suggested by Components, CMEEC, EPA guidance documents, and EPA and DoD IG Reports. Practices were then rated and prioritized. These ratings are assigned by the EDQW based on an evaluation relative to whether the practice adds to quality, saves time, and reduces costs. Ratings of Best Practices are compiled in Table 1.

RATINGS:

1. Definite demonstrated improvement. Improvement is quantified or quantifiable.

2. Probable improvement. May not be immediately quantifiable.

3. Neutral.

4. Definitely will not improve.

DoD BEST PRACTICES FOR DATA QUALITY OVERSIGHT OF ENVIRONMENTAL SAMPLING AND TESTING ACTIVITIES

USING DATA QUALITY OBJECTIVES

The Data Quality Objective (DQO) Process is a strategic planning approach that is used to prepare for data collection activities. The DQO Process establishes specific objectives for an environmental study or sampling program and focuses data collection and analysis to meet those objectives. Appropriate use of the DQO process achieves two major objectives: (1) it assures that the type, quantity and quality of data collected are appropriate for the decision at hand and (2) it eliminates the collection of unnecessary, redundant and overly precise data.

Involvement of regulatory technical staff is needed throughout the DQO process. In particular, Federal, State, and regional regulatory agency technical staffs need to be involved up front in site investigation and remediation projects. Working with regulators throughout project planning and execution helps to assure that data quality objectives are appropriate for their intended use, information is shared by all parties, and they reach agreed upon goals.

USE A SYSTEMATIC PLANNING PROCESS FOR DATA COLLECTION ACTIVITIES

Best Practice: Use a systematic planning process for designing data collection activities to ensure that the requisite type, quality and quantity of data are obtained to meet project objectives. DQOs are established for each project by technical staff in consultation with stakeholders, such as regulators, at the beginning of an investigation and in the design and execution of data collection and remedial action activities. The DQO process is typically documented in the Quality Assurance Project Plan (QAPP) and may be further defined in site-specific Field Sampling Plans (FSPs).

Implementation Status: DoD uses DQOs extensively for the cleanup program and to a lesser extent in the compliance program. DQO guidance is provided in US EPA Guidance for the Data Quality Objectives Process, EPA QA/G-4, September 1994. DoD incorporates this document by reference in many service-specific documents. Other guidance is provided by the USACE in Engineering Manual 200-l-2, Technical Project Planning Process, Guidance for HTRW Data Quality Design. Recently, the USACE has updated EM-200-l-2, which outlines a four-phase Technical Planning Process (TPP). The TPP can be used at small, simple sites as well as large, complex sites.

Discussion: In the DQO process, decision-makers define data requirements and acceptable levels of data error based on data uses during planning, site investigation, engineering design, and remediation. The goal of the DQO process is to minimize expenditures while producing data of sufficient quality and quantity needed to make decisions. Data requirements are determined by site and project strategies as well as the effects of cost, schedules, and other constraints. The advantages of this approach to project planning are that the right data are gathered within the constraints of the project so that data quality and quantity are based on intended use at various stages of the process. The short-term disadvantage is the up-front planning time required by technical personnel and stakeholders to properly establish definitive DQOs. The DQO Process, as defined by EPA in QA/G-4, is a seven-step process for "data collection efforts that will require or result in a substantial commitment of resources."

In the Technical Planning Process, the USACE has defined a graded approach for planning data collection activities, which is designed to provide a sound basis for site decisions and accelerates progress to site closeout. The process includes four phases, including the establishment of DQOs, and it implements an overarching quality management system based on ANSVASQC E-4. (See DoD Best Practice "Implement ANSVASQC E-4.")

Because DQOs are performance based, the process promotes the use of expedited site characterization and innovative monitoring technologies that may prove to be more cost effective or technically superior. DQOs provide an operational tool for facilitating the use of Performance Based Measurement Systems (PBMS), thereby replacing traditional reference methods with improved technology, where appropriate.

Recommendations: The EDQW should continue to emphasize DQOs and incorporate a systematic planning process for data collection activities into policy documents for both the cleanup and compliance programs. The EPA QA/G-4 document and the USACE TPP provide models to accommodate both small and large projects and include the use of definitive DQOs for sound decision making within project restraints. Appropriate technical staff (chemists, geologists, engineers, etc.) must be involved in setting and assessing DQOs to ensure proper use of the process. In addition, laboratories should be involved up front in the DQO planning process. Finally, appropriate personnel, such as remedial project managers and sampling personnel, should receive DQO training as part of their initial training process, and refresher training at specified intervals, to ensure an operable understanding of DQO application.

INVOLVE REGULATORS

Best Practice: Involve EPA, and other cognizant regulatory agency technical staff, throughout the project. This is especially critical at junctures such as developing Data Quality Objectives (DQOs) and incorporating the use of innovative monitoring and analytical technologies, EPA and DoD should also share information on laboratory capabilities.

Implementation Status: DoD policy promotes timely acceptance of EPA and other regulatory agency approved performance based improvements in sample collection, preparation and analytical techniques. DoD encourages up-front planning which involves the regulators so that cost effective data are gathered to meet project needs.

Discussion: Involvement by Federal, State, and regional regulatory agency technical staffs working in partnership throughout the life cycle of DoD restoration projects will ensure that appropriate DQOs and Quality Assurance Project Plans (QAPPs) are established and implemented. Joint participation will enable all parties to focus on crucial issues and identify prompt and appropriate resolutions. Involvement of technical staff will also facilitate using Performance Based Measurement Systems (PBMS), which promote the use of new monitoring technologies, field analytical techniques and laboratory testing methods to take advantage of cost efficiencies which can be realized from state of the art innovations.

Recommendations: The EDQW should continue to engage cognizant regulators regarding proactive involvement in environmental programs, and in particular seek involvement of regulatory technical staffs for setting and assessing data quality objectives. In addition, the EDQW and EPA headquarters should work together to promote appropriate use of PBMS and provide consistent guidance to the field, both on a program-wide and project specific basis.

Previous Section: Acknowledgements and Executive Summary
Next Section: Section 2: Improving Policy, Guidance, and Documentation