Department of Defense BEST PRACTICES FOR DATA QUALITY OVERSIGHT OF ENVIRONMENTAL SAMPLING AND TESTING ACTIVITIES - IMPROVING LABORATORY OVERSIGHT PRACTICES

The EDQW is responsible for the quality of the data used to make environmental decisions. Using a quality systems approach such as IS0 Guide 25 to set standards, DoD can improve laboratory -. oversight while reducing costs.

PERFORM LABORATORY AUDITS

Best practice: Laboratory assessments consist of on-site audits to review and verify compliance with general quality systems, methods and project specific criteria. An initial audit is performed prior to sample submission. In addition, periodic audits are performed during the life of the contract to assess maintenance of proficiency.

Implementation Status: DoD components have QA/QC programs in place which typically require on-site assessments of contract testing laboratories. Some components have accreditation requirements that include on-site assessments. DoD is working to develop a uniform quality system, standard audit criteria and a program of reciprocal recognition of each component's audit systems.

Discussion: Laboratory audits evaluate numerous items which impact the quality of data. Audits include the evaluation of management, technical expertise, facilities, equipment, reference materials, methods, calibration, training, documentation and reporting. A pre-performance audit can identify the capabilities of a laboratory before any samples are submitted. Annual follow-on audits can be used to identify problems and deficiencies so they can be corrected early in the project saving both time and money. Audits also send the message that the government will closely monitor contract laboratory performance which may be a deterrent to fraud.

Recommendations: Audits should be performed to evaluate a laboratory's conformance with IS0 Guide 25 quality systems criteria, specific testing procedures, and, where applicable, the EPA's Good Automated Laboratory Practices (GALP). Audits should be performed initially and periodically throughout the life of the project or contract. The EDQW should promote the exchange of audit information between the components. Copies of the audit report should be provided to the appropriate DoD Quality Assurance Officer for dissemination. Historical audit reports should be used as a reference for follow-on audits.

INCLUDE PROFICIENCY TESTING SAMPLES

Best Practice: Proficiency testing (PT) samples can demonstrate a laboratory's proficiency to analyze selected analytes. Periodic analysis of PT samples can provide an on-going check to determine if proficiency is maintained. Single blind and double blind samples are used as an effective QA/QC tool for detection and deterrence of environmental laboratory performance problems, including fraud. DoD components should share the results of PT sample testing.

Implementation Status: DoD reviews the EPA Water Pollution/Water Supply PT sample results and each component uses PT samples to evaluate laboratory performance. The Army has used PT samples developed in-house, the Air Force uses double-blind PT samples, and the Navy uses commercially available PT samples.

Discussion: PT samples are not only useful for assessing proficiency and identifying laboratory problems, but they also send a message to the laboratory community that DoD intends to actively assess lab performance. These PT tools can be used in a variety of combinations and at variable frequency depending on the size, duration, and complexity of a project or contract.

Recommendations: The EDQW should develop a program by which components can share individual laboratory PT sample results. The EDQW should work with EPA as they transition to using commercial PT sample providers and consider using these sources for qualification of laboratories to perform DoD work. The EDQW should monitor the AFCEE double-blind PT sample program for cost and effectiveness and consider using it DoD-wide as a method for monitoring lab data quality. The EDQW should also consider using the USACE single-blind Program as an additional DoD-wide QA resource. The EDQW should review available PT sample services and make recommendations on how to incorporate the EPA PT sample program and existing DoD PT sample programs to support an overall QA oversight strategy for DoD environmental testing.

USE STANDARD ELECTRONIC DATA DELIVERABLES

Best Practice: A standard electronic data format should be selected for use by all components. All chemical data should be provided in this format, which should be compatible with global information system (GIS) database requirements. Basic data validation should be performed electronically, using a program based on the standard electronic data format. Laboratories should comply with the EPA's Good Automated Laboratory Practices (GALP).

Implementation Status: The Department of Energy (DOE) has developed an electronic data format titled "Department of Energy Environmental Management Electronic Data Deliverable Master Specification," commonly known as DEEMS. DoD is evaluating the use of DEEMS as a standard electronic data deliverable (EDD) and as a tool for electronic data validation. Currently the Defense Environmental Security Corporate Information Management (DESCIM) Program Office is developing a standard EDD and data base structure. Components are supporting this effort and participate in a DESCIM work group to define requisite data elements for sampling and testing. The U.S. Army Corps of Engineers has developed an electronic data format and accompanying data processing software. The data format and data processing software are in use by two divisions, other federal agencies, and private industry. The Air Force Center for Environmental Excellence (AFCEE) has developed and implemented a similar program, the Environmental Restoration Program Information Management System (ERPIMS).

Discussion: DESCIM plans to develop a standard EDD and database, and then allow each component to determine if it will be adopted. Common electronic data formats will allow transfer of data among the components. Also, a common format enables efficient data entry and use of GIS databases to manage, track and query historical data. Standard electronic data will facilitate computer validation of the data. While electronic data validation cannot replace manual data validation, it can save time and increase accuracy for assessment of general data quality indicators such as spike recoveries, holding time excursions, and blank contamination. Some commercially available data validation software is capable of detecting certain types of fraud. Use of such software can serve as a deterrent to fraudulent laboratory practices.

Recommendation: The EDQW should evaluate the available electronic data formats and select one as the DoD-wide data transfer standard. The EDQW should also evaluate electronic data validation software and make recommendations regarding it's use.

VALIDATE DATA

Best Practice: Review and validate data collected for restoration or compliance program support. Determine the amount of data validation required during the DQO process. Summarize and report results.

Implementation Status: DoD currently reviews the quality and usefulness of the data collected as part of the Data Quality Objective (DQO) Process.

Discussion: Data review and validation ensure the reliability of analytical data. When performed in conjunction with previously determined DQOs, data of sufficient quality and quantity will be obtained for making decisions.

Recommendations: Data validation requirements should be identified and documented in advance of any sampling and analysis. Data validation requirements should be specified using a tiered approach oriented to the DQOs and specified in the QAPP, where applicable. Sample collection information should be included in this review because the external environment can impact the validity of the sample and the usability of analytical data. Summary tabulation of data and associated "flags" should be provided in a standard format to facilitate data review. The EDQW should continue to work with the IDQTF to develop common data validation practices for Federal departments/agencies.

INSTITUTE THE NATIONAL ENVIRONMENTAL LABORATORY ACCREDITATION PROGRAM (NELAP)

Best Practice: DoD, DOE, EPA and other affected departments and regulatory agencies are partnering to develop and implement a national program which sets minimum criteria for laboratory competency, assesses laboratories against those criteria, and monitors. on-going proficiency through a uniform laboratory accreditation system, such as the USEPA National Environmental Laboratory Accreditation Program (NELAP). In addition, components should consolidate program requirements and institute a DoD-wide laboratory approval program, consistent with the NELAP, to achieve uniformity in program requirements.

Implementation Status: EPA is working to develop and implement the NELAP, and DoD is an active participant in standing committees and subgroups tasked with program development. The proposed program incorporates uniform quality standards and reciprocal recognition of laboratory accreditation based on IS0 Guides 25 and 58, respectively. The DoD has also recognized the need to develop a consolidated laboratory approval program among Components, consistent with NELAP criteria, and the EDQW has already begun this process.

Discussion: EPA has the lead to set uniform quality and accreditation requirements for environmental laboratory testing, which will facilitate the comparison of laboratory performance and reciprocal recognition of laboratory services. The DoD fully supports the NELAP initiative.

Recommendations: DoD and EPA policy makers should work to achieve uniform laboratory quality and accreditation standards so that laboratories performing environmental testing meet minimum performance criteria and demonstrate on-going proficiency. Established standards must conform with International Standards for laboratory testing to ensure the widest acceptance of decisions based on testing data. The EDQW should incorporate IS0 standards through Component policy. In addition, the EDQW should continue to be involved in the NELAC process and consider applying for NELAC recognition as an Accreditation Authority. This would enable DoD to accredit in-house laboratories, thereby reducing national security concerns from external inspections and inconsistency from using State programs, while achieving mutual recognition from all Federal, state and territorial NELAP Accreditation Authorities. In addition, this would allow DoD to accept NELAP accreditation, on a matrix and method specific basis, as initial demonstration of a private sector laboratory's competency to perform DoD testing. This would reduce costs, by eliminating laboratory pre-approval inspections (restoration testing), and allow DoD to focus scarce resources on DoD and project specific requirements, including proper oversight of environmental sampling and testing activities.

Previous Section: Section 2: Improving Policy, Guidance, and Documentation
Next Section: Section 4: Improving Management and Contracting Practices