Department of Defense BEST PRACTICES FOR DATA QUALITY OVERSIGHT OF ENVIRONMENTAL SAMPLING AND TESTING ACTIVITIES - IMPROVING MANAGEMENT AND CONTRACTING PRACTICES

Management needs to facilitate exchange of laboratory performance information throughout DoD to rapidly identify data quality problems so that they do not become widespread. In addition, -. using performance based criteria as a basis for contracting laboratory testing services will improve acquisition as well as reduce costs. Best Practices include:

SHARE LABORATORY PERFORMANCE DATA

Best Practice: DoD shares laboratory performance information within DoD and other federal agencies. DoD considers past environmental laboratory performance during laboratory selection.

Implementation Status: DoD Components currently operate separate laboratory approval programs, and they typically contract for laboratory services through a prime contractor, using different laboratory acceptance criteria. This results in program dissimilarities which make sharing lab performance data difficult. There is currently no centralized database that tracks laboratory performance analogous to the Architect/Engineer Contract Administration Support System (ACASS) or Construction Contractor Appraisal Support System (CCASS) for tracking contractor performance.

Discussion: Setting uniform requirements among DoD components is requisite to effect a level playing field for sharing laboratory performance reviews and performance data. The DoD plans to achieve this goal through development of consolidated program requirements, which parallel those developed for NELAC. Development of an easily accessed database that contains laboratory performance information will facilitate use of quality laboratories and recognition of laboratory problems. The database could include information about laboratory performance similar to contractor performance recorded in ACASSKCASS. This is an interim step to streamline the system while standard guidance procedures using IS0 Guide 25 and IS0 Guide 58 are developed and instituted by the NELAC. Under NELAC, lab audit and PT results will be made available in a national database.

Recommendations: DoD, DOE, EPA, and other government agencies should share environmental laboratory performance data during laboratory selection and ongoing proficiency testings. The EDQW should resolve program differences that make reciprocity difficult among the components. The EPA should proceed with NELAC. The EDQW should develop a database to track laboratory performance so laboratory strengths and weaknesses can be monitored between components and across programs.

USE STANDARD PERFORMANCE BASED LABORATORY QA/QC CONTRACTS

Best Practice: DoD environmental contracts for data services should require laboratories to have in place a quality system that meets IS0 Guide 25 criteria and demonstrates compliance through an accreditation program which meets IS0 58 criteria. Contracts should require NELAP accreditation when the program is implemented. DoD contracts for environmental testing services should be based on best value and not purely on cost.

Implementation Status: DoD components' laboratory contracts have many of the same general requirements. DoD is increasing the use of quality-based contracts, even for compliance testing services, which were historically low-bid contracts.

Discussion: The EDQW is tasked with improving contracting procedures among the services. DoD needs to incorporate additional performance-based standards for acquiring commercial laboratory services. This should include developing contract award criteria, setting on-going performance standards, developing standardized Statements of Work, and having appropriate remedy clauses. Incorporating Performance Based Measurement Systems (PBMS) also introduces contract flexibility, which encourages the use of innovative technologies for sampling and testing activities. Use of innovative technologies can reduce cost, increase timeliness, and increase data reliability.

Recommendations: The EDQW should facilitate setting DoD policy for quality systems in sampling and testing and unify laboratory QA system requirements for contract testing among components. These policies can be incorporated in contract specifications and serve as a basis for improving DoD contracts, sharing performance information and exercising remedy clauses. Quality system criteria also provide a basis for awarding value based contracts. In addition, the EDQW should provide templates for use in preparing contracts in the field and new contracts should include Performance Based Measurement Systems (PBMS) flexibility, where appropriate quality systems and accreditations are in place. The EDQW should also investigate the feasibility of using centralized or regional contracting. Part of this investigation should include benchmarking industry and tracking the success of a comparable centralized contracting program which has been in operation for at least one year. Recommendations on the use of centralized contracting will be based on the investigation.

MAINTAIN DoD CORE CAPABILITY IN ENVIRONMENTAL DATA ANALYSES

Best Practice: DoD maintains a core capability in environmental testing for the restoration and compliance programs.

Implementation Status: Presently DoD components have a core capability in environmental analyses. Numerous initiatives are underway to reduce infrastructure, consolidate, and regionalize in-house laboratory services.

Discussion: Although DoD makes extensive use of commercial laboratories for environmental testing, DoD also needs to retain a core technical capability in order to develop contract specifications, manage contracts for testing services, assess contractor performance, and protect the government's interests throughout environmental data collection and analysis activities. The DoD must also maintain core competencies for component unique testing, and provide the capability and capacity to conduct short turn-around, mission critical, and emergent sampling and testing services. As a whole, the DoD currently contracts out about 80% of testing services. Each component continuously reviews these activities for opportunities to improve efficiency and reduce cost through increased out-sourcing. DoD components are also reducing infrastructure and consolidating laboratories to achieve a core capability structure which is cost effective and can be sustained for mission readiness. The Navy is tasked as the lead service for environmental data quality and in this capacity interfaces with private and public sector agencies to coordinate, review, and comment on legislation and regulations which could adversely impact maintaining functions which are inherently governmental or mission critical.

Recommendation: The EDQW should develop a core capability model and rationale. This model and rationale should focus on maintaining core laboratory competencies necessary to maintain the capability to perform quality assurance oversight of contracted services and laboratory infrastructure required to support mission needs at minimum costs.

USE A QUALITY ASSURANCE OFFICER

Best Practice: All DoD projects involving environmental analyses should have a DoD employee, acting on behalf of the DoD, as a laboratory data quality assurance officer (QAO). The Quality Assurance Officer (QAO), however named, provides independent review and oversight of data collection. Laboratories performing testing must also have a designated QAO per ISQ. 25 quality system criteria.

Implementation Status: DoD uses QAOs on many large projects. In-house laboratories also have QAOs to provide independent review and QA/QC oversight of laboratory services. Typically, commercial laboratories also have a designated QAO. EPA's Executive Order 5360.1 requires assignment of a quality assurance manager (QAM) to function independently of direct environmental data generation, model development, or technology development responsibility and reports on quality issues to the senior manager having executive leadership authority for the organization. The QAM must possess sufficient technical and management expertise and authority to conduct independent oversight of and assure the implementation of the organization's quality system.

Discussion: An IS0 Guide 25 based quality system requires that laboratories have a designated QAO. The QAO should be technically qualified and independent of the project manager or laboratory supervisor responsible for the testing performed. The QAO is directly involved in the project from the requirements planning stage through closure. Project QAOs ensure that DQOs are established and incorporated into the Field Sampling Plan (FSP) and Quality Assurance Project Plan (QAPP). The QAO develops a systematic review plan for sampling and data collection. Laboratories must ensure the independence of the QAO in reviewing data and reporting results.

Recommendation: The EDQW should review the role of QAO's in laboratory and field testing, sampling operations and project management across DoD. The review should include the description of duties and the level of independence relative to the oversight function. A report will be issued detailing the adequacy of the various QAO oversight functions and any needed improvements.

Previous Section: Section 3: Improving Laboratory Oversight Practices
Next Section: Section 5: Next Steps