![[logo] US EPA](http://www.epa.gov/epafiles/images/logo_epaseal.gif){kind=logo}
|
Department Of Defense Audit Report - Laboratory Support Services for Environmental Testing - Part I: Audit ResultsAudit BackgroundEnvironmental laws and regulations were enacted to promote an environment free of hazardous substances by establishing guidelines for the use, storage, and disposal of hazardous substances. Since 1978, DoD organizations have been required to comply with Federal laws and regulations. State and local regulatory agencies have implemented Federal laws and regulations by issuing permits that specify the type and amount of contaminants permitted to be released into the environment under normal operating conditions. To ensure compliance with those permits, DoD organizations monitor the level of contamination released into the environment through sampling and environmental testing. We identified 72 DoD organizations that used 402 commercial laboratories and 41 Government laboratories to perform environmental tests included in contracts valued at $3.7 billion and $30.5 million, respectively. Commercial laboratories were the primary source for DoD organizations to obtain environmental tests. Three common contracting methods were used to obtain the environmental tests from the commercial laboratories: indefinite-delivery contracts, blanket purchase agreements (BPAs), and purchase orders. The Government purchase card was another contracting method; however, it was not used by any of the DoD organizations reviewed. Indefinite-Delivery Contracts: An indefinite-delivery contract is established for a fixed period of time with a qualified supplier. Orders are placed against the contract by issuing delivery orders. From FYs 1993 through 1995, the 18 DoD organizations that we reviewed used 25 different indefinite-delivery contracts, valued at about $21.3 million, to support their environmental compliance program and 16 different indefinite-delivery contracts, valued at about $557.5 million, to support their environmental restoration program. Compliance Program: The compliance program was established to ensure compliance with environmental laws, such as the Clean Air Act, the Clean Water Act, and the Resource Conservation and Recovery Act, by verifying that normal DoD operations did not contaminate the environment. Environmental tests are used in compliance programs to routinely monitor the levels of contamination present in the environmental media (air, water, and soil). Restoration Program: The restoration program was established to clean up existing contamination on DoD property. Environmental tests are used in restoration programs to identify the type and extent of contamination and to prove that the hazardous substances have been cleaned up so that the contamination does not exceed a specified level. Restoration contracts are established to accomplish all phases of the cleanup process; environmental tests are just one part of that process. Therefore, the value of the restoration contracts, $557.5 million, represents the costs for the entire cleanup process, not just testing costs. The DoD organizations that we reviewed with restoration contracts did not account separately for the amounts spent on environmental tests. We were unable to review the testing costs for the restoration contracts because the costs were not identified on the contracts. However, we were able to identify and review the quality assurance procedures used by the DoD organizations. Blanket Purchase Agreements: A BPA is a simplified procurement method of filling anticipated repetitive needs for supplies or services by establishing buying agreements with qualified suppliers. BPAs identify the ordering procedures, a general description of the needed supplies or services, a specified period of performance, a dollar limitation for each purchase, invoicing procedures, and a list of individuals authorized to make purchases under the BPA. From FYs 1993 through 1995, the 18 DoD organizations that we reviewed issued 34 different BPAs, valued at about $5.2 million, for environmental tests that included different requirements and deliverables. Purchase Orders: A purchase order is an offer by the Government to make a one-time purchase of specific supplies or services from a commercial source under specified terms and conditions for an aggregate amount not exceeding the simplified acquisition threshold. From FYs 1993 through 1995, the 18 DoD organizations that we reviewed awarded 51 different purchase orders, valued at about $0.6 million, for environmental tests. Government Purchase Card: The Government purchase card is an alternative to traditional contracts and simplified purchasing methods. It is used to purchase supplies or services not exceeding $2,500 from commercial sources. From FYs 1993 through 1995, none of the 18 DoD organizations reviewed used the Government purchase card for laboratory support services for environmental testing. Audit ObjectivesThe overall audit objective was to evaluate the cost and operational effectiveness of laboratory support services provided for environmental testing. Specifically, we determined whether DoD organizations were effectively managing their contracts for environmental test services and whether DoD organizations were effectively performing quality assurance procedures on test results received from commercial laboratories. We also reviewed the management control programs at the audited organizations applicable to the audit objectives. See Appendix A for a discussion of the audit scope and methodology and for the results of the review of the management control program. See Appendix B for a summary of prior coverage related to the specific audit objectives. Procurement and Administration of Environmental Testing ContractsDepartment of Defense organizations were not efficient and effective in the procurement and administration of environmental testing contracts. This occurred because contracting and environmental personnel did not consolidate procurements, used inadequate procurement and administration procedures, and did not have adequate quality assurance procedures to determine the quality of environmental test results. This resulted in environmental test prices ranging from a difference of 23 to 662 percent for the same tests, high procurement and administrative costs, and unreliable test results. Environmental Testing Contracts: DoD organizations that we reviewed were not efficient and effective in the procurement and administration of environmental testing contracts. Commercial laboratories charged DoD organizations significantly different prices for environmental tests. The contracts awarded and administered by DoD organizations to obtain environmental tests were not always the most efficient and cost effective. DoD organizations that we reviewed did not adequately determine the quality of environmental test results that they received from commercial laboratories. Environmental Test Prices: The DoD organizations that we reviewed did not always pay the lowest price for environmental tests. DoD organizations were charged significantly different prices for the same environmental tests by the same commercial laboratory. Also, individual DoD organizations were charged significantly different prices by different commercial laboratories for the same environmental tests. Different Prices Charged to DoD Organizations by the Same Commercial Laboratory: Test prices charged by the same commercial laboratory for the same environmental tests varied significantly for the DoD organizations that we reviewed. Table 1 shows a sample of price differences charged by three commercial laboratories to DoD organizations during FYs 1994 and 1995. Consolidated procurements could have eliminated these differences and could have resulted in lower test prices for the affected organizations. For example, if Public Works Center (PWC) Jacksonville and PWC Pearl Harbor had used a consolidated procurement administered by one procurement activity with Anacon, Incorporated, for the “total petroleum hydrocarbons” test and the unit test price was negotiated at the price charged to PWC Jacksonville ($20 each), PWC Pearl Harbor could have saved at least $130 for each of these tests ordered. We could find no differences in the test methods used by the commercial laboratories or any reasons for cost differences, except for each organization accepting a price that they considered adequate because procurements were not consolidated. 
Table 2 shows the highest and lowest prices charged by 18 commercial laboratories during FYs 1994 and 1995 for 5 test methods approved by the Environmental Protection Agency (EPA). Appendix D shows additional examples of significantly varied test prices charged by commercial laboratories during FYs 1994 and 1995.
Different Prices Charged to the Same DoD Organization by Different Commercial Laboratories: Different commercial laboratories performing testing for the same DoD organization charged significantly different prices for the same environmental tests. Table 3 shows the price differences that five commercial laboratories charged the U.S. Army Corps of Engineers, South Pacific Division for the same four EPA-approved test methods during FY 1995. Personnel from the U.S. Army Corps of Engineers, South Pacific Division rotated the contract orders and sent samples to all five commercial laboratories with varying prices for the same environmental tests, rather than determining the most economical prices for each environmental test and using that contractor for only those specific tests.
|
8080 | |||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
*The contractor’s price list did not include a price for this test.
As shown in Table 3, the U.S. Army Corps of Engineers, South Pacific Division, could spend from $325 to $401 (a 23-percent difference) for the EPA-approved 8270 test or from $98 to $289 (a 195-percent difference) for the EPA-approved 8080 test. We could not determine the exact amount of savings that could be realized by using the lowest priced laboratory because not all of the ordering and invoicing records were available.
Procurement and Administration Procedures
DoD organizations that we reviewed were not efficient and effective in the procurement and administration of environmental testing contracts because contracting and environmental personnel used inadequate procedures to award and administer the contracts. DoD organizations did not determine which type of contract was the most efficient and cost effective method to procure environmental tests. DoD organizations that we reviewed did not establish adequate procedures and training for personnel who administer and account for environmental testing contracts.
Awarding of Environmental Testing Contracts: All 18 DoD organizations that we reviewed did not determine which method of procuring environmental testing was more efficient and cost effective. They did not research or document their actual costs associated with awarding the different contracts. The DoD organizations that we reviewed had not considered consolidating procurements with other organizations. The cost to award indefinite-delivery contracts was higher than the cost to award BPAs or to use the purchase card. During FYs 1993 through 1995, 18 DoD organizations initiated 2,600 environmental testing requests (407 indefinite-delivery orders and 2,193 BPA orders) that were valued at less than $2,500 each. Indefinite-delivery contracts involved the extensive processes of preparing a solicitation describing the work required, advertising the solicitation to prospective contractors, reviewing contractor proposals (including its costs to complete the required work and its technical capabilities), and selecting the most cost-effective company best qualified to accomplish the required work. BPAs required the preparation of a written agreement with any laboratory that the DoD organizations chose; the Government purchase card did not require any award process, which saved additional time and money. All of the contracting personnel stated that indefinite-delivery contracts would have the highest award costs and consolidated procurements would provide lower contract costs.
Administration of Environmental Testing Contracts: The 18 DoD organizations used inadequate procedures to manage their environmental testing contracts. The DoD organizations had not established adequate procedures to administer environmental testing contracts and did not properly train the personnel responsible for ordering, administering, and accounting for the contracts. The 18 DoD organizations had numerous weaknesses in administering environmental testing contracts, such as:
-
Environmental personnel split test requirements into several smaller orders to avoid exceeding the dollar limitation for BPA orders that was established by the contracting department.
-
Environmental personnel did not validate test results.
-
Environmental personnel rarely adjusted the estimated dollar values of BPA orders after receipt of contractor invoices to determine the amount available for additional orders.
-
Contracting personnel issued and environmental personnel used purchase orders to pay for orders previously placed against BPAs because funding limitations were exceeded on the BPAs. Order amounts on three of five BPAs reviewed exceeded the U.S. Army Corps of Engineers, South Pacific Division, established funding limitations by about $32,500.
-
Unauthorized personnel placed orders against BPAs.
-
Environmental personnel duplicated or skipped BPA order numbers.
-
Personnel did not maintain complete contract files that included key documents, such as the original contract, contractor price lists, invoices, and environmental test results.
-
Prior to acquiring copies of reports or complete invoices, personnel approved contractor invoices for payment.
Appendix E identifies a detailed analysis of all the weaknesses identified in administering environmental testing contracts at the 18 DoD organizations we reviewed.
Procurement and Administration Costs: DoD organizations incurred unnecessarily high procurement and administrative costs. The 18 DoD organizations did not use the least expensive method, the Government purchase card, to procure environmental tests below the $2,500 micro-purchase threshold. In addition, the 18 DoD organizations did not use consolidated procurement and administration procedures for environmental tests above the $2,500 micro-purchase threshold.
Government Purchase Card: The 18 DoD organizations did not use the least expensive method to purchase environmental tests that did not exceed the $2,500 micro-purchase threshold. The least expensive alternative to traditional contracts and simplified purchasing methods is the Government purchase card, as shown in Table 4.
| Indefinite Delivery | BPA | Government Purchase Card | |
|---|---|---|---|
| Average Cost to Administer Orders* | $129 | $72 | $40 |
*Defense Logistics Agency cost.
Since none of the organizations could determine the actual costs involved with administering indefinite-delivery contracts, BPAs, and purchase card purchases, we applied the Defense Logistics Agency estimates. We estimated that the 18 DoD organizations could have saved about $106,400 in administrative costs from FYs 1993 through 1995 if the purchase card had been used for 2,600 environmental testing requests valued at less than $2,500 each. This savings represents the difference when using the purchase card rather than indefinite-delivery contracts or BPAs ($36,200 for 407 testing requests that were ordered through indefinite-delivery contracts and about $70,200 for 2,193 testing requests that were ordered through BPAs).
Consolidated Procurement and Administration Procedures: DoD organizations did not consolidate procurement and administration procedures to efficiently and economically procure environmental tests above the $2,500 micro-purchase. During FYs 1993 through 1995, the 18 DoD organizations placed 482 testing requests (183 indefinite delivery orders, 268 BPA orders, and 31 purchase orders) valued at more than $2,500 each. Consolidated procurement and administration procedures would eliminate the duplicative tasks that the DoD organizations perform under their non-consolidated approach. Each DoD organization performed the same procedure on multiple contracts, rather than having one procuring organization perform all of the procedures.
Quality Assurance Procedures
The 18 DoD organizations did not have adequate procedures to determine the quality of environmental test results provided by commercial laboratories. Quality assurance methods to verify the quality of environmental test results received from commercial laboratories were not standardized by DoD. The most effective quality assurance methods were not always used by DoD organizations to ensure the accuracy of environmental test results received from commercial laboratories. The DoD organizations prepared inadequate statements of work that did not ensure that the commercial laboratories provided accurate environmental test results.
Quality Assurance Standardization: DoD did not establish standardized quality assurance methods to verify the quality of environmental test results received from commercial laboratories. All of the 18 DoD organizations used different quality assurance methods to verify the quality of environmental test results. For example, the U.S. Army Corps of Engineers, Missouri River Division, inspected commercial laboratories every 18 months, validated the environmental test results, and used laboratory certifications as ways of assessing the quality of environmental test results. In addition, Hill Air Force Base verified the quality of environmental test results by auditing the laboratory and ensuring that the laboratory was certified. The Defense Fuel Supply Center and the Marine Corps Combat Development Command, Quantico, only relied on the contractor to be certified by the state or EPA.
Senate Appropriations Committee report number 102-408, September 17, 1992, directed the Secretary of Defense to provide financial data and management information on the cost of performing environmental tests. Also, DoD was to report on the full extent of all laboratories used by DoD and their potential use for environmental sampling, analysis, and monitoring of laboratory work. The Army, in responding to the Committee, identified the overall lack of quality assurance standardization and stated that standardized quality assurance procedures were needed to ensure the consistent quality of environmental test results. However, no standardized quality assurance procedures were established. We agree with the Army that standardized quality assurance methods should be established to verify the quality of environmental test results.
Quality Assurance Methods: DoD organizations did not always use the most effective quality assurance methods to confirm the quality of environmental test results. The 18 DoD organizations used differing methods to determine the quality of environmental test results received from commercial laboratories. Seven of the 18 DoD organizations used the quality assurance methods identified by EPA as being ineffective in verifying the quality of environmental test results. The remaining 11 of the 18 DoD organizations used a combination of ineffective and effective quality assurance methods. EPA recommended five quality assurance methods as the most effective and three additional methods as less effective. Table 5 shows the effective and ineffective quality assurance methods used by the 18 DoD organizations.
Ineffective Methods: Of the 18 DoD organizations, 7 organizations administering contracts valued at $12.9 million, to include $9.3 million for restoration contracts, only used the quality assurance methods that EPA identified as ineffective in verifying the quality of environmental test results such as laboratory certification, past performance of the commercial laboratories, and single-blind performance evaluation (PE). See Appendix F for definitions of these quality assurance methods.
Laboratory certifications only identify the type of tests that the laboratories were qualified to perform at some point in time, based on the requirements of the regulatory agency at the time of certification, and do not detect problems with environmental test results. Past performances of the laboratories are only effective in verifying the quality of environmental test results when the current results of the environmental test are determined to be unusual as compared with past test results. Our analysis showed that laboratories rarely submitted current environmental test results that deviated from past test results that had been determined to be normal or usual. Another ineffective method that was used by the DoD organizations was the single-blind PE sample. Although a laboratory would not know the concentration levels of contamination in a single-blind PE sample, it would know that the sample is a PE, prompting the laboratory to exert an extra effort to perform a thorough analysis.
The above methods were not an effective means of detecting problems in the quality of environmental test results or laboratory falsification of environmental test results. Using the above methods for determining the quality of environmental test results from commercial laboratories does not assure that the environmental test results received from the commercial laboratories are accurate.
Effective Methods: Of the 18 DoD organizations, the remaining 11 organizations used a combination of quality assurance methods identified by EPA as effective and ineffective. The EPA has identified and recommended the following methods as being the most effective for evaluating the quality of environmental test results and providing a greater assurance of the quality of environmental test results.
-
Data validation - provides assurance that data are adequate for the intended use.
-
Double-blind PE sample - shows the ability of a laboratory to produce accurate results because the laboratory is unaware that the samples are a performance evaluation.
-
Laboratory audit - identifies technical areas that may cause laboratories to improperly identify or quantify chemicals.
-
Magnetic tape audit - detects whether complying with its contract, the integrity of the laboratory’s and the appropriateness of any software editing. the laboratory is computer systems, and the appropriateness of any software editing.
-
Split sample - verifies the use of proper analytical methodology and detects unusual data trends.
None of the 18 DoD organizations used magnetic tape audits, and only 2 organizations used double-blind PE samples to verify the quality of environmental test results. Methods that organizations used most often included: data validation, split samples, and past performance. We could not determine if organizations combined both effective and ineffective methods for every contract or only used one effective or ineffective method because documentation was not always available at the DoD organizations specifying which quality assurance method was used to assess each commercial laboratory.
Statements of Work: The statements of work and description of services provided to the commercial laboratories were not sufficient to ensure that the commercial laboratories reported accurate environmental test results. The statements at the 18 DoD organizations did not always adequately identify the tasks to be performed by the commercial laboratories under the contract and did not always identify Government remedies for inadequate contractor performance.
Detailed Clauses: The statements of work for 27 of the 59 indefinite-delivery contracts and BPAs reviewed did not include the tasks to be performed by commercial laboratories under the contract. In addition to the 59 indefinite-delivery contracts and BPAs, 51 purchase orders were reviewed that did not contain any detailed description of services to be provided by the commercial laboratories. Tasks to be performed by the commercial laboratories should include using EPA-approved test methods, providing test results in a standard format within a specified time frame, and performing quality assurance procedures to ensure the reliability of the test results.
Indefinite Delivery Contracts and BPAs: The 27 indefinite-delivery contracts and BPAs only contained general statements of work that did not ensure the contractors would provide quality test results. The statements did not contain any details on what environmental tests to perform, how to perform the environmental tests, or how and where to report their results. For example, the only contractor requirement on the statements of work used by the Corps-SPD, was for the contractor to perform chemical analyses when requested by the contracting officer or an authorized representative. In contrast, a better example of a contract with some detailed clauses in its statement of work was a contract issued by PWC Jacksonville. The statement of work included a well-written description of the work to be performed under the contract; sample collection, shipment, and preservation procedures; proper analysis procedures; quality control procedures to implement; and reporting requirements.
Purchase Orders: The 51 purchase orders did not contain any detailed description of services the commercial laboratories were to provide. The only commercial laboratory requirement on these purchase orders was for the laboratory to perform environmental testing.
Government Remedies: Only one of the statements of work and descriptions of service, by the Naval Weapons Station Yorktown, specified what Government actions would be taken to penalize contractors for inadequate performance. Commercial laboratories performed required tests incorrectly, which required retesting that eventually delayed environmental projects. Also, commercial laboratories submitted late reports of test results and committed gross clerical errors on test reports, without those laboratories receiving any penalties for their poor performance. For example, one of the commercial laboratories used by PWC Jacksonville incorrectly analyzed samples and had to retest the samples. This retesting compromised PWC Jacksonville’s ability to meet Federal and State regulatory compliance time frames for hazardous waste holding and monitoring, as specified in its permit requirements, but the commercial laboratory was never penalized for the incorrect analysis.
Relying on Test Results
DoD organizations that we reviewed could not rely on environmental test results because no standardized quality assurance procedures existed to verify the quality of the results. The DoD organizations used quality assurance methods that were not the most effective, and the statements of work and description of services provided to the commercial laboratories were not sufficient to ensure that the commercial laboratories reported accurate environmental test results.
Organizations could have increased their level of quality assurance if they had used the quality assurance methods recommended by EPA. For example, by using the recommended quality assurance methods such as magnetic tape audits and double-blind PE samples, the EPA was able to determine that the computer used by a commercial laboratory to produce test results and the laboratory itself were incapable of producing accurate data. As a result, EPA detected the falsified data produced by the laboratory before it affected the using organizations. The quality of environmental test results is ensured only when effective quality assurance methods are used.
Summary
DoD organizations were not efficient and effective in procuring and administering environmental testing contracts because the organizations did not use consolidated procurement procedures, used inadequate procurement and administration procedures, and did not have standard quality assurance procedures for determining the quality of environmental test results. The results of our audit are specific to the 18 DoD organizations that we reviewed. However, the conditions identified could apply to every DoD organization because every organization is required to comply with all environmental laws and regulations. Therefore, all DoD organizations could have the same adverse conditions as those reported because DoD non-consolidated procurement and administration procedures have resulted in numerous inefficiencies. Consolidated procurement of environmental tests could be performed using the procedures in the Defense Federal Acquisition Regulation, Subpart 208.70, “Coordinated Acquisition, ” December 3 1, 1991, which governs procurements for which the contracting responsibility is assigned tthene agency, such as Military Traffic Management Command and its consolidated procurements for household goods shipments.
Recommendations, Management Comments, and Audit Response
Revised Recommendations: We deleted the portion of the recommendation about centralizing procurements for laboratory testing. The Principle Assistant Deputy Under Secretary of Defense (Environmental Security) stated that “centralization is the antithesis of procurement reform and empowerment of contracting and technical personnel, ” and that the data in the report does not support centralization as a solution to the problems. the data, we revised the recommendation. After further analysis of We want to point out that under acquisition reform, regionalization and consolidation of contracts is occurring for items and services as diverse as canned motor oil, pharmaceuticals, and Travel Agency Services. This consolidation of contracting is reducing the umber of procurement actions and costs while helping improve quality. We believe that with the new Laboratory and Testing Group working on the issue and expanded use of the Government purchase card, reduced contracting costs for environmental laboratory testing services should occur.
We recommend that the Deputy Under Secretary of Defense (Environmental Security) establish an integrated process action team to include the Services and the Defense Logistics Agency, obtaining assistance from the Office of the Deputy Under Secretary of Defense (Acquisition Reform) and the Director of Defense Procurement as needed, to consolidate the requirements for the procurement of environmental tests during FY 1997. The process action team should also accomplish the following tasks.
Management Comments: The Principle Assistant Deputy Under Secretary of Defense (Environmental Security) nonconcurred with the recommendation to establish an integrated process action team. However, he stated that they have established a DoD Laboratory and Testing Work Group in September 1996 to develop coordinated environmental positions and make recommendations for action. He is adding representatives from the Director, Defense Procurement and Under Secretary of Defense (Acquisition Reform) to the group to address the actions recommended in this report.
Audit Response: The establishment of the DoD Laboratory and Testing Work Group in September 1996 satisfies the intent of the recommendation. The addition of acquisition and contracting officials to the Working Group will provide needed expertise to resolve the problems identified.
-
Develop standardized statements of work and description of services for environmental testing to be used in all environmental testing contracts. They should include a well-written description of the work to be performed under the contract, sampling procedures, proper analysis procedures, and reporting requirements.
Management Comments: The Principle Assistant Deputy Under Secretary of Defense (Environmental Security) nonconcurred with the recommendation to develop standardized statements of work and description of services. He stated that accepted Environmental Protection Agency procedures exist for performing required tests and the suggested formats could be helpful.
Audit Response: The acceptance of existing Environmental Protection Agency procedures and suggested formats for customizing the requirements for performing required environmental tests satisfies the intent of the recommendation.
-
Establish standardized DoD procedures to ensure that Environmental Protection Agency approved quality assurance methods are used to adequately determine the quality of environmental test results received from commercial laboratories.
Management Comments: The Principle Assistant Deputy Under Secretary of Defense (Environmental Security) nonconcurred with the recommendation. He stated that the DoD is examining the need for additional guidance on the appropriate use of various Environmental Protection Agency quality assurance guidelines through the DoD Laboratory and Testing Work Group.
Audit Response: Examining the need for additional guidance on the appropriate use of various Environmental Protection Agency quality assurance guidelines through the DoD Laboratory and Testing Work Group satisfies the intent of the recommendation.
-
Establish adequate procedures to administer environmental testing contracts.
Management Comments: The Principle Assistant Deputy Under Secretary of Defense (Environmental Security) partially concurred and stated that the DoD Laboratory and Testing Work Group will address contract administrative procedures as part of its review.
Audit Response: The planned actions satisfy the intent of the recommendation.
-
Establish procedures to ensure remedy clauses are included in environmental of holding the contractor accountable performance. that adequate Government testing contracts as a means for inadequate contractor
Management Comments: The Principle Assist mt Deputy Under Secretary of Defense (Environmental Security) partially concurred and stated that the DoD Laboratory and Testing Work Group will address the need for installations to make full use of existing remedy clauses.
Audit Response: The planned actions satisfy the intent of the recommendation.
Previous Section: Executive Summary
Next Section: Appendix A: Scope and Methodology