Identified Limitations with Analyzing and Interpreting Annual Results Data and Charts

EPA's Office of Enforcement and Compliance Assurance (OECA) annually reports on its accomplishments including formal enforcement and compliance monitoring activities. OECA’s accomplishments are displayed in charts, which include a snapshot for the most recent fiscal year and the previous nine fiscal years. We use the term “snapshot” because the data presented was extracted on a specific date and data often changes. To ensure the highest quality data, we are continually reviewing data and will make minor corrections to records even after publication of the annual results charts. The Voluntary Disclosure chart does not show a previous 9-year trend as the methods for reporting these activities changed during this time.

EPA manages national information systems that include data provided by EPA regional and headquarters’ offices. Given the complex set of transactions, occasional problems occur with the direct entry or electronic transfer of data into the national databases. The data are compiled from:

  • the Integrated Compliance Information System (ICIS);
  • the eDisclosure system;
  • the Superfund Enterprise Management System (SEMS);
  • the Resource Conservation and Recovery Act - RCRAInfo data systems;
  • the Online Criminal Enforcement Activity Network;
  • the Budget Formulation System (BFS); and
  • manual submissions.

EPA places a high priority on ensuring the integrity of the information in the national enforcement and compliance databases. Despite the confidence EPA has in the data, there is potential for inaccurate record counts due to the large volume of information managed. The information in the underlying data systems is subject to correction and revision as errors or omissions are identified, so results may change after they are reported. While the total number of errors detected is typically small in any year, the charts may reflect different totals than reported in prior years for the same fiscal year and metric. Superfund charts are not updated for prior years. Annual Results is intended to illustrate general trends in enforcement activities with the most up-to-date data available at the time of publication.

While Annual Results charts present the total estimated quantity of pollutants within several categories (for example air, water, toxics, hazardous and non-hazardous waste, contaminated soil, contaminated water, etc…), individual pollutants included in each reporting category possess different types and magnitudes of risk. The summed estimated quantity of pollutants eliminated, treated, etc... cannot reliably be used for assessing the risk mitigated within each reporting category or from fiscal year to fiscal year.

More information on publicly available data (including further data breakouts) is available at the Enforcement and Compliance History Online (ECHO) and the EPA State Dashboards. Since data may be updated after Annual Results is published, precise numbers and values from ECHO may exhibit a small variance from the data presented in the Annual Results charts.

ECHO also provides information on state enforcement activities and identifies limitations on using state data on the ECHO Known Data Problems webpage.     Annual Results charts only include EPA activities, therefore limitations on state data do not apply. In addition, the Annual Results presented do not provide information on EPA informal enforcement activities or compliance assistance, which are other compliance assurance tools the agency employs to attain compliance and protect human health and the environment.

Below is a list of some specific known data limitations for the Annual Results data and trends charts.

In this section:

Injunctive Relief (aka Complying Action)

ICIS is the database of record for monetary value of injunctive relief (also known as cost of complying actions) and includes Comprehensive Environmental Response Compensation and Liability Act (CERCLA; also known as Superfund), cleanup agreement values. Those values comprise both the estimated value of cleanup work responsible parties agree to perform and payments to EPA for future cleanup costs in advance of the work being done (also known as cash out agreements). SEMS is the database of record for Superfund cleanup agreements and cost recovery settlements. The ICIS complying action data field includes the estimated value of CERCLA cleanup work responsible parties agree to perform as injunctive relief but does not capture CERCLA cashout values. Therefore, the injunctive relief and Superfund enforcement commitments charts include some of the same values for each measure.

Compliance Monitoring

Although Annual Results includes the total number of compliance monitoring activities conducted within a given fiscal year (FY), it is important to keep in mind that compliance monitoring can vary widely in terms of resources needed and activities completed. There can be significant variations in terms of complexity between statutory programs. 

In April 2020, OECA began employing a standard definition for “on-site inspections” and offsite compliance monitoring activities. An inspection is a compliance monitoring activity performed on-site at a regulated facility, whereas an off-site compliance monitoring activity is not conducted at the regulated facility. Due to these changes, caution should be employed when comparing counts of compliance monitoring activities conducted between FY 2020 – FY 2024 to activities conducted in FY 2019 and earlier.

Safe Drinking Water Act (SDWA) – Underground Injection Controls (UIC)

Beginning in FY 2021 the Agency collected all SDWA UIC compliance monitoring activities in ICIS. Prior to FY 2021, EPA received data on SDWA UIC compliance monitoring activities in both electronic and manual formats. For manually reported SDWA UIC activities, the Agency did not collect any facility details (such as company name or location) or the on-site/off-site breakouts. The manual SDWA UIC counts should not have included any SDWA UIC activities that were electronically entered into ICIS; however, EPA Headquarters does not have a mechanism to ensure that the manually reported activities were not also inadvertently reported electronically. On the federal compliance monitoring graph, the manually reported SDWA UIC counts are presented as a stacked bar and are included in the overall compliance monitoring counts.

Cleanups

Cleanups (both those conducted or required by EPA) can be very complex, and result in a wide range of costs and estimated completion times. Although EPA tallies and presents the total quantity and estimated cleanup costs in the Agency’s Annual Results, it is important to note that estimated cleanup costs and completion times are not comparable among the different facilities addressed.

Hazardous and Non-Hazardous Waste

In FY 2016, EPA began combining estimated reductions in hazardous waste and non-hazardous waste into one measure. The totals presented for FY 2015 are for estimated hazardous waste reductions only. Nevertheless, most of the estimated reductions throughout all fiscal years are attributed to hazardous waste. Bevill wastes are excluded from the reported total estimated amounts of hazardous waste reduced.

Contaminated Soil and Water

In FY 2018, the Agency began reporting estimated volume of contaminated soil and water to be cleaned up from all cases, whereas in previous fiscal years estimated volume of contaminated soil and water to be cleaned up was only reported for CERCLA and RCRA Corrective Action cases. CERCLA cases (both from private and federal facilities) account for most of the reported estimated total volume of contaminated soil and water to be cleaned up.

The estimated volume of contaminated medium addressed (VCMA) is reported in the year in which the enforcement action that initially required the cleanup is issued or entered. If a unilateral administrative order (UAO) is followed by a judicial consent decree that requires the same cleanup (i.e., the respondent did not comply with the UAO), the same VCMA is not counted again.

Environmental Justice (EJ) Data

FY 2014 was the first full fiscal year that EPA used EJSCREEN (a mapping and screening tool using nationally consistent data that combines environmental and demographic indicators into maps and reports) to identify facilities with enforcement actions located in an area of potential environmental justice (EJ) concern. Beginning in FY 2022, EPA began reporting EJ data for inspected facilities. 

A facility’s EJ determination is made at the time of the inspection or at the time a case is initiated. The number of conclusions shown for each FY on the Total Civil Enforcement Case Conclusions in Areas of Potential Environmental Justice Concern chart are lower than the reported civil enforcement case conclusions for each FY because cases initiated prior to FY 2014 are excluded.

Outcomes (penalties, injunctive relief, pounds reduced) are displayed from cases initiated since FY 2014, which have at least one facility in an area of potential EJ concern. For multi-facility cases where only some facilities are in an area of potential EJ concern, the EJ outcome (such as penalty, injunctive relief, or pounds reduced) reported is calculated by applying the percentage of the facilities located in an EJ area of potential concern to the total case outcome.

Not all cases are subject to EJ determinations and these cases are marked in ICIS as “Exempt” and in the charts as “EJ Not Applicable.” Cases that qualify as exempt from the EJ screening review are those that lack a meaningful facility location. Examples of cases that lack a “meaningful location” include:

  • the location of the violation, or location of identification of the violations, is not relevant to an EJ analysis (e.g., a product stopped at customs from entering the U.S.);
  • the location of the violations is such that the violations can’t affect a community (e.g., an offshore spill with no onshore impacts, a headquarters location for a Safe Drinking Water Act enforcement action addressing mobile vehicles such as trains, planes, or ships); or
  • the location of the facility is not relevant (or is only minimally relevant) to the enforcement action (e.g., bankruptcy, collection actions, cost recovery actions).

ICIS has a listing of EJ exemptions. If an activity and/or inspection addresses a facility that lacks a meaningful location and should be exempt from EJ analysis (aka EJ Not Applicable) but there is not a corresponding exemption listed in ICIS, then the facility may have been identified in ICIS as either “Not Exempt” or an incorrect EJ exemption may have been selected to correctly identify the facility as exempt.

Some older facility records in the ICIS data system had an EJ status of “TEMP,” which was a data entry placeholder used until the facility’s potential EJ concern could be determined. In FY 2021, the Agency retroactively updated facility records that still had the “TEMP.” status. These retroactive determinations were completed using FY 2021 information, which in some situations could be different than the possible EJ status at the time the case was initiated.

Enforcement

Enforcement: Contact Us
Contact Us to ask a question, provide feedback, or report a problem.
Last updated on December 5, 2024