![[logo] US EPA](http://www.epa.gov/epafiles/images/logo_epaseal.gif){kind=logo}
Phase 3: Prepare Program
- Introduction
- Roadmap
- ERP Resources and Contacts
This phase is a time for you to make key decisions that will shape your ERP. You will lay out the parameters and timeframe for the program, as well as begin to think about who within your agency will be implementing it.
Identify Target Universe
The target universe comprises all of the facilities in your sector that you think match the qualifications you determined in Phase 1 for participation in your ERP.
Questions to consider
- How will you identify the universe of facilities you are targeting?
- Are you capturing informal facilities (i.e., those that are unregistered or unlicensed) as well as formal facilities?
- What resources can you use to make your identification more complete and accurate? Are there additional data sources from government or private industry, surveys by community groups or other such resources you can use?
- Are your sources of information reliable and consistent with your definition of the sector? Keep in mind that some sources of information may be out of date or incomplete. Others may define a sector differently than you plan to for the purpose of your ERP. Any of these factors will make your list of facilities less accurate, and therefore you should anticipate spending some additional time and resources to develop a complete list of facilities that matches your definition of the sector.
Resources to consider
- Establishing the Universe of Regulated Facilities
[Cached version] (PDF, links to page 3 of a 154 page document, 620 KB, About PDF)
(Excerpt from The Massachusetts Environmental Results Program: User's Guide for Government Agencies (March 2002))
Select EBPIs and Create Inspector Checklist
The Environmental Business Practice Indicators (EBPIs) represent the core indicators of environmental performance that you wish to measure. The inspector checklist is the primary means of collecting information about facility performance, including EBPIs. Both the EBPIs and inspector checklist should be developed in tandem with the statistical methodology. In addition, the inspector checklist should inform development of the self-certification form, described in Phase 4.
Questions to consider
- What aspects of performance are most important to measure?
- Can you adapt EBPIs and/or inspector checklists from other agencies that have developed an ERP for this sector?
- How detailed do you want to make your inspector checklist?
- Will it include primarily EBPIs, or will it be more comprehensive?
- Do you want to be able to make a complete determination of compliance status on the basis of the checklist?
- To what extent will best management practices and pollution prevention opportunities be included?
- What other information on facility characteristics do you want to collect?
Resources to consider
Devise Statistical Methodology
The statistical methodology will delineate the statistical capabilities and limitations of your program. The statistical methodology should be developed in tandem with the data collection tools. The statistical methodology has implications not only for data collection tools, but also for the inspection strategy and schedule, as well as how you can interpret program results. For example, the statistical methodology will set the stage for the types of conclusions you can draw from your analysis of ERP data, and the level of confidence you can have in your results.
Questions to consider
- How do you want to analyze results (e.g., by estimating baseline performance, comparing baseline inspection data with post-certification data, comparing samples from different regions or between different types of facilities, comparing self-certification data to inspection data, etc.)
- Will you analyze results on whole facility basis or equipment basis? This is especially important for the UST sector, where the decision must be made between analyzing on a facility or tank basis.
- How many inspections can you conduct with current or projected inspector resources? Will this provide a large enough sample size so that your measurements are sufficiently reliable and precise for your purposes?
- If inspection resources are severely limited, are there creative ways to expand inspection capacity (e.g., cross-training inspectors so that fewer inspectors need to be on each team, teaming with regional EPA inspectors, training interns/retired engineers/public health officials, or others to assist with inspections; reducing the number of questions; improving data entry efficiency)?
Resources to consider
- ERP Statistical Methodology (PDF, 463 KB, 97 pages, About PDF)
- ERP Sample Planner (MS-Excel Spreadsheet, 96 KB)
- ERP Results Analyzer (MS-Excel Spreadsheet, 128 KB)
- Statistical Methodologies
Develop Data Management Capabilities
The way that you will process data will affect the way that you collect data and vice-versa. Creating a data management plan will help you consider different options for managing and automating data collection, storage, analysis and reporting. It is important to realize the implications of decisions made at this step; some choices you make now may have implications for future development of your ERP program. For example, if you think you want to use optical scanning in the future to automate data entry, you should design self-certification forms now to enable optical scanning. Likewise, make sure your data is stored in a database format that can be easily expanded if you expect the program to grow over time.
Questions to consider
- What data systems does your agency already have in place that can serve as a foundation for your ERP data management approach?
- What databases currently exist for this population, and can they be used/modified for ERP?
- Will data from inspections and self-certification be collected on paper, electronically, or both?
- How will data be entered (e.g., manually, scanning, direct electronic submission)?
- What will the quality control plan be?
Resources to consider
- ERP Data Management Guide (PDF, 152 KB, 38 pages, About PDF)
- ERP Data Management Plans
Develop Data Evaluation Plan
The data evaluation plan considers what you will do with the data you collect through inspections and the self-certification process.
Questions to consider
- How will inspection data and self-certification data be analyzed?
- Who will receive the data, and what will trigger detailed review of a specific facility?
- To what extent will analysis of each data set be automated?
Resources to consider
- Evaluation Methodology [Cached version] (PDF, links to page 11 of a 154 page document, 620 KB, About PDF)
(Excerpt from The Massachusetts Environmental Results Program: User's Guide for Government Agencies (March 2002))
Phase 3: Stakeholder Involvement
Internal stakeholders are particularly important in this phase. Internal groups can assist in developing the target universe, providing feedback on the EBPIs and inspector checklist, and planning for data management and analysis.
Internal
- Involve program staff in decision making and program material development.
- Involve IT staff in data management and evaluation planning.
Resources to consider