![[logo] US EPA](http://www.epa.gov/epafiles/images/logo_epaseal.gif){kind=logo}
Developing Program Evaluation Capacity
Demand for information about the performance, operation and results of programs has continued to increase. However, the use of program evaluation - here defined as the application of systematic analytic methods to address questions about program operation and results - has been undertaken on a more informal basis at EPA.
Program Evaluation Drivers
The Government Performance and Results Act (GPRA) has served as a driver for EPA to again focus on the need to more systematically utilize evaluation as it implements GPRA. GPRA requirements to measure program results, outcomes and performance in terms of goals and measures are creating an opportunity for the Agency to think more critically about the underlying logic of its programs. Program evaluation can help identify program impacts, reasons for observed outcomes and how programs can be improved. However, program evaluation should be viewed in a context broader than GPRA alone.
Other drivers for conducting program evaluation include Office of Management and Budget's (OMB) Program Assessment Rating Tool (PART). The PART is used to assess and evaluate programs across government and emphasizes use of "independent and quality evaluations" to determine if programs are effective and achieving results.
The President's Management Agenda (PMA) identifies five areas for improving the management and performance of the federal government using a scorecard to rate federal agencies on each initiative using a "score" of red, yellow, or green.
Finally, EPA's Innovation Strategy supports evaluation as a critical tool to mainstream innovation. Evaluation identifies opportunities that lead to system change, by focusing on effectiveness, significance, and potential transferability of innovative approaches.
Program evaluation provides a range of strategies and tools that can support the ongoing performance measurement that is required under GPRA and the other drivers discussed above. In supporting performance measurement, evaluation can be recognized as a continuous function that can encompass many opportunities including:
- Collecting useful and valid data,
- Analyzing and interpreting collected data,
- Defining outcomes to be measured,
- Effectively communicating results, and
- Developing useful documents to communicate lessons learned
By constantly evaluating its environmental programs, EPA is able to capitalize on lessons learned and incorporate that experience into other programs and environmental arenas. Evaluation of EPA's environmental programs improves the quality of public health and environmental protection, allows the Agency to continuously streamline and modernize its operations while managing our programs. This leads to continuous improvement, supports innovation, and better informs management decisions.
Building Evaluation Capacity
In addition to outlining its use in supporting performance measurement efforts, the Evaluation Support Division (ESD) is working to identify, build and leverage evaluation capacity throughout the Agency. This work demonstrates how program evaluation can be a valuable tool that can be considered as an integral component of the Agency's emerging results-based management framework.
ESD has continued its leadership role in identifying and building evaluation skill and knowledge throughout the Agency through a number of activities. These have included:
- Co-chairing the EPA Program Evaluation Network;
- Designing and conducting evaluation training.
- Serving as an advisor to program offices conducting evaluations.
- Networking with state, academic and other program evaluation practitioners outside of EPA.
- Serving as an information clearinghouse for information resources encompassing program evaluation standards, tools, journals and associations.
In continuing to build capacity for program evaluation in the Agency, ESD will continue to look to identify and diffuse knowledge of evaluation and seek to harness diverse information resources and tools. Environmental program evaluation has recently emerged nationally as a recognized sub-discipline. It applies the principles and approaches of traditional program evaluation, tailored to the specific needs and challenges of environmental policy management.
This evolving discipline is being undertaken with increasing frequency at the Federal, State, Tribal, and local levels of government. As it continues to develop, ESD will take a lead role in contributing to the understanding of how traditional evaluation methods can be employed in the environmental arena in the face of issues such as: imperfect information; data gaps; measurement difficulties; and lag times between activities and observed outcomes and effects.