EPA Quality Management Tools for Projects
You will need Adobe Reader to view some of the files on this page. See EPA’s About PDF page to learn more.EPA has developed or adapted several quality management tools to assist in implementing its quality system. The tools and scope of activities supporting the EPA Quality System differ at each level of the organization.
See the EPA's Terminology Search Web service for definitions of common terms used in the context of EPA's Quality System.
Data Quality Assessment
Data Quality Assessment is used to assess the
- type,
- quantity,
- and quality
of data in order to verify that the
- planning objectives,
- Quality Assurance Project Plan components,
- and sample collection procedures
were satisfied and that the data are suitable for its intended purpose. Data Quality Assessment is a five-step procedure for determining statistically whether or not a data set is suitable for its intended purpose.
This assessment is a scientific and statistical evaluation of data to determine if it is of the
- type,
- quantity,
- and quality
needed and may be performed either during a project to check the process of data collection or at the end of a project to check if objectives were met.
References
- Data Quality Assessment: A Reviewer’s Guide (QA/G-9R) provides general guidance to organizations on assessing data quality criteria and performance specifications for decision making. G-9R is non-technical document and shows a reviewer what constitutes an appropriate Data Quality Assessment (DQA), and how to recognize situations or reports where a DQA has been conducted.
- Data Quality Assessment: Statistical Tools for Practitioners (QA/G-9S). This document can be considered the technical aspect of G-9R. The document is designed as a "tool-box" of useful techniques in assessing the quality of data. The overall structure of the document will enable the analyst to investigate many different problems using a systematic methodology.
- Checklist for Quality Concerns - Draft checklist and list of resources for evaluating secondary data.
- EPA's Science Policy Council Assessment Factors - General Assessment Factors for Evaluating the Quality of Scientific and Technical Information
Training
Visit our Training Courses on Quality Assurance and Control Activities page to review the courses below and many more.
- Introduction to Data Quality Assessment - This one-day course demonstrates how to perform a data quality assessment to evaluate data and provides detailed information on graphical and statistical tools. This course will familiarize participants with the process for performing a data quality assessments. It does not involve detailed instructions on the statistics involved in the process.
- Interpreting Monitoring Data - This half day course teaches participants how to deal with monitoring data using statistics. At the end of the course the participants will
- understand the need to incorporate systematic planning into monitoring activities,
- realize the importance of representativeness,
- be able to graphically view the data,
- and gain an insight into the complexities of statistical analyses of monitoring data.
- Interpreting Multivariate Analysis - This half day course covers multivariate analysis techniques to explore relationships among several variables in an effort to understanding an environmental problem. A prerequisite of the course is a general understanding of data collection and elementary statistical analysis.
Examples and Other On-line Resources
- Software - EPA has a site license for SAS for EPA employees - information on the EPA Intranet at the EPA SAS Support Home Page.
QA Project Plans
A Quality Assurance Project Plan documents the
- planning,
- implementation,
- and assessment procedures
for a particular project, as well as any specific quality assurance and quality control activities. It integrates all the technical and quality aspects of the project in order to provide a "blueprint" for obtaining the type and quality of environmental data and information needed for a specific decision or use.
All work performed or funded by EPA that involves the acquisition of environmental data must have an approved Quality Assurance Project Plan.
References
- EPA Quality Manual for Environmental Programs EPA CIO 2105-P-01-0 - Chapter 5 contains Quality Assurance Project Plan specifications for EPA organizations.
- EPA Requirements for Quality Assurance Project Plans (QA/R-5) - March 2001 (Reissued May 2006), EPA/240/B-01/003. Reissue Notice Defines specifications for Quality Assurance Project Plans prepared for activities conducted by or funded by EPA. These specifications are equivalent to Chapter 5 of EPA Manual CIO 2105-P-01-0.
- Guidance for Quality Assurance Project Plans (G-5) - December 2002, EPA/240/R-02/009. Guidance on developing Quality Assurance Project Plans that meet EPA specifications for new and existing data. Note: This document replaces EPA/600/R-98/018 issued in February, 1998.
- Guidance for Geospatial Data Quality Assurance Project Plans (G-5G) - March 2003, EPA/240/R-03/003. Guidance on developing Quality Assurance Project Plans for geospatial data projects.
- Guidance for Quality Assurance Project Plans for Modeling (G-5M) - December 2002, EPA/240/R-02/007. Guidance on developing Quality Assurance Project Plans for modeling projects.
- Guidance on Choosing a Sampling Design for Environmental Data Collection (G-5S) - December 2002, EPA/240/R-02/005. Guidance on applying standard statistical sampling designs (such as simple random sampling) and more advanced sampling designs (such as ranked set sampling, adaptive cluster sampling) to environmental applications.
- Guidance on Quality Assurance Project Plans for Secondary Research Data. Example Quality Assurance Project Plan specifications for secondary research data developed by the QA Managers in EPA's National Risk Management Research Laboratory.
- Example Checklist for a Developing or Reviewing a QA Project Plan (Example of a QA Project Plan Review Checklist (DOC)(14 pp, 153 K) ). From Appendix C of the Guidance on Quality Assurance Project Plans (G-5).
Training
Visit our Training Courses on Quality Assurance and Control Activities page to review the courses below and many more.
- Introduction to Quality Assurance Project Plans - This one-day course is designed for individuals who write, review, and/or approve Quality Assurance Project Plan and is designed with a "how-to" approach. Participants are expected to have some basic grounding in quality assurance principles.
- Introduction to Data Quality Indicators - This one-day course discusses the principal data quality indicators (DQIs):
- precision,
- bias (accuracy),
- representativeness,
- comparability,
- completeness,
- and sensitivity.
- verification,
- validation,
- and integrity
Examples and Other On-line Resources
- The Uniform Federal Policy for Quality Assurance Project Plans
The Intergovernmental Data Quality Task Force developed a policy and guidance document to fulfill the project-specific requirements of Part B of ANSI/ASCQ E4, to ensure that Federal departments and agencies will produce consistent quality assurance project plans (QAPPs) that reflect a systematic planning approach to collection and use of environmental data.
- Quality Assurance for Environmental Technology Design, Construction, and Operation
- References
- Examples and Other On-line Resources
Quality Assurance for Environmental Technology Design, Construction, and Operation
Environmental technology is an all-inclusive term used to describe
- pollution control devices and systems,
- waste treatment processes and storage facilities,
- and site remediation technologies and their components
that may be utilized to remove pollutants or contaminants from or prevent them from entering the environment.
Environmental technology is utilized in many configurations and is applied to many environmental problems, including devices and systems used in environmental programs to duplicate environmental conditions for test purposes or to
- control,
- prevent,
- treat,
- or remediate
waste in process discharges (e.g., emissions, effluents) or the ambient environment. Usually, this term will apply to hardware-based systems; however, it can also apply to general methods or techniques used for
- pollution prevention,
- source reduction,
- or containment
of contamination to prevent further movement of the contaminants.
Users applying environmental technology to these problems should have an understanding of the basic quality assurance (QA) and quality control (QC) practices that may be needed in
- planning,
- implementing,
- and assessing
the
- design,
- construction,
- and operation
of environmental technology. The needs of the user community may vary from applying QA and QC to design activities to the construction and fabrication of equipment systems to the testing and operation of completed systems.
Project managers should understand when and how QA and QC practices should be applied to engineering work, based on the range and scope of the needed environmental technology and its application.
The QA and QC practices necessary for environmental technology design and development are similar in function to those tools used for environmental data collection and use, but there are important differences in order to ensure that environmental technology performs as needed to resolve environmental problems.
References
- Guidance on Quality Assurance for Environmental Technology Design, Construction, and Operation (QA/G-11) - January 2005, EPA/240/B-05/001. Guidance provides users with an understanding of the basic quality assurance (QA) and quality control (QC) procedures that may be used in planning, implementing, and assessing the: design, construction, and operation of environmental technology, and complements the requirements defined in the American National Standard Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology Programs (ANSI/ASQ E4-1994) Exit by providing basic guidance on applicable QA and QC practices; outlining engineering planning, construction, and operation processes that may require QA and QC elements; and identifying resources and references that may be utilized by environmental professionals in the application of engineering-based technologies to environmental problems.
This document is not a manual on engineering but is a guide for project managers in environmental programs to help them to understand when and how QA and QC practices should be applied to engineering work.
This guidance is not written expressly for engineers but may be used by managers with non-engineering backgrounds. As a further aid, the guidance uses and refers to good engineering principles/practices (GEPs) when discussing the application of QA and QC during a project design, construction, or operation.
- ANSI/ASQ E4-2004, Quality Systems for Environmental Data and Technology Programs - Requirements with Guidance for Use. American Society for Quality (August 2004). American National Standard for quality management systems for environmental sector applications.
Examples and Other On-line Resources
American Society for Quality/Design and Construction Division Exit
American Society for Quality/Energy and Environmental Division Exit
Standard Operating Procedures
Standard Operating Procedures are written documents that describe, in great detail, the routine procedures to be followed for a specific
- operation,
- analysis,
- or action.
Consistent use of an approved Standard Operating Procedure ensures
- conformance with organizational practices,
- reduced work effort,
- reduction in error occurrences,
- and improved data comparability, credibility, and defensibility.
Standard Operating Procedures also serve as resources for training and for ready reference and documentation of proper procedures.
ReferencesYou will need Adobe Reader to view some of the files on this page. See EPA’s About PDF page to learn more.
- Guidance for Preparing Standard Operating Procedures (G-6) - April 2007, EPA/600/B-07/001. Guidance on the development and documentation of Standard Operating Procedures.
Examples and Other On-line Resources
Systematic Planning
EPA uses systematic planning to plan projects and link goals, cost and schedule, and quality criteria with the final outputs. Systematic planning ensures that all participants understand the needs and expectations of the customer and the product or results to be provided by the supplier.
Resources
- Resources for Planning New Data Collections
- Resources for Planning Projects that Use Existing Data
- Resources for Planning Information Products
Examples of Systematic Planning Processes
- Data Quality Objectives Process (EPA) - Planning process for decision making.
- Guidance on Systematic Planning using the Data Quality Objectives Process contains guidance on how to perform the Data Quality Objectives Process. Clarification of Terminology for the EPA Quality System. Memorandum from Norine Noonan dated March 2, 1999 that clarifies the use of "data quality objectives" vs. "acceptance or performance criteria."
- Systematic Planning: A Case Study for Hazardous Waste Site Investigations contains examples on applying the Data Quality Objectives process to hazardous waste site investigations.
- Systematic Planning: A Case Study of Particulate Matter Ambient Air Monitoring shows how the DQO Process was applied to a particulate matter ambient air monitoring problem.
- DOE Hanford Exit
- DOE Pacific Northwest National Laboratory Exit
- Dynamic Field Activities (EPA) - Planning process for Superfund.
- The Triad Approach (EPA) Exit- Planning process for hazardous waste remediation programs.
Software
- Visual Sample Plan (Pacific Northwest National Laboratory) Exit is a simple, defensible tool for defining an optimal, technically defensible sampling scheme for site characterization. VSP is applicable for any two-dimensional sampling plan including surface soil, building surfaces, water bodies, or other similar applications.
Technical Audits
Technical audits are systematic and objective examinations of a program or project to determine whether environmental data collection activities and related results comply with the project's Quality Assurance Project Plan and other planning documents, are implemented effectively, and are suitable to achieve its data quality goals.
Technical audits are not management assessments nor are they data verification/validation processes, which occur during the assessment phase of the project. Technical audits include
- readiness reviews,
- technical systems audits,
- surveillance,
- and performance evaluations.
Top of Page
References You will need Adobe Reader to view some of the files on this page. See EPA’s About PDF page to learn more.
- Guidance on Technical Audits and Related Assessments for Environmental Data Operations (G-7) - January 2000 (Reissued May 2006), EPA/600/R-99/080. Reissue Notice. Guidance to assist organizations in selecting, planning, and implementing technical audits in support of an environmental programs.
- Example Checklist for a Technical Systems Audit of a Laboratory Measurement System. This is an editable version of Appendix B of the above guidance. MSWord format | Example of a Technical Systems Audit Checklist for a Laboratory Measurement System (DOC)(8 pp, 136 K) .
Examples and Other On-line Resources
Verification and Validation
Data verification and validation is used to evaluate whether data has been generated according to specifications, satisfy acceptance criteria, and are appropriate and consistent with their intended use. Data verification is a systematic process for evaluating performance and compliance of a set of data when compared to a set of standards to ascertain its
- completeness,
- correctness,
- and consistency
using the methods and criteria defined in the project documentation. Data validation follows the data verification process and uses information from the project documentation to ascertain the usability of the data in light of its measurement quality objectives and to ensure that results obtained are scientifically defensible.
References
- Guidance on Environmental Data Verification and Data Validation (G-8) - November 2002, EPA/240/R-02/004. Guidance on environmental data verification, validation, and integrity.