Chapter 2

Quality System and Description

2.1 Graded Approach to Research Projects

Quality Assurance Project Plans (QAPPs) must be prepared to document QA/QC requirements for all research projects that involve the collection, generation, use, or reporting of environmental data (e.g. laboratory and field measurement projects, modeling projects, method development projects, etc.). A QAPP must be prepared before a research project begins. For QA planning purposes, a research project begins as soon as a TLP is prepared to collect data/information to support Agency decisions or for inclusion in research products (e.g., reports [both paper and electronic], journal articles, symposium/conference papers, extended abstracts, computer products/software/models/databases, or scientific data). Before proceeding with the collection of this data/information, a QAPP applicable to the project's QA category must be prepared. As part of the planning process for a research project, it may be necessary to perform preliminary work to gather information used to define its scope. This preliminary work must comply with the requirements specified in ORD Policy 13.4. Data collected during this preliminary work cannot be published.

The NRMRL quality system utilizes a "graded approach" for establishing appropriate requirements for QAPPs for various types of research activities. Under the graded approach, the intended use of the data and the visibility of the research effort dictate the required level of quality. For example, the quality controls of a fundamental research program are different from that of a regulatory compliance program because the purpose or intended use of the data is different. Once a graded approach has been applied to the components of a quality system, a graded approach is then applied to the documentation required by that system. The graded approach utilizes four (4) QA categories for research projects.

The four QA categories are:

• Category I – establishes QAPP requirements for projects involving areas such as enforcement activities, litigation, or research projects involving human subjects;
• Category II – establishes QAPP requirements for important, highly visible Agency projects involving areas such as supporting the development of environmental regulations or standards;
• Category III – establishes QAPP requirements for projects involving applied research or technology evaluations; and
• Category IV – establishes QAPP requirements for projects involving basic research or preliminary data gathering activities.

For both Category I and Category II projects, NRMRL follows, in its entirety, the R-5 document for QAPP preparation developed by EPA's Quality Staff (QS) (40 pp, 120 KB). Guidance for Quality Assurance Project Plans, EPA QA/G-5 (PDF) (111 pp, 401 KB) (EPA/240/R-02/009) December 2002 may be used to help address these requirements. If a particular R-5 requirement does not apply to an individual Category I/II project, the TLP provides a brief explanation for why the specific requirement does not apply. The Quality Staff has developed other guidance documents that maybe used in developing QAPPs. These documents are: Guidance for Geospatial Data Quality Assurance Project Plans, EPA QA/G-5G (PDF) (106 pp, 1.42 MB) (EPA/240/R-03/003) March 2003; Guidance for Choosing a Sampling Design for Environmental Data Collection for Use in Developing a Quality Assurance Project Plan, EPA QA/G-5S (PDF) (178 pp, 1.02 MB) (EPA/240/R-02/005) December 2002; and Guidance for Quality Assurance Project Plans for Modeling, EPA QA/G-5M (PDF) (121 pp, 615 KB) (EPA/240/R-02/007) December 2002. For projects of QA Category I or II involving design, construction, and/or operation of environmental technology, the requirements in ANSI/ASQ E4-2004: Quality Systems for Environmental Data and Technology Programs - Requirements with Guidance for Use, 2004, American Society for Quality (ASQ), Milwaukee, WI, 2004, Section 7 (Part C) and Guidance on Quality Assurance for Environmental Technology Design, Construction, and Operation EPA QA/G-11 (PDF) (86 pp, 244 KB) (EPA/240/B-05/001) January 2005, will be followed.

For Category III and IV projects, a subset of applicable R-5 requirements is utilized (see Appendix B). The outlines of NRMRL QAPP Requirements for various project types are condensed from typically applicable sections of R-5 and are intended to serve as a starting point when preparing a QAPP. Requirements are included for projects involving measurement research; secondary data; method development; modeling; software development/data management; and design, construction, and operation of environmental technologies. (In some cases, these types of projects may be designated Category I or II; additional requirements in R-5 would then be applicable.) As guidance, the requirements in Appendix B include short descriptions of the applicable types of research. These lists and their format may not fit every research scenario, and QAPPs must conform to applicable sections of R-5 in a way that fully describes the research plan and appropriate QA and QC measures to ensure that the data are of adequate quality and quantity to fit their intended purpose. Some research may not neatly fall into just one of the project types listed above and may combine features of more than one subset of requirements. For such a project, the EPA TLP should consult with the division's QA personnel to determine the most appropriate QAPP requirements.

The required QA category and associated QAPP requirements are determined by the EPA TLP at the beginning of a project, with the concurrence of the Division QAM. This is documented on the forms discussed in Chapter 4 and Chapter 7 in this QMP.

The graded approach applies to other aspects of the quality system. The following table summarizes these requirements for the different QA Categories.

*Additional sets of requirements may also be required depending on the nature of the project.
**If problems are found, all data sets will be audited.

Top of page

2.2 Quality-Affecting Activities

NRMRL's data generating research and testing activities always require QA oversight. However, NRMRL also performs other functions that may not always be attributed to project-specific efforts; certain aspects of NRMRL's operations, such as the use of in-house data and software systems, may be inherently part of projects and the quality of these systems is often not addressed in a QAPP. Establishing quality policy for these types of activities and monitoring these activities is typically performed through the development of SOPs, and in other cases, facility manuals. In addition, some generally accepted laboratory practices need to be implemented to ensure that a minimum level of quality is established in each laboratory. The list of minimum requirements for laboratory research activities is presented in ORD Policy 13.4. These requirements are a significant aspect of NRMRL's Laboratory Competency Implementation Plan.

Top of page

2.3 Principal Components of the Quality System

The following are the principal components of the NRMRL quality system.

2.3.1 NRMRL Quality Management Plan

The NRMRL Quality Management Plan documents the NRMRL quality system and encompasses the management and technical activities necessary to plan, implement, and assess the effectiveness of QA and QC operations applied to all research and environmental programs at NRMRL. The NRMRL Laboratory Director is responsible for the development of the NRMRL QMP, although its preparation is normally delegated to the NRMRL DQA with assistance from the Division QAMs. The QMP must be approved and signed by the NRMRL Laboratory Director, Division Directors, and the DQA. The QMP is submitted to the OEI QS for review. It must be approved and signed by the QS Director and the Assistant Administrator of OEI (AA/OEI). It is reviewed annually by the NRMRL DQA and QAMs and updated as needed. The QAARWP will be used to identify changes in the QMP prior to the next scheduled revision of the document. The approved QMP is valid for five years unless there is a significant change in the organization or in the research areas emphasized. After five years or a significant change, the QMP is reviewed and revised as needed and subjected to the same approval processes described above.

2.3.2 Quality Assurance Annual Report and Work Plan (QAARWP)

NRMRL assesses the effectiveness of its quality system on an annual basis. That assessment and planned work activities for the coming fiscal year related to the quality system are documented in the NRMRL Quality Assurance Annual Report and Work Plan (QAARWP). The NRMRL QAARWP is submitted to the QS at the conclusion of each fiscal year. Division-specific input for the QAARWP is prepared by the QAM for each Division. After review by the respective Division Director, the inputs of the Divisions for the QAARWP are provided to the NRMRL DQA. The DQA prepares the NRMRL QAARWP, which is reviewed by and discussed with the Laboratory Director before being submitted to the QS under the Director's signature.

2.3.3 Extramural Quality Management Plans

EPA contractors, assistance agreement recipients, and some EPA programs may be required to submit a Quality Management Plan. Again, a graded approach is used by the EPA TLP in consultation with the QAM to determine if a QMP is necessary (see Chapter 4). If a QMP is required, it is reviewed and approved by the QAM and EPA TLP. The NRMRL requirements for QMPs are given in Appendix B.

2.3.4 Systematic Planning

NRMRL research projects use a systematic planning process that involves defining project objectives and then designing and refining a plan that meets these objectives. All project stakeholders are included in the planning process. The graded approach is used to ensure that the level of detail is in accord with the intended data use and the resources available.

2.3.4.1 Planning Process

The NRMRL TLP is responsible for using the systematic process when planning their work effort. Although not mandatory, the Data Quality Objectives (DQO) Process developed by the EPA is an example of a systematic planning process. This process is described in Guidance on Systematic Planning using the Data Quality Objectives Process (PDF) (121 pp 1.61 MB) (EPA/240/B-06/001) February 2006.

2.3.4.2 Quality Assurance Project Plans

As discussed in Section 2.1 and Section 2.3.3 of this QMP, NRMRL uses a "graded approach" (i.e., project type) for the development of QAPPs. Determination of the appropriate project type is performed by the NRMRL TLP in consultation with other project participants and the Division QAM when needed. Different project types have different requirements as specified in Appendix B. The QAPP must be reviewed and approved by the Division QAM, with concurrence by the TLP's branch chief, before a research project begins.

NRMRL has developed QAPP requirements for each category of its graded approach (see Appendix B) and uses guidance documents developed by EPA's Quality Staff (QS) to assist technical staff in addressing both NRMRL and Division-specific quality specifications e.g., the R and G series of documents developed by QS.

2.3.4.3 Facility Manuals

NRMRL facility managers, lead researchers, and QA staff may develop facility manuals to assist technical staff (both EPA and contractor) in documenting the performance of routine operations. Facility manuals have been primarily used by NRMRL's division in North Carolina. A facility is defined as a building (or portion of a building) housing a set of equipment designed for a particular area of research such as a furnace to study the impact of combustion parameters on the pollutants emitted or a pilot plant to simulate a water distribution system. Facility manuals may contain the following information on a specific NRMRL facility:

1. Introduction
2. Facility Charter (purpose)
3. Management
4. Description
5. Equipment
6. Documentation
7. Operation
8. Quality Assurance
9. Quality Control
10. Data Handling
11. Corrective Action
12. Health and Safety
13. QA/Test Plans

Appendices may contain:

1. Current Facility Personnel
2. Operating Procedures
3. Standard Methods
4. Technical Systems Audit Checklist
5. Performance Evaluation Audit Ranges

Facility manuals are reviewed by the facility manager or lead researcher, QAM, and health and safety (H&S) personnel if H&S requirements are included in the manual; they are maintained and updated by facility managers. Formal revisions and reviews are initiated when substantial changes are made to a facility manual. A facility manual documents instrumentation and procedures, provides a historical record of facility use, standardizes procedures, provides a training aid for new employees, and facilitates preparation of QAPPs (which can reference standard sections in the facility manual, with description of the specific tests and schedules for testing).

2.3.4.4 Standard Operating Procedures and Methods

NRMRL develops SOPs for those operations that have become or will become routine (e.g., laboratory operations such as balance use, analytical procedures, sampling procedures, instrument calibration procedures) which are used for more than one project or performed by more than one person. Before SOPs are implemented, they are reviewed by a second person in the specific technical area and reviewed/approved by the QAM. The QAM may serve as the second technical person if they have expertise in the technical area. Otherwise, the QAM's review is to verify that the second technical review was performed and that QA/QC is addressed in the SOP. Concurrence by the branch chief (or designee) is required for branch SOPs, by the division director (or designee) for division SOPs, and the Laboratory Director (or designee) for NRMRL-wide SOPs.

Project-specific SOPs may also be written, and are particularly recommended to facilitate updating the QAPP or when the procedure is performed by contract personnel. Project-specific SOPs are reviewed and approved as part of the QAPP approval process. The purpose of the SOPs is to facilitate the uniform performance of routine procedures.

When appropriate, NRMRL uses approved standard methods (e.g., SW 846, MCAWW, EPA 500 and 600 series, Standard Methods, CFR) to meet project-specific objectives.

Top of page

2.3.5 Assessments

Audits are scheduled for specific systems, programs, and projects based on required periodic review, the importance of or public interest in the program or project, problems in the measurement system (noted by the TLP or QAM), requests by the TLP or managers (branch chiefs, division directors), or selection by the division QAM. The audits planned by the TLP at the beginning of a project or required by the program (e.g., SITE, ETV) are specified in the QAPP or QMP. For all assessments, the lead auditor/assessor (usually either the DQA or a QAM) must ensure that the auditor(s)/assessor(s) has/have no real or perceived conflicts of interest with the project/system/organization being audited/assessed.

2.3.5.1 Quality Systems Audits

A quality systems audit (QSA) is an on-site review of the implementation of an organization's quality system as documented in the organization's approved QMP. This review is used to verify the existence of, and evaluate the adequacy of, the internal quality system. A QSA may be a self-assessment or an independent assessment.

Independent QSAs of NRMRL are scheduled and conducted by personnel external to NRMRL, such as QS personnel. At his/her discretion, the NRMRL DQA may schedule a QSA of a Division to assess the implementation of the quality system in the Division and may enlist the help of personnel from other Divisions. In any event, the Division QAM is always invited as a member of the evaluation team to witness first hand the implementation of the QMP in their Division observed during the QSA and to respond to questions as needed.

2.3.5.2 Technical Systems Audits

A technical systems audit (TSA) is a qualitative on site evaluation of sampling and/or measurement systems. The objective of the TSA is to assess and document acceptability of all facilities, maintenance, calibration procedures, reporting requirements, sampling and analytical activities, and quality control procedures. Normally, an approved QAPP provides the basis for the TSA. Independent TSAs are most often scheduled by the TLP and conducted by the Division QAM. Assistance for the TSA is available to each QAM from other Divisions or from QA support contractors. TSAs are most useful when conducted early in the life cycle of a project when corrective actions (if necessary) can be performed that will minimize any loss of data. However, a TSA can be performed any time during a project's life cycle.

The NRMRL DQA may schedule project-specific TSAs to assess the quality system for individual projects in one or more Divisions and may enlist the help of personnel in other Divisions. In any event, the Division QAM is always invited as a member of the evaluation team.

TSAs are required for all Category I and II projects (both intramural and extramural). For Category III and IV projects, each division will audit two Category III/IV projects each fiscal year. At least one Category III/IV project must be an intramural project.

2.3.5.3 Audits of Data Quality

An audit of data quality (ADQ) is an examination of data after they have been collected and verified by project personnel. Assessing whether the Data Quality Indicator (DQI) goals specified in the QAPP were met requires a detailed review of the recording, transferring, calculating, summarizing, and reporting of the data. ADQs are required for QA Category I and II projects. An ADQ may also be conducted as required by specific programs or as requested by the TLP or other EPA stakeholder. ADQs may be conducted by EPA or contractor personnel.

2.3.5.4 Performance Evaluations

A performance evaluation (PE) is a quantitative evaluation of a measurement system. Although each measurement in a test program could be subjected to a performance evaluation, the critical measurements (designated in the QAPP) are more commonly evaluated. An evaluation of a measurement system usually involves the measurement or analysis of a reference material of known value or composition. The value or composition of reference materials must be certified or verified prior to use, and the certification or verification must be adequately documented. Ideally, the identity of the reference material is disguised so that the operator or analyst will treat the material no differently than a test program sample. PEs are conducted as required by specific programs or projects or at the request of the TLP or other EPA stakeholder.

2.3.5.5 Laboratory Surveillances

Laboratory surveillances are conducted using the requirements specified in ORD QA Policy 13.4. Each division will conduct surveillances of all laboratory areas at least once every three years. Surveillances of all or a portion of the laboratory areas will be conducted each year.

Top of page

2.3.6 Data Quality Assessments

A Data Quality Assessment (DQA) is a scientific and statistical evaluation to determine if validated data obtained from environmental data operations are of the right type, quality, and quantity to support their intended use. The DQA process is described in Data Quality Assessment: A Reviewer's Guide, EPA QA/G-9R (PDF) (61 pp, 258 KB) (EPA/240/B-06/002) February 2006. For research projects, these assessments are normally conducted by the TLP, and they should be performed routinely on all projects.

2.3.7 QA Review of Research Products

Final product reviews are conducted to evaluate the credibility of data, realization of project goals, comparability of data, validity of conclusions, and quality of data. Research products must be subjected to QA review prior to being distributed outside of NRMRL. A signed NRMRL Product QA/QC Verification Report form (Appendix C) must accompany the NRMRL products (reports [both paper and electronic], journal articles, symposium/conference papers, extended abstracts, computer products/software/models/databases, or scientific data) submitted for QA review if they contain environmental data. Environmental data are defined in EPA Order 5360.1 A2 as any measurements or information that describe environmental processes, location, or conditions; ecological or health effects and consequences; or the performance of environmental technology. For EPA, environmental data include information collected directly from measurements, produced from models, and compiled from other sources such as databases or the literature.

2.3.8 Computer Tracking System

Different computer tracking systems for the various elements of the quality system are in place in the various divisions. These systems track funding packages, QAPPs, QMPs, SOPs, audits, and project products.

2.3.9 Training

Mandatory quality system training for all NRMRL management and technical personnel is described in Chapter 3. The NRMRL DQA and QAMs discuss quality system training needs in the first quarter of the fiscal year during the regular monthly NRMRL QA teleconference. The goal of the discussion is to develop a consensus on the minimum quality system training to be required for all of NRMRL for that year and how it will be accomplished. Possible training methods are in-house by DQA or QAMs, contractor, agency presentations (e.g., IQGs), or computer-based courses. Each QAM will use Division-specific practices to determine the quality system training needs for the Division prior to this discussion. Division-specific quality system training will be in addition to NRMRL Laboratory-wide QA training.

Top of page

You will need Adobe Reader to view some of the files on this page.
See EPA's PDF page to learn more.