Quality Management Plan: Grantees and Contractors

The Human Resources Office (HRO) conducts both employment and training activities for the Region, based on expressed management and staff needs. Management defines the duties, responsibilities and required performance levels for personnel involved in environmental programs and data collection. This is to ensure that personnel have the needed academic background, training and experience to fulfill the necessary function. HRO experts then translate these requirements into the appropriate position classifications and grade levels.

In general, technical staff is hired under scientific job classifications, i.e., environmental scientists, life scientists, environmental engineer, chemist, etc. The knowledge and qualifications necessary to receive these designations are dictated by the Federal Office of Personnel Management classification standards.

The knowledge and certifications for technical personnel performing certain duties are further specified by Agency directives:

• Compliance inspectors/field investigators (personnel who conduct field activities that may lead to or support enforcement actions) (EPA Order 3500.1)
  
  
• Contract managers (EPA Contract Management Manual, Chapters 7 and 8)
  
  
• Health and safety (EPA Orders 1440.2 and 1440.3)

Training requirements for technical staff generally are specified by Agency directives as prerequisites for certification. Supervisors may specify training for individuals on a case-by-case basis. Staff members also have individual responsibility for maintaining and expanding their qualifications. Each office has been requested to designate a training contact who organizes training to meet the program's needs. Training courses developed to comply with regional policies are generally non-technical, in areas such as cultural diversity.

To encourage professional development beyond initial qualifications, HRO sponsors training to maintain or increase the work effectiveness of technical employees. The Regional Training Catalog identifies the following training categories:

Regional management supports the belief that regular training is needed to maintain and improve performance by providing training information, and by paying costs of training taken in-house and also at external institutions. Rotational work assignments, such as the Intergovernmental Personnel Agreement Act (IPA) and career rotation programs, are also encouraged.

Training, although resource consuming, is a necessary component of the RQAM's responsibilities, as many of the POs/RPMs and contractors lack the experience and training to personally carry out technical duties, or are reluctant to take ownership of QA functions.

Technical specialists involved in providing training are usually those experienced in performing the function for which training is being provided (i.e., QA review, data validation, sample collection, custody and shipment, project reporting limits, analytical methods and QA oversight).

The primary means for identifying training needs are through the audit process, surveys, and one-to-one discussions between the RQAM and the programs. Training priorities are jointly developed and negotiated by the RQAM and the program offices as part of the annual QAARWP development.

QAS and technical specialists consist of environmental scientists, chemists, engineers, and life scientists. QA Training for technical specialists is encouraged by the RQAM to keep them abreast of innovations affecting their areas of responsibility.

RQAM designated technical specialists who conduct the majority of the QA review functions or oversees the contractor's performance of those activities, become qualified primarily through previous laboratory workbench and instrumentation experience and concurrent work assignments with experienced staff members.

Basic QA training is included in the 4-day Basic Inspectors Training Course required to be taken by all regional staff involved in EDCA. This training introduces the staff to the regional QA policies, QA guidance documents, the required QA-planning documents prior to data collection activities and the Regional QMP available on the Region 10 website. The RQAM and/or QAS also present QA workshops, open houses, and training as needed or upon request. In addition, a basic on-line QA training module shall be developed by the RQAM and the regional technical specialists. The RA through the QA re-commitment letter will require all regional staff to take the QA on-line training on an annual basis. The RQAM and/or QAS may provide training to Regional staff, representatives of other federal agencies, state grant recipients, and Indian tribes on QA-related topics including but not limited to:

• Orientation to QA management
• Preparation of QMPs, QAPP, and navigating the review and approval of these documents
• DQO process
• Sample collection, sample packaging and shipping, paperwork for sample collection
• Data generation and acquisition
• Assessment and oversight
• Data validation and usability
• QA oversight activities

Within OEA, all assessment team members receive appropriate training prior to conducting assessments. Region 10's QA Staff have attended Quality System Training Conferences or Workshops presented by the Agency's Quality Staff and received training for management system reviews, quality system assessments and/or data quality assessment. This training is recommended for new QA Unit assessors as available. All Drinking Water Certification Officers team have successfully completed a laboratory certification course offered by NERL-CI and are all familiar with quality assurance and quality control practices necessary to support data integrity, usability and defensibility. Annual refresher training (Training Module 97.00) is conducted for drinking water program assessors prior to the first on-site evaluation of that year. Personnel conducting assessments under NELAC are required to have the appropriate NELAC training (accreditation authority evaluator or laboratory assessor). Region 10 has 4 drinking water COs and two NELAC evaluating body COs.

Quality system assessors need to have direct experience with the implementation of quality systems before conducting assessments. Similarly, personnel leading technical systems audits should have experience or familiarity with the technical procedures they are auditing. Assessors in the Environmental Services Unit have been cross-trained (e.g., both field sampling and laboratory analysis) so that in addition to their own areas of technical expertise, they are familiar with other areas as well. When resources are available, a minimum of two assessors is preferred for conducting each assessment. In addition to increasing the degree of experience, this practice increases the level of competence and helps to prevent disputes over findings. The Assessment Team Leader and the RQAM review assessment plans to ensure that the designated assessors have no direct involvement or responsibility for the work being assessed, and that there are no real or perceived conflicts of interest. In the case of peer review, assessment personnel (reviewers) are required to respond to a Conflict of Interest Inquiry to avoid any potential conflicts.

The Region's industrial hygienist and Health and Safety Officer and HRO are responsible for maintaining personnel training records. The effectiveness of the training is assessed by HRO's review of the course evaluation portion of the SF-182 and by the supervisor's observation of work performance. Course evaluation forms are used to provide feedback to course instructors. Training completed is documented on EPA form SF-182, which is maintained in the employee's personnel file.

In response to the Clinger-Cohen Act (41 USC 433d, in collaboration with an interagency committee, the Federal Acquisition Institute (FAI) created the Acquisition Career Management Information System (ACMIS). ACMIS is a web-accessible database system that contains training and education records of the Agencies' acquisition workforce. ACMIS is designed to assist managers to make decisions on budget, staffing and training and to allow employees to easily access and update education, training and other career information records.

Region 10's OEA website contains a considerable amount of QA material, as well as, links to other useful QA sites. Access to Region 10's OEA website can be from the intranet for internal users, but also from the internet for outside users (i.e., contractors, PRPs, etc.) Here's the direct link to Region 10's QA webpage where a listing of QA Reference Documents, Analytical Methods, and Data Review Guidelines could be found.

The EPA Quality and Superfund Contract Laboratory Program (CLP) website also provides a wealth QA information and guidance that could be referred to during environmental measurement activity planning, implementation and evaluation.

Information Resources Unit of the Office of Management Programs (OMP) has the primary responsibility for setting policy and guidance for the management and development of computer-related program support in Region 10. The Director of this Office directs the Information Resources Unit (IRU) responsible for the Local Area Network, Geographic Information Systems, database management, information security, personal computing and information access, application development, desktop support, training, and records management. Personal Computing/Local Area Network coordinators in each Office act as liaisons between IRU and their office co-workers. The IRU is responsible for the following computer hardware and software support activities:

• Installing, configuring, testing, and troubleshooting network operating system and LAN based application software



• Troubleshooting and solving problems for the LAN server, data switches and routers, and any related gateways and communications equipment



• Supporting personal computer (PC) hardware and software procurement, system configuration, testing and installation of PC and peripheral equipment



• Coordinating system software and hardware changes for PC equipment



• Troubleshooting and fixing software and hardware problems reported by users



• Overseeing PC and LAN security



• Ensuring security software, system patches, system controls procedures and policies are implemented to prevent introduction of malicious software and computer viruses as well as to prevent unauthorized system/network access by hackers via the Internet



• Reviewing security practices to ensure that appropriate levels of information security are maintained. This includes review and implementation of policies and practices such as those that apply to user Ids and passwords, remote access, Internet, server system/data backup and recovery, physical access to regional data center and communication closets

All information management system development, improvements, and updates will comply with EPA Directive 2100, Information Resources Management Policy Manual and will include a systematic and comprehensive dialogue among the data providers, data and system users, and system developers, prior to the design and installation of the systems.

It is Regional policy to work closely with Regional information system customers, as well as OEI and national program offices as appropriate, on all phases of system development, improvements, and updates, including contractor developed and other entity developed systems. During all life cycle phases of information management systems, the Region will comply with requirements within EPA Directive 2100, Information Resources Management Policy Manual, the Interim Agency System Life Cycle Management Policy (PDF)(13pp, 229K) and Interim Agency System Life Cycle Management Procedure (PDF)(94pp, 1.3MB). Compliance with the applicable information resource management standards will ensure that all hardware and software configurations are developed, installed, and tested prior to use, to guarantee they perform as expected, are well documented, and meet user requirements.

The Region conforms to the System Design and Development Guidance and Operations and Maintenance Manual and the Delegation of Procurement Authority Guide. This ensures that purchased hardware and software will meet user requirements and will conform to OEI's technology and procurement guidelines. Other pertinent hardware and software standards utilized by the Region include: EPA's Network Security guidelines, Energy Star and Section 508 guidelines, and EPA's Hardware and Software Standards Roadmap.

The Information Technology Policies and Processes website also provides a comprehensive process followed by Region 10 staff when procuring computer hardware and software on a short-, medium- and long term priority scale. General information on information security, data access, confidential data and roles and responsibilities of each Region 10 employee are explicitly discussed in the Information Security website.

All federal agencies are required to adhere to federally mandated data standards and regulations. It is the policy of the Region to comply with all applicable regulations, guidance, executive orders, and internal policy documents concerning data standards. These include EPA's core information data standards: the Locational Data Accuracy Standard, the Facility Identification Standard, the Groundwater Data Element Standard, the Chemical Abstract Number Standard, and the Electronic Transmission of Lab Measurements Standard. It is the responsibility of individual Offices within the Region to be aware of the current standards and regulations. These standards and polices can be found at http://www.epa.gov/irmpoli8/. Other relevant data product standards include: standards for web product development contained in the EPA Web Guide and two related EPA Orders: 2190.1 Cookies and other User Tracking Methods and 2190.2 Children's Privacy and Copyright Issues.

Some of the program offices have their own database administrators who coordinate activities relating to their associated databases. Since these databases are national databases, requirements are defined by the national program offices at HQ. Regional data are collected, processed, and managed by the program offices. IRU manages the hardware, software and networking platforms. IRU also coordinates with the program offices on hardware and software issues, purchases and upgrades, and pilot programs.

For more information regarding Region 10's information technology strategic plans, roles and responsibilities, policies, security, resource management and projects, please click on this link: Region 10 Information Technology website.

The primary vehicles for annual planning in the Region are the budget process, GPRA and the State/EPA Agreement process. Depending on the sensitivity and level of work that may be involved, sites are prioritized by the RA/DRA. The DRA allocates resources to each office for the management and operation of specific programs, based on the Region's anticipated budget. Program managers balance their available resources with their projected need for support, e.g., QA and technical support from OEA and other Regional Offices to meet GPRA and program goals and commitments. Programs' full time employee (FTE) distribution (in terms of man-hours) in the Region's Technical Support Offices are revisited and negotiated. Acceptance of the program's proposal for FTE distribution results in work commitments.

Most regional work activities are mandated by policy guidance, congressional initiatives, commitments to HQ, and Program Office Work Projection Plan (WPP) prepared by each Program Office prior to the start of a fiscal year. It contains estimates of the work activities for that year. Ideally, the WPP is developed in consultation with the Program Offices' technical support offices. The RQAM seeks input from the offices it supports in preparing the QAARWP prepared for each fiscal year.

All program and project managers are responsible for (1) specifying, prior to the initiation of data collection and/or usage, their data quality needs; (2) providing adequate resources to ensure that the needs will be satisfied; and, (3) ensuring that the quality control and quality assurance requirements are written and satisfied for all environmental data collection activities (EDCAs). This includes modeling projects and the use of historical data for decision making purposes. All personnel involved with EDCAs, including field, laboratory, project and data processing personnel, as well as, grant and contract officers, have the responsibility for ensuring the quality of the data associated with those measurements.

For new data collection activities, it is Region 10’s policy that high level decision makers use a systematic planning process, i.e. the data quality objectives (DQOs) process, as a mechanism for balancing conflicting demands and data quality needs to ensure that environmental monitoring will effectively support decision making. DQOs are qualitative and quantitative statements that define the type, quality, and quantity of data necessary to support defensible environmental management decision-making. The DQOs are developed before data are collected and consist of the following 7 steps: (1) description of the problem, (2) identification of the decision needed to solve the problem, (3) identification of the information and measurements needed to support the decision, (4) condition specifications (5) definition of the conditions which the decision maker will choose among alternative actions, (6) specifications of acceptable limits on decision error and (7) evaluation of the results of the 6 previous steps and develop the most resource efficient design for data collection that meets all DQOs. The DQO systematic planning process ensures that adequate and appropriate data will be collected to support project needs.

Region 10 also uses the project teaming approach when planning and implementing projects for all programs. During the planning phase of the project, the project team, comprised mostly of the technical staff from OEA, and the clients, i.e., program PMs, CORs, contractors and stakeholders, meet regularly to discuss concerns and issues related to project goals, schedule and milestones, roles and responsibilities, logistic needs, project data objectives, measurement objectives, data needs, measurement and data performance criteria and specifications, QA/QC requirements and criteria, sources of data – primary or secondary and criteria for data usability. Project decisions are reached through a general consensus of the project team members. Regional managers make decisions based on information provided by the technical advisory committees and by regulatory requirements. Development of performance acceptance criteria is a normal part of the planning process and accomplished based on cost-effectiveness and realistic capabilities of the measurement process. The results of the DQO process are documented in the QA Project Plan. EPA developed the following documents for the DQO process: “Guidance for the Data Quality Objectives Process (PDF)(121pp, 1.6 MB),” G-4, EPA/600/R-96/055, February 2006, or “Data Quality Objective Process for Hazardous Waste Site Investigations, EPA QA/G4HW(PDF)(143pp, 599K),” January 2000 or most recent version, depending on the program involved.

Before QA concerns are identified, the project team shall first establish the intended use of the data. This will determine the criteria that will be used to assess the quality of the data. Existing data that will be used for screening purposes do not require stringent QA requirements. A more robust data quality assessment will be needed for data that will be used to support decision making and at a minimum may need the following documents associated with the data: chain of custody, QA Project Plan, QC Narrative or Data Validation Report, QA/QC report and laboratory analytical data report. It is Region 10's QA policy to conduct data quality assessment of existing data (secondary data), prior to another project’s intended use. The region follows the EPA guidance document on data assessment, “Data Quality Assessment: A Reviewer's Guide (PDF)(61pp, 258K),” EPA QA/G9R, February 2006. The EPA Quality website provides a list of useful tools that can be used for the assessment of secondary data.

All environmental measurement activities require a mechanism for monitoring the effectiveness and adequacy of the QA measures integrated into the program. The standard oversight mechanisms used by the Region for Quality System Assessments are: (1) quality assessment review/audit (2) technical assessment review/audits (3) data verification and validation and (4) data assessment. Each type of review follows EPA specified planning, implementation and evaluation procedures as documented in the following guidance: Guidance on Assessing Quality Systems (EPA QA/G-3)(PDF)(83pp, 555K),” March 2003; “Guidance on technical Audits and related Assessments for Environmental Data Operations, (EPA QA/G-7),” May 2006; “Guidance on Environmental Data Verification and Data Validation, (EPA QA/ G-8)(PDF)(96pp, 387),” January 2008, “Data Quality Assessment, A Reviewer's Guide (PDF)(61pp, 259K),” EPA QA/G9R, February 2006; “Data Quality Assessment; Statistical Methods for Practitioners (PDF)(198pp, 2.3 MB),” EPA QA/ G9S, February 2006, and the “2003 NELAC Standard, EPA/600/R04-003 (PDF)(324pp, 2.4 MB).”

Overall, the outcome of the assessment is expected to: (1) identify strengths and weaknesses, (2) cause corrective actions to be taken to alleviate problems, (3) facilitate the initiation of changes to enhance the QA program, (4) serve as a vehicle for providing technical assistance, (5) enhance awareness and understanding of QA/QC policies and procedures and (6) provide a measurement of the effectiveness of QC in assuring the quality of data.

The qualifications and responsibilities of the assessors are discussed in section 3.8 of this QMP.

9.2.1 SUPERFUND'S CONTRACT LABORATORY ASSESSMENT PROGRAM

9.2.2 NELAP CERTIFICATION/RE-CERTIFICATION ASSESSMENTS

9.2.3 SAFE DRINKING WATER ACT (SDWA) ACCREDITATION PROGRAM

Assessments conducted under the Assessment Program must, by definition, produce a written report which summarizes the assessment, and states the findings, observations and recommendations for response actions. Without documentation in a final report, a review or evaluation is not considered an assessment.

The objective of the report is to communicate assessment results to the responsible level of management of the organization being assessed. Efficient communication of results allows management to implement timely, effective response actions so that the quality objectives can be met. Quality system assessments are typically reported to senior managers of the organization responsible for the work. Assessments of project activities are reported to the EPA Project Manager. Copies of reports for internal assessments of project activities conducted by Lead Organizations also are sent to the EPA Project Manager. The EPA Project Manager may request a review of an audit report by the QA Staff. Senior managers of the assessed organization are responsible for ensuring that any deficiencies found in quality system assessments are appropriately addressed. Project Officers and Project Managers are responsible for ensuring that findings from assessments of project activities are appropriately addressed.

The process for corrective action is the response to the findings of an assessment. The authority of the RQAM is relied upon by the programs and regional partners to point out the necessary corrective actions needed to address any QA vulnerability that may arise in environmental data collection activities conducted in Region 10, external agreements, regulatory processes, or day-to-day operations (e.g., use of non-approved methods for Clean Water Act compliance monitoring by a State program). If the RQAM and assessment team identify vulnerabilities through the audit processes, these vulnerabilities will need to be addressed by the respective office or organization audited. The RQAM will follow-up with the respective office or organization to ensure that a corrective action plan is developed and implemented by the office or organization to correct and/or resolve the vulnerability addressed in the audit report. Follow-up is performed by the RQAM or designee with the organization to ensure that the resolution reached has been implemented.

Essentially, the principal responsibility for the implementation of response/corrective action is that of the assessed organization. A written response is provided by the assessed organization for all assessment findings with objective evidence of the effectiveness of the correction, and with specified time frames for those actions in progress or planned for the future. For project activities, copies of all corrective action response letters and corrective action forms should be included as attachments to the QA Management Reports and included in the Final Project Report.

The authority and responsibility for verifying the timeliness and effectiveness of corrective action resides with the senior management ultimately responsible for the work that was assessed. The responsible senior manager or the RQAM may request the assessors who conducted the assessment to verify the effective implementation of corrective actions. Assessment follow up is documented and reported using the same process as the original assessment.

Where QA concerns remain unaddressed, the RQAM may elevate the vulnerability to an OD as discussed in Section 1.6 of this QMP.

QA implementation is a continuous improvement process. It is the responsibility of the program managers to ensure staff participation in all program reviews and to annually review all QA activities of their program and staff, e.g., DQOs and QAPPs are developed and approved in advance of project start-up, project –specific SOPs are in place and revised if necessary, and DQAs are conducted at the end of the project. All deviations and discrepancies noted during any independent or self-assessment review will be corrected promptly and modifications made, if necessary.

As an added activity for quality improvement process, routine contractor on-site evaluations are also performed by QAS to minimize errors, at the start of each sampling season, (March –May) each year. Field and laboratory on-site evaluations are conducted to determine the efficiency and technical proficiency of the contracted staff assigned to implement the projects. Each of the contractors is evaluated. The EPA approved site specific QAPP and the SOPs are used as a basis for the evaluation. Deviations from the set procedures and specifications of the QAPP are noted. Mistakes, poor techniques, documentation weaknesses as well as good practices are identified and noted. A report addressed to the PM is written by the QAS within 7 days after the on-site evaluation. The evaluation process determines techniques that could be improved, weaknesses and errors that could be eliminated in the future and good practices that could be used and adapted by other organizations. The RQAM communicates results of audit, assessments, and reviews with program counterparts to identify areas of excellence, and areas needing corrective actions.

Similarly, when the program or a partner to the Region communicates that QA vulnerability exists within its program or the program it oversees, the RQAM or designee may elect to perform a review. Upon completion of the review, offices or responsible organizations are expected to address any audit findings in writing, documenting the corrective measures to be implemented. Follow-up is performed by the RQAM or designee with the organization to ensure that the resolution reached has been implemented.

QSRs and direct communication with organizations responsible for performing environmental data collection activities for Region 10 is a necessary tool for the RQAM in detecting and preventing data quality problems. These quality problems may be related to the services provided by the organization, or more systemic problems which affect an organizations ability to produce data of known quality.

QSRs and direct communication with organizations responsible for performing environmental data collection activities for Region 10 is a necessary tool for the RQAM in detecting and preventing data quality problems. These quality problems may be related to the services provided by the organization, or more systemic problems which affect an organizations ability to produce data of known quality.

QSRs and TSRs will be periodically conducted by the RQAM to check the performance of the program, organization and/or offices performing environmental data collection activities by and for Region 10. Field and laboratory audits will be utilized as a preventive measure to ensure that data collection activities are supported by EPA approved QA documentation and data generated are of known and intended quality. It is expected that when deficiencies are identified, corrective action is initiated, documented and effectively implemented by the responsible project officer or manager.

The RQAM relies on the QSRs conducted by OEI every three years on the Region's QS to contribute towards its continual quality improvement. The OEI review is based on the QS described in this QMP. The QSR assesses the effectiveness of the Quality System implemented in the Region and verifies conformance of the implemented system with what is documented in the QMP. OEI identifies areas of QA vulnerability in the Region and ensures that these concerns are adequately addressed by the RQAM and the QAS. The RQAM also works with the RA/DRA and different offices in responding to the comments and developing Corrective Action Plans, if needed.