Jump to main content.

Procedures for Detection and Quantitation

Policy Workgroup

Federal Advisory Committee on Detection and Quantitation Approaches and Uses in Clean Water Act (CWA) Programs

Summary of Policy Work Group Conference Call #6
January 9, 2006
1:10 – 2:55 p.m. EST

The next Policy Work Group will meet on Monday, January 23 at 1:00 PM EST. The call-in number is 1-866-299-3188. The participant code is 2025661045#.

Future calls are scheduled on Mondays, also at 1:00 PM EST as follows:

Welcome and Introductions

Facilitator Alice Shorett convened the conference call at 1:10 PM EST and called the roll of participants. She briefly reviewed the protocols for the call and the agenda for the call.

Top of Page

Meeting Summary Review and Approval

The draft summaries of the November 21 and the December 19 Policy Work Group meetings were approved as drafted.

Top of Page

Discussion of Measurement Quality Objectives/Data Quality Objectives (MQOs/DQOs)

Mary Smith, EPA, reported that she had run into vacation schedules in the agency when developing a document to present measurement quality and data quality objectives (MQOs/DQOs) for EPA programs. The chart she distributed to the Policy Work Group before the call presented the information she had obtained. She said she expected to receive materials from other programs later in the week and would update and distribute the chart in advance of the January 23 Policy Work Group call.

Ms. Smith asked Chris Hornback of Public Utilities to explain his caucus' desire (expressed during the advisory committee's December meeting) to set MQOs/DQOs in advance of the pilot study. She noted that EPA's guidance on how to validate methods directs the agency to learn from the validation process rather than setting MQOs/DQOs in advance. In response, Mr. Hornback said that public utilities use MQOs/DQOs more broadly, to define standards of what is and is not acceptable. With respect to pilot testing, the MQOs/DQOs could be a range of acceptable values. Through pilot testing the committee could see which procedures fell into that range.

David Kimbrough added that the Technical Work Group subgroup developing the design for a single laboratory pilot study, which intends to use real data, felt it would be difficult to know what to look for or when to recognize when it had been found without having MQOs/DQOs set in advance. Without MQOs/DQOs, which are tied to accuracy, to establish LQ, they would have nothing to go on. So, the subgroup felt it needed actual numbers to test, not outer bounds or ranges.

Larry LaFleur added that, in the pilot test, labs would have to pick spiking concentrations to meet MQOs/DQOs; labs would need them to figure out where to spike the concentrations and MQOs/DQOs were needed for that purpose. Nan Thomey pointed out that it would not be possible to pick a number for all analytes in all methods. Mr. Kimbrough said he believed this was a policy question, not a technical one: for a decision-making process for LQ, what level of bias and accuracy was acceptable? If no single number would work, he said, the net effect would be to raise LQ where accuracy is higher. Ms. Thomey said that she thought it was a circular policy decision because one also had to know where LQ was relative to water quality criteria.

Tom Mugan said he could understand both sides of the argument. One should be able to set the level - say 10% for accuracy and precision - and hit that at some point by increasing the concentrations. But he also appreciated Ms. Thomey's comments about the limit of sensitivity. From a policy perspective as a regulator, he said, if one had to raise LQ to "5" to achieve better certainty, while the WQBEL was at 1, LQ would have to be set at 5.

Ms. Smith asked if it made sense to agree on a set of five or six analytes - the less complicated ones (excluding phenols, for example) -- and agree on a number for them. Mr. Kimbrough agreed that it made sense to develop a model that worked for most analytes and methods and to develop a different approach for problems like phenols.

Ms. Smith asked if the committee needed to answer this question for the pilot study to proceed or could it pick a half dozen analytes and work on them. Mr. LaFleur responded that the committee needed to give general guidance on the range to explore because it affected spiking concentrations. They, in turn, would drive the design and implementation of the pilots. He said the Technical Work Group could proceed with design without the range but would need to have it set before implementation of the pilot test. He then asked if MQOs/DQOs should be set depending on the use, for example, for Clean Water Act regulation. One of the questions was: what was needed to have legally defensible data. Mr. Kimbrough posed the issue as follows: what level of bias in data or how inaccurate can data be for decisionmaking.

Ms. Thomey said that a key question for her, before agreeing on MQOs, was what were LC and LQ going to be used for. Mr. LaFleur agreed that MQOs for compliance might be different than those for setting permits. Under the Clean Water Act, Ms. Thomey said, there is not a single use so there is not a single set of MQOs. Mr. Kimbrough suggested identifying the most rigorous use - for compliance - and setting them based on that use.

Ms. Smith again asked whether or not the MQOs had to be set before the pilot test. She suggested instead that it might be productive to see what we could learn from the pilot, to see how the procedures worked, and save the tougher issues for continued discussion while the pilots were underway. Mr. Kimbrough repeated his belief that the pilot could not be implemented without the numbers and the uses. Ms. Thomey pointed out that there are costs associated with increasing and with decreasing uncertainty; zero uncertainty, she said, isn't affordable.

Mr. LaFleur pointed out that the Technical Work Group was looking for existing data for detection and quantitation. Until the Technical Work Group had target MQOs, it could not evaluate the data. He suggested that the subgroup designing the multi-/inter-laboratory study of quantitation could pose this issue at the March meeting for the committee to resolve.

Ms. Thomey asked if it would be possible to mine the existing data for information to help decide the MQO/DQO values. Mr. Kimbrough reiterated his belief that the question of MQOs was a policy issue, not a technical issue. Ms. Thomey said she felt the question was both and suggested that if a policy decision were made without consideration of technical feasibility, it wouldn't have been very useful. Ms. Smith again suggested learning from the data, to see what labs had been able to achieve.

Ms. Shorett asked Ms. Smith to describe the existing situation at EPA. Ms. Smith said that, in general, the air program was more flexible on accuracy while the water program was less so. One could take the Drinking Water/Clean Water Act approaches and use them, although she recalled that industry was suggesting different numbers. Mr. LaFleur said that her point was accurate. Industry proposed that precision and accuracy be set at ± 10% for more difficult analytes, with limits on LQ never below LD. For alpha (false positives) and beta (false negatives), industry favored 1%.

Top of Page

MQO/DQO Format/Template

The discussion then turned to a draft template prepared by facilitator Bob Wheeler and distributed with the agenda for the call. He reviewed the various sections of the document, noting particularly a table designed to show the relationship between the procedure options and MQOs/DQOs. The document would also explain the background, uses, and implications of the MQOs. If the Policy Work Group found the draft useful, he said, it could be developed as a product for the March advisory committee meeting.

In the discussions of the different parts of the draft that followed, Mr. Wheeler explained that the values in the draft were intended as illustrative examples only; the committee would determine the actual values that should be included. There were a number of questions and suggestions for revisions to enhance its value. Mr. Mugan encouraged the group to take a "middle of the road"/moderate approach for most analytes, such as 20% of standard deviation. He commented that while the committee could suggest how regulatory agencies should use the committee's results, in the end, the states themselves would react to what the committee came up with. He also pointed out that states typically did not take enforcement action unless there had been a "significant" violation; states, he said, look at multiple datasets and trends, rather than focusing and acting on a one-time result.

Mr. LaFleur reported his conversations with Jim Pletl about other possible parameters to consider for evaluating procedures. He said he was coming to realize that there are two levels of data: the quality of the data themselves and then evaluating the procedures from a DQO perspective. Perhaps, he said, the committee should set criteria to decide if different procedures met representativeness and robustness. He said that Mr. Pletl wanted to discuss this issue before pilot testing began. Later in the call it was suggested that the Technical Work Group could address the issue as part of its work on the pilot and desirable characteristics and frame it for the January 23 Policy Work Group call.

Mr. Murray questioned how valuable representativeness and robustness were relative to such other important features as do-ability, feasibility, reasonable cost, standard operating procedures, etc. Mr. LaFleur responded that the design subgroups were trying to allow for differences to emerge in the data sets. He said they were also trying to evaluate whether or not differences were a result of lab performance or lab design.

Ms. Thomey noted that the committee's task, ultimately, was to determine which procedures best solved the problems in the existing procedure. She suggested that the group should review a procedure to see if it solved the problems in the existing procedure. If not, she asked, why would one switch? That could be an evaluation criteria. Mr. LaFleur agreed that the committee was looking for a method that gave reliable detection and quantitation, based on what the committee sets. He said the Technical Work Group recognized that it had to do some method validation, so, it needed to set a number and see if procedures could hit the mark.

In summary, there was general agreement that the draft template was a useful approach and that revisions would improve its usefulness. David Kimbrough, Mary Smith, Tom Mugan, Mike Murray, and Nan Thomey agreed to work with Mr. Wheeler on revisions. Larry LaFleur said he would find someone to participate from industry. The group's assignment was to revise the draft document for discussion on the January 23 Policy Work Group call. Mr. Wheeler said that work would proceed by email and possibly conference call and that he would contact the subgroup with recommended times.

Top of Page

Draft Interim Report/Framework

Ms. Shorett turned discussion to the draft Interim Report/Framework that had been distributed with the agenda. She said it was based on an outline the committee had reviewed and approved during the December meeting. Its purpose, she said, was to serve as a common document for committee members to use with their constituents to describe the committee's work, to report progress, and to get feedback and input on issues the committee was addressing. It was to be provided to the committee by the end of January.

In the comments on the draft, it was suggested that references to committee documents at the EPA website be added. It was also suggested that specific questions for each of the issue areas would be a valuable addition. Dave Akers, Mary Smith, Chris Hornback, Cary Jackson and Mike Murray agreed to work with Ms. Shorett on revisions prior to the next January call. Larry LaFleur said he would recommend someone from industry to participate with the subgroup.

Top of Page

Additional Work on Uses

Mr. Wheeler pointed out that the committee had reviewed some but not all of the decision trees. For example, the decision tree on "reasonable potential" had not yet been discussed by the committee. He also asked for discussion on whether or not the MQO/DQO discussion would cover the decision tree use charts and the sections related to LC, LD, and LQ which basically referred to MQOs. He additionally asked how the committee wanted to proceed on the remaining uses that had not yet been addressed.

Mr. Kimbrough referred Policy Work Group members to a Technical Work Group document, dated 11/30/05 (Background Document from Technical Work Group (PDF3 pages, 18 Kb), which posed questions related to the issues of descriptive/prescriptive approaches, method validation and the lowest possible numbers for LC, LD, and LQ. He said he felt that the Policy Work Group needed to make policy recommendations on these issues. It was suggested that these issues would fit into discussion of MQOs/DQOs and the other decision trees.

It was agreed that David Kimbrough and Larry LaFleur would work with Mr. Wheeler to develop a recommendation to bring to the January 23 call on how to tie the prescriptive/descriptive issue into the decision trees. Mr. Wheeler suggested they have a call on Tuesday, January 11, to discuss how to proceed.

Top of Page

Wrap-up and Closing

In the wrap up, Ms. Shorett said that Triangle would propose times for the subgroups to work on their respective assignments. She said that she would try to use email to make progress on developing questions for the draft interim report.

She adjourned the call at 2:55 PM EST.

Attendance

Top of Page

Local Navigation

Jump to main content.