On-Site Technical Assistance Program - Customer Feedback Survey - 104(g)(1)

March 1999

Executive Summary

The purpose of this report is to summarize the findings of the On-Site Technical Assistance Program customer feedback survey.

What is the On-Site Technical Assistance Program?

Since the enactment of the Federal Water Pollution Control Act of 1972, more than $85 billion in federal, State and local funds have been invested in constructing and upgrading municipal wastewater treatment facilities. The 1996 Needs Survey estimates an additional $75.9 billion in wastewater treatment needs alone (Secondary Treatment, Advance Treatment, Infiltration/Inflow Correction, Replacement/Rehabilitation, New Collector Sewers and Interceptor Sewers). In FY 1982, EPA implemented an on-site technical assistance program designed to address non-compliance at small wastewater treatment facilities through on-site operator training and other operation and maintenance (O&M) assistance.

This program is authorized under Section 104 of the Clean Water Act (CWA). Section 104(g)(1) of the Act authorizes EPA to finance pilot programs, in cooperation with State and interstate agencies, municipalities, educational institutions, and other organizations and individuals for the purpose of providing an adequate supply of trained personnel to operate and maintain existing and future treatment works and related activities. For the past six years, the guidance has expanded to encourage States to use these funds to promote Municipal Water Pollution Prevention, Energy and Water Use Efficiency, and recently we added sewer system maintenance to control Infiltration/Inflow and Storm Sewer Overflow (SSO).

The need for individualized technical assistance has been great and directly related to community size. There are over 12,500 municipal wastewater treatment plants smaller than 1.0 million gallons per day (MGD) operating today in this country. Over half of these plants have sophisticated activated sludge treatment technologies, and many of these small town operators may not possess the highly developed skills needed to operate such plants. Small plant turnover rates are high, budgets and salaries are low and community support may be lacking. All these ingredients may contribute to non-compliance. Towns with these problems are candidates for the On-site Assistance Program.

Each year, grants are awarded to State Water Pollution Control Agencies or State Environmental Training Centers which provide this assistance to facilities that are generally less than 5 MGD design flow. Since the program began in 1982, more than 5,000 Publicly Owned Treatment Works (POTW) have received on-site assistance.

Greater than seventy-one percent (71.6%) of respondents asked for assistance and over seventeen percent (17.2%) of the respondents were referred by a State.

Since 1982, assistance providers have submitted semi-annual reports as part of the grant requirements. These reports provide information on the number of facilities assisted, the number of facilities that achieved compliance or improved performance, and the number of facilities that demonstrated improvement. For the purpose of this 1998 survey, three hundred and forty surveys were mailed through out the entire U.S. for this project. Of the 274 total responses, only one hundred and twenty-four (124) of the respondents from thirteen (13) States, were eligible to participate in the On-Site Technical Assistance Customer Survey. The 124 responses that were used to create this survey resulted from facilities that were able to qualify that they received their compliance assistance from the Wastewater Treatment Plant Operator On-Site Technical Assistance Training

Program - 104(g)(1). The other respondents, 150 of them to be exact, claimed that their compliance assistance was provided by their State, or that they were unaware of the origins of who provided the compliance assistance at their facility. Even though compliance assistance was provided by this Program to the above-mentioned 150 respondents, the responses supplied by these 150 facilities were not used due to the apparent disconnect between the assistance that was provided and this Program.

Survey Findings-

The On-Site Technical Assistance Customer Survey findings are as follows:

1. Ninety-five percent (95%) of respondents felt the assistance was helpful;



2. Almost ninety-six percent (95.8%) of respondents would recommend providers to another facility;



3. More than ninety-two percent (92.4%) of respondents felt their employees' skills and capabilities were enhanced;



4. Almost ninety-two percent (91.8%) of respondents would request further technical assistance;



5. More than seventy percent (70.5%) of the respondents were returned to compliance after assistance was provided;



6. Ninety-four percent (94%) of the respondents waited less than 120 days for assistance. Furthermore, more than seventy six percent (76.2%) of the respondents waited less than one month and over thirty-four percent (34.7%) of the respondents waited less than one week for assistance from the Program;



7. Almost eighty-nine percent (88.8%) of the respondents knew how they were referred to the on-site technical assistance providers;



8. Almost ninety-seven percent (96.3%) of respondents said that it would have cost them more if the service would have been provided by someone else; and



9. More than sixty-six percent (66.1%) of respondents did agree with the statement that their facility would benefit from having technical assistance more easily accessible.

Recommendations-

The analysis of survey results suggests the following recommendations:

1. The facility in need should be informed up-front about the wait time for on-site training;
2. Establish follow-up procedures to clarify when a facility has reached compliance;
3. Data supports continuation of Program; and


4. Clarify accounting procedures to determine who is participating in the technical assistance Program.

The Program's Operation and Maintenance Coordinators in conjunction with Headquarters are in the process of developing a national tracking database for the Program. This database will allow the Program Coordinators to better serve its customers by identifying the areas of the Program that need improvement. The database will help to estimate a waiting time for assistance, help establish follow-up procedures (if necessary), and clarify who exactly is participating in the Program. The database will be developed and in use by June 1999.

Introduction

The Purpose of this Report-

The purpose of this report is to illustrate the On-site Technical Assistance Program's customer survey. Superintendents of wastewater treatment plants from 13 states were questioned for this survey. Furthermore, this report describes the background, objectives, research design, data collection procedures, analysis, and findings of the customer survey.

On-site Technical Assistance Program Work Group-

The employee On-site Technical Assistance Program Work Group was established to collect information on participants of the Program. The work group developed a two-phase approach to systematically acquire preliminary survey feedback on the proposed survey and the implementation of the survey instrument in 13 identified states.

The first phase consisted of individual, in-depth interviews, with Washington, D. C. area superintendents at wastewater treatment facilities. The purpose of these interviews was to test the instrument for clarity of instructions and content applicability.

The second phase of this project was to define the subject of this report. The final survey was developed and administered in 1998 by the Municipal Assistance Branch of the Municipal Support Division in Headquarters EPA. Regional EPA on-site technical assistance program coordinators were asked to send Headquarters the names and addresses of those facilities that participated in the training program. The survey was mailed to those facilities that were identified as having participated in the On-site Technical Assistance Program.

Research Design

Research Design Plan-

The research design developed for the On-site Technical Assistance Program had the objective of systematically collecting quantitative information on the Technical Assistance Providers from those facilities that actually received such technical assistance. The data collection was done in a manner to protect the privacy of all facilities completing the survey.

Completed questionnaires were returned directly to the Municipal Support Division's Project Officer.

A few weeks after the questionnaires were mailed, post cards were sent to all of those persons who had received a survey. The purpose of the strategy was to encourage the completion and return of the surveys to the above mentioned office.

Survey Pretest-

A field test of the survey was conducted in 1998. The purpose of the field test was to collect feedback and further refine the existing questionnaire from individual whose responsibilities were similar to those would potentially be participating in the On-site Technical Assistance Program. Specifically, the purpose of the field test was to ensure clarity of instructions, undercover any difficulties in following the format of the questionnaire, understand terms, provide feedback regarding the length of the survey, and request suggestions for information on items that might be added to the survey.

Several modifications were made to the survey after completion of this field test. Several types of problem areas that could potentially be experienced by wastewater facilities were added to the list on the survey.

Data Collection and Analysis

Data Collection Procedures-

Each of the respondents, identified by the Regional on-site technical assistance program coordinators, was sent a cover letter explaining the purpose of the survey, a questionnaire, and instructions on how and where to send the completed survey.

Each of the names on the list was sent a copy of the questionnaire, along with a cover letter, explaining the purpose and objective of the survey. The potential respondents were provided with a self-addressed return envelope.

As soon as the envelope arrived at the designated site, they were provided with an identification number for ease of data management. A state code was provided to help with the management of the data file.

Confidentiality Plan-

Each of the envelopes was stored in a secure location with access allowed only to the work group. The envelopes were check to make sure that no identifying information was located anywhere on the envelopes or questionnaires.

Data Coding and Editing-

Standard procedures were followed to ensure data entry quality and the tracking of the information. The data file was edited before any final data analysis. Any errors in the data entry were corrected, and missing data was accounted for in the data coding and editing process.

Data Analysis Plan-

The survey used several types of questionnaire items to collect respondent attitudinal and informational data. Informational items were used to gather data about the types of problems the facilities were experiencing and which were most important to them (see Appendices - Tables A through H). The attitudinal items pertain to the utility of the service, enhancement of staff, and whether they would recommend technical assistance providers to other facilities (see Tables 2A, 3, 4 and 11).

Results

The initial analysis of the results involved generating frequency distribution tables for each of the questions on the questionnaire. This approach allows for a discussion about the number and percentage of respondents providing responses, regarding the informational questions and attitudinal items. Each frequency table is followed by a brief discussion of the content of the table. For more information, please see pages 19 and 20 of this document which supplies the survey's summary and conclusions.

Description of Respondent Sample-

One of the customer survey questions requested a response about whether the facility was involved in technical assistance training. Of the two-hundred and seventy four (274) responses to the survey, one hundred and twenty-four (124) facilities indicated that they had received technical assistance training through the Operator Training Program. The data from these facilities was the information analyzed and used in this survey. The respondents were located in thirteen (13) States which are listed in Table 17 of this report.

Table Description-

The lists below provide information about the question asked in this survey, for specific results regarding this information please refer to the corresponding Table.

Table 1 = Specific problems addressed by the Program.

Table 2 = Requesting technical assistance again with another problem at the facility.

Table 2A = Recommending technical assistance to another facility.

Table 3 = Services received from the technical assistance provider.

Table 4 = The enhancement of the skills and capabilities of the wastewater treatment plant's staff.

Table 5 = Achieving compliance.

Table 6 = Waiting period for assistance.

Table 7 = Identifying a facility as a candidate for the Program.

Table 8 = Length of time to diagnose the problem at the facility.

Table 9 = Length of time to solve the problem at the plant.

Table 10 = Meeting the facility's needs through the Program.

Table 11 = Benefit to the facility with assistance being more accessible.

Table 12 = Which area of assistance was most important to the facility,

see Appendices - Tables I through O.

Table 13 = Facility treatment process.

Table 14 = Specific treatment process.

Table 15 = Program technical assistance versus "other" assistance.

Table 16 = Increase of cost over the past five years.

Table 17= Respondents.

Appendices Description-

Table A = Process design problems.

Table B = Process control problems.

Table C = Laboratory procedure problems.

Table D = Equipment problems.

Table E = Hydraulic I/I problems.

Table F = Preventative maintenance problems.

Table G = Management and administrative problems.

Table H = "Other" problems.

Table I = Availability of assistance in administrative support.

Table J = Availability of assistance in hydraulics.

Table K = Availability of assistance in process control.

Table L = Availability of assistance in laboratory support.

Table M = Availability of assistance in sewers.

Table N = Availability of assistance in financial management.

Table O = Availability of assistance in the "other" category.

Tables

Fifty- four and two tenths percent (54.2%) of the respondents had two or less problems that were addressed by the technical assistance provider. Thirty-four and one tenth percent (34.1%) of the respondents had between three and four problem areas to address with the technical assistance provider. As such, 88.3% of the respondents had four or less problems to address (see Table 1).

Ninety-one and eight tenths percent (91.8%) would request further assistance if they had another challenge at their facility. The mean for this distribution was 3.3 on a scale from 1 [Strongly Disagree] to 4 [Strongly Agree], (see Table 2).

When asked about whether they would recommend the technical assistance provider, 95.8% of the respondents said yes.

Ninety-five percent of the respondents agreed that the technical assistance provider was helpful (see Table 3).

Ninety-two and four tenths percent (92.4%) of the respondents agreed that the organization's staff's skills and capabilities were enhanced. The mean on a scale from 1 [Strongly Disagree] to 4 [Strongly Agree] was 3.2 (see Table 4).

In responding to the question of whether the assistance helped them return to compliance, 70.5% of the respondents stated that the assistance program helped their facility return to compliance, while 29.5% of the respondents said that the technical assistance program did not help them come into compliance (see Table 5).

Seventy-six and three tenths (76.2%) percent of the respondents waited for less than a month for technical assistance, 17.8 % waited less than a quarter and 5.9% waited more than a quarter for assistance (see Table 6).

Seventy-one and six tenths of a (71.6) percent of those responding to the survey said that they asked for assistance. Seventeen and two tenths percent (17.2%) of the respondents were referred by a State, and only two and six tenths percent (2.6%) did not know how they were identified as a candidate for the Program (see Table 7).

The majority of respondents (62.6%) reported that, once the technical assistance provider was on-site, it took a week or less to diagnose their problem (see Table 8).

Forty-nine and two tenths percent (49.2%) of the respondents stated it took one month or less to solve the facility's problems once that problem was identified. Twenty-eight and two tenths percent (28.2%) of respondents stated their problems were corrected in one week or less (see Table 9).

The vast majority of the respondents (93.3%) felt that the technical assistance met their needs (see Table 10).

Sixty-six point one percent (66.1%) of the respondents agreed that their facility would benefit from having technical assistance made more accessible. However, thirty-four percent (34%) of the respondents disagreed with the statement above, and felt as though technical assistance was readily available to their facility (see Table 11).

Process control was an area where the majority (51.9%) of the respondents indicated that was most important. A distant second was financial management at 12.3%

(see Table 12).

The majority of the respondents indicated that they used mechanical treatment processing at their facilities (77.6%) (see Table 13).

The type of treatment process ranged from activated sludge (59.0%) to trickling filters (5.7%). Other methods included lagoons at 19%, oxidated ditches at 10.5%, and other at 5.7% (see Table 14).

The vast majority of the respondents (96.3%) reported that using someone else other than the technical assistance provider that helped them would have cost them more money

(see Table 15).

The thirty five percent (35%) of the respondents reported that costs increased more than 10 percent over the last five years. Thirty percent (30%) of respondents reported that costs went up 5 to 10 percent over the last five years. Finally 35 percent (35%) of respondents reported that costs had remained the same over the last five years (see Table 16).

The 124 respondents were located in thirteen states. The largest number of respondents came from the state of New York (30). Connecticut and Rhode Island had the lowest number of respondents with one each (see Table 17).

Summary and Conclusions

1. Ninety-five percent (95%) of respondents felt assistance was helpful.



2. Ninety-five and eight tenths percent (95.8%) of the respondents would recommend providers to another facility.

1. Ninety-two and four tenths percent (92.4%) of the respondents felt their employees skills and capabilities were enhanced.



2. Ninety-one and eight tenths percent (91.8%) of the respondents would request further technical assistance.

This data identifies the effectiveness of the Program and supports its continuation.

5. Ninety-four and one tenth percent (94.1%) of the respondents had to wait a quarter or less for assistance once they had been notified.

Is this an appropriate waiting period?

6. Seventy and five tenths percent (70.5%) of the respondents were returned to compliance after the assistance.

What happened to the rest of the respondents? Should there be a follow-up procedure?

7. Eighty-eight and eight tenths percent (88.8%) of the respondents knew how they were referred to technical assistance providers.

Is there something that the Agency can do to help clarify the way to make referrals to this program?

8. Sixty-six and one tenths percent (66.1%) did agree with the statement that their facility would benefit from having technical assistance more easily accessible.

Almost one third did not agree. Efforts should be made to clarify this issue and address it.

9. Ninety-six and three tenths percent (96.3%) of the respondents stated that it would have cost them more if the service would have been provided by someone else.

Facilities saved money by using the technical assistance program.

10. Three hundred and forty surveys were mailed through out the entire U.S. for this project. Of the two hundred and seventy-four (274) total responses, only one hundred and twenty-four (124) respondents from thirteen (13) States, were eligible to participate in the On-Site Technical Assistance Customer Survey.

How many participants are there in the program? Are there only 13 states involved? How will future participants be selected for feed back?

Appendices - Tables A through H

Of those that responded to this question, 37.9% identified process design as a problem (see Table A).

Process control was identified as a concern by 73.4% of the respondents (see Table B).

Forty-four and four tenths (44.4%) percent of the respondents identified laboratory procedures as a problem for their facility (see Table C).

Process equipment was identified as a problem by 34.7% of the respondents (see Table D).

Hydraulic I/I was a problem for 21.8% of the respondents (see Table E)

Preventive maintenance was a problem for 18.5% of the respondents (see Table F).

Management and administrative factors were a problem for 16.1% of the respondents (see Table G).

Other problems were a factor for 10.5% of the respondents (see Table H).

Appendices - Tables I through O

The following Tables identify the need at small wastewater treatment plants:

Sixty-six and three tenths (66.3%) percent of the respondents stated that there was a need for an increase in the availability of administrative support (see Table I).

Seventy and nine tenths percent (70.9%) felt more assistance in the area of hydraulics should be made available (see Table J).

Seventy-five and three tenths percent (75.3%) felt more assistance in the area of process control should be made available (see Table K).

Seventy-four and two tenths percent (74.2%) of respondents felt that more assistance in area of laboratory support should be made available (see Table L).

Sixty-seven and four tenths percent (67.4%) of the respondents felt that more assistance in the area of sewers should be made available (see Table M).

Sixty-five and eight tenths percent (65.8%) of the respondents felt that more support should be available in the area of financial management (see Table N).

"Other" was a problem category where 93.3% of the respondents felt that more assistance could be provided, the "Other" was not defined (see Table O).