Analysis of the Arizona IM240 Test Program and Comparison with the TECH5 Model

EPA420-R-97-001, February 1997
Download PDF version of this fact sheet formatted for print. (60 pp, 2MB, About PDF Files)

Index

Summary and Conclusions
1.0 Introduction
2.0 Arizona I/M Program and Data
- 2.1 Arizona I/M Program
- 2.2 Arizona I/M Data
3.0 Analysis of the Arizona I/M Data
4.0 Comparsion of Arizona I/M Results with MOBILE5/TECH5 Projections
5.0 Comments from Peer Review
List of Tables & Figures

Summary and Conclusions

This document describes an analysis of Arizona IM240 data collected from the state I/M lanes during the second half of 1995 and the first half of 1996. It includes full 239 second initial test results from more than 16,000 cars and light-trucks. Retests on failing vehicles using the full 239 second test are also available. The analysis focused on vehicles which were initially tested during the 1995 calendar year to increase the number of failing vehicles which could finish the I/M process in 1996. This produced a total useable sample of about 7,650 vehicles.

This document is divided into two principal sections. The first section describes and presents the results of the analysis done on the raw data obtained from Arizona. The analysis is descriptive in nature, but includes some calculations of I/M benefits. It presents the average emission levels before and after I/M by model year as well as estimates of the emission reduction benefits. It also briefly presents average test time durations, provides a failure rate distribution by pollutant type, and touches on gas cap failure rates. The analysis also presents in a limited fashion possible seasonal variations in average emission levels, and vehicle fuel metering technology influences on average emission levels. However, this section only presents analysis of data that exist (i.e., first and subsequent I/M tests). Thus, it is not possible to directly compare the data to various "what if" scenarios of No I/M or continuation of the previous I/M program.

The second section of the document compares the emission levels and I/M benefits obtained from the data with those predicted by EPA's TECH5 emission factor model. The TECH5 model is a sub-model of the overall MOBILE5 model, and is used to calculate the basic vehicle emission rates and the I/M credits. Since TECH5 covers only cars in full detail, only comparisons for cars are shown. The results of the comparsion are shown by model year and for each pollutant.

This analysis produced some interesting findings. These are summarized below along with a few recommendations.

Based on the analysis of the Arizona IM240 data collected from July, 1995 through May, 1996, the overall VMT weighted light-duty vehicle I/M reduction for HC is: 13.7%, the CO reduction is 15.0%, and the NOx reduction is 6.9%. In terms of mass emissions of a typical car or truck in the fleet, the reductions are approximately 0.08 g/mile HC, 1.43 g/mile CO, and 0.09 g/mi Nox, on average, over the entire fleet (including both passes and failures). These are meaningful emission reductions which should improve the air quality of the Phoenix, Arizona area
However, part of the observed decrease in emissions of vehicles that were retested may be natural "regression towards the mean" that would occur even if no repairs were undertaken. This phenomenon, sometimes seen in longitudinal studies of human populations, would, of course, only be evident in high-emitting vehicles since only those are subject to retest. Documentation of this effect in emissions testing should be studied before attempting to measure its importance in the current results.

Because Arizona had a well regarded I/M program in previous years, these reductions should be viewed as an incremental improvement in the program due to the implementation of the IM240 testing rather than reductions from a No I/M baseline. Thus, they should be smaller than the first year reductions from a new IM240 program in an area which has never had I/M (i.e., certain cities in Ohio). Also, since the analysis uses IM240 data exclusively, the reductions mentioned in this report should not be viewed as hypothetical reductions to the loaded-idle type I/M program which was in force in Arizona in prior years.
The number of vehicles which appear to fail their final test or did not receive a final passing I/M test appears to be substantial - about one in three. This phenonmenon is believed to be the result of waivers and vehicles leaving the area. However, the data at hand did not explicitly identify whether each vehicle had recently been offered a waiver.
If one assumes that all the vehicles which had failing final retest scores are repaired to the average level of the vehicles which passed their final I/M test, then the overall I/M program HC reduction would be 24.0%, the CO reduction would be 20.6%, and the NOx reduction would be 9.7%. This assumption and the increased benefits may not be unrealistic, if a sizeable number of vehicles actually left the area and no longer contribute to the VMT. To possibly verify this assumption, the State should consider utilizing data from their remote sensing testing to get a better estimate of the number of vehicles which have dropped out of the testing program but still operate in the area, and the reasons why these vehicles seemingly disappear from the program prior to a final retest.

Subsequent to this analysis, the State of Arizona investigated the issue of unresolved initial failures using their entire database of approximately 2,000,000 vehicle tests, and information on vehicle waivers (not available to us). They found that approximately 15 percent of the failures could not be resolved rather than the 30 percent that we found. The difference can be partially explained by the existence of waiver information, vehicles dropping out of the full IM240 sample, and a longer time frame of tests (test results past May, 1996 are now available to the State).

To possibly resolve the remaining 15 percent of the failures, the State is planning to match the I/M records with the state license plate records from the Department of Motor Vehicles. This should help determine whether the unresolved failures are being registered outside the program area.
The I/M benefits estimated from the I/M data are quite similar in percentage terms to those predicted by the EPA TECH5 emission factor model for HC and CO. However, the estimated benefits for NOx based on the data are smaller than those predicted by the TECH5 model. For example, for cars the data indicates a 14.3% benefit while the TECH5 model predicts a 16.9% benefit. For CO, both the data and the TECH5 model predict a 16.2% reduction. However, for NOx the data indicates a 7.6% reduction while the TECH5 model predicts a 16.7% reduction. In absolute terms for cars, the observed versus predicted benefits are 0.08 g/mi versus 0.09 g/mi HC, 1.24 g/mi versus 1.08 g/mi CO, and 0.08 g/mi versus 0.20 g/mi NOx.
The average before and after I/M, HC and NOx fleet emission levels based on the data are generally lower than the corresponding average emission levels predicted by the TECH5 model. For CO, the average fleet emission levels are generally slightly higher than the TECH5 model predictions.
A considerable portion of the failures are NOx only failures or HC only failures (this is particularily true for late model trucks). The relatively large NOx failure percentage is seemingly at odds with the relatively low NOx I/M benefits. Vehicles failed for NOx are not producing sizeable NOx reductions, either because they do not actually need repair, or because proper repairs are not being done.
The failure rates of light-duty trucks in model years 1981 through 1986 are considerably lower than those of cars, although trucks are just as "dirty" on an absolute basis and relative to their new vehicle emission standards. Consideration should be given to increasing the stringency of the light-duty truck cutpoints to bring the failure rates of the two vehicle types more into parity
A slight seasonal variation in mean emission levels seems to exist. Average CO emission levels tend to increase in the summer months, and decrease during the winter months. NOx emission do just the opposite, while HC is relatively unchanged by season. The likely cause for this phenomenon is canister loading and purging which varies on a seasonal basis due to changing temperatures and fuels. However, this contention cannot be proved by the available data.

top of page

1.0 Introduction

Enhanced Inspection and Maintenance (I/M) programs were implemented beginning in January, 1995. At that time two states (Arizona and Colorado) began implementation of their comprehensive programs. Following Arizona and Colorado, Ohio and Wisconsin also began implemation of their Enhanced I/M programs. All four of these programs are still in operation as of the end of 1996.

The enhanced I/M programs differ from the traditional idle test I/M programs in several important ways. One difference is that the enhanced programs conduct the IM240 test (or the Fast-Pass / Fast-Fail (FPFF) or Fast-Pass (FP) tests). These tests are conducted on a dynamometer which applies a road-load and inertia weight to the vehicle and operates it over a transient driving cycle that includes accelerations and deceleration. This type of operation more closely simulates actual in-use driving than previous I/M tests such as the idle test or a steady-speed test. The IM240 type tests also have a higher correlation with the new-vehicle certification test (the Federal Test Procedure (FTP)) to which all vehicles are designed. Also, the emission measurements are made using more accurate analytical equipment that reports the results in units of mass (grams/mile). In contrast the idle type tests are typically performed at only one vehicle operating mode - idle. In the case of Arizona the previous I/M program also used a lightly loaded steady-speed cruise type test for 1981 and later model years. This type of test is generally recognized as a more effective test than the idle test. However, both tests generally correlate poorly with the FTP, and the measurements are made using less expensive and less precise equipment that only reports the emission readings in units of concentration (% or ppm).

Table 1.1 provides a list of the phase-in cutpoints recommended by EPA, and used by the Arizona I/M program if the vehicle completes the entire 239 second test. All of the vehicles in this analysis received the full IM240 test. The composite cutpoints are applied against the entire test emission measurement. The phase 2 cupoint is essentially a second way to pass test. For example, if a vehicle fails the composite cutpoint, but passes the phase 2 cupoint, it is an overall passing vehicle.

Now that the programs have been operating for more than one year, it is time to take a preliminary look at the effectiveness of the enhanced I/M concept in Arizona. This was done by analyzing a sample of available data from the Arizona I/M program, and comparing it with the results from the EPA model. This report presents the finding from an analysis of the Arizona I/M program data. EPA is focusing first on Arizona because of the availability of significant quantities of full IM240 tests on randomly selected vehicles. Also, Arizona has been long recognized by EPA as conducting one of the best centralized I/M programs.

top of page

2.0 Arizona I/M Program and Data

2.1 Arizona I/M Program

The Phoenix and Tucson, Arizona I/M program conducts approximately 800,000 IM240 equivalent tests per year. The program is biennial in structure with half of the fleet receiving the test in the first year, and the other half receiving it in the second year. All 1981 and later model years are tested except for the current year model year (i.e., 1996). The three pollutants HC, CO, and NOx are measured, and a partial functional evaporative system test (i.e., gas cap check) is performed. Later in May, 1996 a test of the integrity of the evaporative system (i.e., pressure test at the fillpipe) was added to the program requirements. Pass / fail standards are assessed against a vehicle based on its emission performance, and its response to the evaporative test. The pass/fail standards which are in-force are those recommended by EPA in its guidance to states. Currently, the phase-in standards (less stringent) are being used to reduce the impact of the new program on the motorists. Both cars and light-duty trucks are tested. The test which is conducted is not strictly an IM240 test, but instead is a proprietary fast-pass / fast-fail (FPFF) algorithm which may either pass or fail the vehicle prior to the end of the 239 second IM240 sequence. This algorithm is used to speed up the testing process without altering the actual pass/fail outcome, and can terminate a test in as little as 30 seconds, or run for the full 239 seconds. On average, the test runs about 60 seconds; however, most passing vehicles fast-pass through the test in 30 seconds. Failing vehicles must run at least 94 seconds, and typically run more than 150 seconds. This is to ensure that vehicles get sufficient operation to apply a pass/fail standard during phase 2 of the IM240 test which runs from 94 seconds to 239 seconds.

As part of its monitoring efforts, the State of Arizona conducts full IM240 tests on a randomly selected 2 percent of the vehicle population. It also does full IM240 tests during special studies in which the vehicles are not necessarily randomly selected. All of the vehicles in the 2 percent sample and the special studies which fail also receive full IM240 tests until they pass or are cost-waivered. These full IM240 test data allow the problem of inconsistent test duration of the FPFF testing to be overcome. In addition, the data also provides information on the emissions from the two sub-IM240 cycles called Phase 1 and Phase 2 (Phase 1 is the first 93 seconds, and Phase 2 is the remaining seconds), and on the emissions from the first 30 seconds of the IM240 test. The two different Phases may be useful in comparing and accounting for the possible effects of uncertain vehicle preconditioning prior to the I/M test. The first 30 seconds is of interest because every 1981 and later light-duty vehicle in Phoenix and Tucson I/M program is tested at least 30 seconds. Thus, it may be possible to utilize the results from this sample of full IM240 tests to develop a correlation which can be applied to the entire population of data. However, this work has not currently been completed, and is not included in this report.

2.2 Arizona I/M Data

The Arizona I/M data were purchased by EPA from the Arizona I/M testing contractor Gordon-Darby. The data were available on a set of 18 CD-ROMs. The set contained complete second-by-second test records on approximately 1.2 million FPFF or Full IM240 vehicle test records. The complete set of data obtained by EPA spanned the time range of January, 1995 through June, 1996. A subset of these data contained approximately 16,000 vehicles which were either randomly selected or part of a special study. All of these 16,000 vehicles received the full IM240 test. Data from the months of January, 1995 through June, 1995 are not available because the random selection feature of the program had not yet been implemented, nor were any special studies of this type conducted.

Only the vehicles which had full IM240 tests, and were randomly recruited were used in the analysis of Arizona's I/M program. This reduced the number of vehicles from approximately 16,000 down to 14,422. The sample used in the analysis was further restricted by considering only those vehicles which received their initial I/M test in 1995 (retest data collected in 1996 was used). This reduced the number of vehicles in the analysis to 7,647 vehicles. The data restriction ensured more complete matching of the initial test with the final retest on those vehicles which failed the initial test by requiring that every failing vehicle have at least six months to obtain repairs and pass through the I/M process. This may seem like an overly conservative restriction; however, Table 2.1 and Table 2.2 show that in some cases it may take a motorist several months and several test iterations to get through the I/M process. For example, the average vehicle which failed the initial test and passed the retest took 19.4 days and 2.7 iterations. The average initial test failure and subsequent waiver/no passing test took 38.7 days and 3.5 iterations. In some rare cases the duration between the initial test and the final available retest was more than 300 days and/or 10 iterations. However, in many of these extreme cases the additional tests may be the result of change of ownership, and should actually be considered new initial tests.

Data on three vehicle classes (cars (LDV), light-duty trucks 1 (LDT1), and light-duty trucks 2 (LDT2)) were available. The LDT1 vehicles are typically the smaller pickup trucks and vans (i.e., less than 6,000 lbs GVW), and the LDT2 vehicles are the larger delivery trucks and vans (i.e., 6,000 to 8,500 lbs GVW). The breakdown in the 7,647 sample is as follows: LDV - 5,031 or 66%; LDT1 - 2,102 or 27%; LDT2 - 514 or 7%. The overall failure rate for the sample of 7,647 vehicles is 12.7 percent or 973 vehicles. Table 2.3, 2.4, 2.5, 2.6, and Table 2.7 provide a complete breakdown of the sample size by test year, vehicle type, and initial test pass / fail status, (explained below) failure rate, and model year distribution.

The initial test results on the 973 vehicles which failed their initial I/M test were matched (or attempted to be matched) by VIN (vehicle identification number) with their final retest record. For this analysis all intermediate test results were ignored. This matching process created three categories plus the first category that includes the vehicles which initially passed. Table 2.6 gives a breakdown of these categories by vehicle type.

Pass Initial Test - No Retest
Fail Initial Test - Pass Retest
Fail Initial Test - Fail Retest
Fail Initial Test - No Retest Available

These four categories represent the typical I/M outcome for most vehicles. The first category includes the majority of the vehicles whose emissions are low and pass the test on the first try. The second category are the I/M successes. These are vehicles which initially fail, but are sufficiently repaired to pass a retest. The majority of the I/M benefits result from these vehicles. The third and fourth categories are those vehicles which did not appear to have a final emission result which passed the I/M standards. In an ideal I/M program in which no waivers were allowed, and every motorist could be tracked down and made to participate, these outcomes would not exist. However, in a real program these factors do exist, and have a tendency to dilute the benefits of I/M.

This analysis cannot offer a complete explanation for why many vehicles in the sample (32 percent of the failures) do not complete the I/M process with emission scores that are below the standards. However, for the case of the #3 category, the data does show that 65 percent (see 2.1 and the individual bar "Waiv-Fail") of these vehicles received their final IM240 test at the state waiver lane, and presumably received a compliance waiver. The remaining 35 percent of these vehicles received their final test at a non-waiver station. These two statistics are shown on the right half of the figure under "Fail Last Known Retest". Because the test database does not contain a flag that provides waiver status, it is not possible to definitively conclude that any of these vehicles received waivers. The left half of Figure 2.1 provides similar statistics on vehicles which passed their retest. For comparison, in this group only about 10 percent of the retests were done at the waiver lane, and 65 percent were performed at the same station as the initial test.

The vehicles in category #4 have a final test status which is completely unknown. Several possibilities exist for why the final test is missing. One possibility is that the retest result is outside of the 2% sample. In other words, the vehicle did receive a final state I/M test, but it was not a full IM240 test. The existence of this outcome was checked by searching the entire program database for the existence of these vehicles. In very few cases were they found with a passing FPFF test to indicate that they had somehow dropped out of the 2% sample. Another possibility is that upon failure of the I/M test, the vehicle owner decided to sell or register the vehicle outside of the program boundaries. If this were the case, the I/M record would not exist. However, the overall State registration records should reflect this outcome. Another possibility, although somewhat remote, is that the vehicle is still in the I/M process, and has not yet received its first retest or its retest was recoded as an initial test. This outcome would require at least a six month time frame between the first I/M test and the retest.

top of page

3.0 Analysis of the Arizona I/M Data

3.1 Calculation of I/M Reductions

The percent reduction (Benefit) in emissions due to I/M was determined for each model year and pollutant. It was done by summing the before and after I/M HC, CO, and NOx emissions from each vehicle and then taking the difference, and the percent difference. The calculations were done separately for each model year, and vehicle type (LDV, LDT1, LDT2).

Mathematically, the equation is:

Benefit = [SUM(ip) + SUM(if)] - [SUM(ip) + SUM(rp) + SUM(rf) + SUM(irm)] (eq1)

The benefit can be expressed as a percentage by:

Benefit% = Benefit / [SUM(ip) + SUM(if)] (eq2)

where:

SUM(ip) - Sum of the initial test emission from vehicles which pass the initial test.
SUM(if) - Sum of the initial test emission from vehicles which fail the initial test.
SUM(rp) - Sum of the retest emissions from vehicles which pass the retest.
SUM(rf) - Sum of the retest emissions from vehicles which fail the retest.
SUM(irm) - Sum of the initial test emissions from vehicles with no retest

When these equations are applied to the database of 7,647 vehicles, the results shown in Figures 3.1, 3.2, 3.3, 3.4, 3.5 and 3.6 are obtained for each model year and vehicle type. In each of these figures, the individual model year benefits are shown as percentage of the initial test levels. For example in Figure 3.1, the HC I/M reduction is approximately 25 percent for the 1981 model year. The composite reduction for all of the model years of a given vehicle type and pollutant are also shown as an insert in Figures 3.1 through 3.6. These results are weighted by both Arizona model year vehicle miles travelled (VMT) and registration distributions. The VMTs which are used for each of the 1981 and later model years are average values from the MOBILE5 model. Only the 1981 and later model years are included, thus ignoring the pre-1981 vehicles in this analysis. The effect of the model year weighting factors is to give the newer model years more contribution to the composite reduction for each vehicle type. This is necessary because newer vehicles on average accumulate more mileage in a given period than older ones.

The overall I/M reduction for each pollutant is calculated by weighting the composite reductions for each of the three vehicle types together. Other vehicle types such as motorcycles and heavy-duty gas trucks are excluded from this calculation since they are not part of the IM240 program. Thus, this analysis does not consider the impacts of the I/M program on the overall mobile source inventory in Arizona, but only on the three vehicle classes which are covered by I/M, and where IM240 data are available. However, these three classes (LDV, LDT1, and LDT2) make up the majority of the VMT and emissions contribution. For this analysis, the LDT1 and LDT2 categories were combined into one group called LDT. The overall weighed reduction from the I/M program is shown in Table 3.1 for each pollutant. The overall VMT weighted program HC reduction is 13.7%, the CO reduction is 15.0%, and the NOx reduction is 6.9%.

3.2 Average Vehicle Emissions

The average vehicle emissions before and after I/M are shown in Figures 3.7, 3.8 and 3.9 for HC, CO, and NOx as function of model year for the LDVs. These figures compare the average test emissions from vehicles which initially pass the I/M test with retest emissions from vehicles which initial fail but pass the retest (category 1 vs category 2) and those which do not pass the final retest (category 3). The emissions shown as category #1 in the figures are initial test results. Those labeled as category #2 or #3 are retest emissions.

The chart shows that the repair technicians in Arizona are on average not returning the failing LDV's upon repair to the level of the initial passing vehicles. This is mostly because of the considerable amount of likely waiver vehicles which continue to fail (category #3). However, even the category #2 vehicles (pass the retest) are not being returned on average to the levels of the passing vehicles. However, the difference is fairly small between the emission levels of vehicles which pass the initial test, and the emission levels of vehicles which pass their retest. An analogous analysis of the LDT's was also performed, but the results are not shown in graphical form. The truck curves exhibited the same general relationship as a function of test status as the car curves (i.e., the failing trucks are not returned to the average levels of the initial passes). However, the average emission levels from all of the trucks curves were higher than the car curves.

Figures 3.10, 3.11 and 3.12 compare the average emissions before and after I/M for the vehicles which pass the retest with those which fail the retest. Two observations can be made regarding these figures: (1) The retest emissions from the vehicles which continue to fail are lower than their initial emission levels. This suggests that waiver vehicles do receive some useful repairs, and do account for some small part of the I/M benefits; although, the possibility exists that some of this reduction is the result of the phenonmenon known as regression to the mean. (2) The initial emission levels of vehicles which fail their retest and are likely waivered (category 3), are generally higher than the initial emission levels of vehicles which pass their retest (category 2). This suggests that the vehicles which are waivered or missing their retest are higher emitting and likely to be more difficult to repair. It also tends to invalidate the assumption that the lost emission reduction from a waivered failure is equivalent to the achieved emission reduction from a non-waivered failure. The results from a similar analysis of the truck data (not shown) produced results which were analogous to the cars.

Table 3.4 shows the average emission levels for all of the vehicles before and after I/M. These average emission levels are weighted by the four levels of test status and by model year. The test status weighting are based on the actual percentage of each of the four test status conditions in the sample. The model year weighting factors are the MOBILE5 travel fractions. The VMT fractions between cars and trucks shown in the table are used to calculate the overall average before and after emission levels in units of grams per mile. The results show the average reduction in HC emissions is about 0.08 grams/mile, the CO reduction is about 1.43 grams/mile, and the Nox reduction is about 0.09 grams/mile. Also, the results show a sizeable difference between the average emissions of cars and trucks both before and after I/M. For comparison, the analogous average emission levels based on the TECH5 model are shown for cars at the bottom of Table 3.4. Except for Nox emissions, the average emission levels and reductions are reasonably close or higher than those predicted by the TECH5 model. For example, the average TECH5 HC reduction is 0.09 g/mi, and the average CO reduction is 1.08 g/mi.

3.3 Alternative Calculation of I/M Reductions

Equations 1 and 2 produce a somewhat conservative estimate of the I/M benefits because they assume that all of the category #3 vehicles do not receive any further additional repair, and that category #4 vehicles do not receive any repair. This may be an unjustified assumption if the final retests were actually performed, but are not present in the database. This could potentially be the case since subsequent to the analysis, a few additional passing retests (six vehicles were identified) were located in the overall Arizona database which contains all of the vehicles which were tested instead of a sample which received only the full IM240 test. These retests were not part of the analysis dataset which received the full IM240 because they were coded as initial tests. The benefits of the program could also be understated slightly if some of the category #4 vehicles were sold outside of the program area, and thus no longer contribute to the VMT and emission inventory of the program area.

To account for the possible scenario that many of the category #3 and #4's were actually repaired or sold outside of the program, Equation 1 was modified by assuming that the category #4 vehicles and the category #3 vehicles which did not receive their final IM240 test at the waiver lane were repaired to the average level of the category #2 vehicles. The same assumptions for model year distributions and VMT that were used in the previous scenario were used again. If these assumptions are made, the the overall I/M benefits are 24.0 percent for HC, 20.6 percent for CO, and 9.7 percent for NOx. The overall results for this scenario and the results for the individual vehicle types are shown in Table 3.2.

The likely maximum I/M benefit given these cutpoints, and vehicle fleet can also be calculated by assuming that all of the initial failing vehicles are repaired to the average level of the vehicles which passed their initial test. This is a reasonable assumption of maximum benefits at the current pass/fail standards since it is unlikely that repair technicians will be able to reduce vehicles emissions on average below those of initial passing vehicles. If this assumption is made, the maximum benefits for the Arizona I/M program for HC, CO, and NOx are 28.1 percent, 25.4 percent, and 11.6 percent, respectively. Table 3.3 summarizes the overall benefits for this scenario for each of the three vehicle types.

3.4 IM240 Failure Rates

Figure 3.13 shows the failure rate by model year and pollutant for the LDV's. Figure 3.14 shows the failure rates for the combined sample of LDT1 and LDT2 vehicles (LDT). The figures are stacked bar types with the failure rates for each individual pollutant stacked on top of each other. The sum of the individual pollutants is the overall failure rate for a given model year. Several points are evident from Figures 3.13 and 3.14. The first is a dramatic decrease in LDV failure rate after the 1988 model year, and LDT failure rate after 1989. For example, for LDVs, between the 1988 and 1989 model years, the failure rates fall from about 8 percent to about 4 percent, and stay fairly low from 1989 onward. This is contrasted with an ever rising failure rate from 1988 back to 1981. The lower failure rate on the newer cars is even more apparent when the lower cutpoints (more stringent) for the 1989 and later vehicles are factored into the situation. The reason for the lower failure rates on the 1989 and later cars versus the newer cars may be the widespread introduction of ported fuel injection starting in the 1988 to 1989 model year. For LDT vehicles heavy penetration of ported fuel injection may have occurred a little later. In other limited, but detailed studies, ported fuel injection has generally been shown to have lower emissions and more durability than previous technology. Another possibility is that it simply takes about 7 years of operation before vehicles deteriorate sufficiently to begin failing at higher rates, and that after that point is reached the failure rates climb steadily each year. Finally, it may be that under the previous Arizona I/M program, vehicles five years and younger were better repaired when needed than were older vehicles.

Another point which is evident from the two figures is the considerable difference in failure rates between light-duty trucks and cars on all of the older model years (1986 and older model years). For example, the failure rate on the 1981 through 1983 model year cars is around 50 percent, whereas for trucks it is less than 20 percent. In addition, the 1981 through 1982 truck failure rates are lower than the rates for several succeeding model years, whereas the average emission levels for these two model years are generally higher . The average emission levels of cars and trucks are shown in Figures 3.23 through 3.27 for all three pollutants. These corresponding lower failure rates translate into smaller I/M benefits, and occur in model years which generally should produce fairly substantial I/M benefits. This sizeable difference in failure rates and benefits is likely the result of the differences in cutpoints between the two vehicle typer. For example, the phase-in HC cutpoint for 1981 and 1982 model year cars is 2.00 g/mi, whereas the corresponding truck cutpoint is 7.50 g/mi. The other pollutant cutpoints have similar differences.

Figures 3.13 and 3.14 also show the breakdown of the sample by pollutant failure mode. The largest pollutant failure categories are the NOx only failures and the HC only failures. Combined NOx and HC failures are also somewhat common. Also, for late model trucks, the NOx only failures make up a very large percentage of the overall failures. The high NOx failure rates (NOx only and NOx and HC) are surprising given that the likely overall NOx reduction benefit is only in the 5 to 10 percent range.

3.5 Gas Cap Failure Rates

Figures 3.15 and 3.16 show the results of Arizona's I/M gas cap test in terms of failure rates, and the percentage of failures which are apparently not retested or repaired. Because FPFF was not a factor in the gas cap test, the entire sample of over 400,000 vehicle tests performed from June, 1995 through December, 1995 were analyzed. The results in Figure 3.15 show the failure rates by model year to be fairly small. They typically range from about 0.5 percent for new cars and trucks up to about 6 percent for 1981 trucks. These results seem reasonable given the fairly simple and durable nature of a gas cap, and the lack of real incentives to tamper by the motorist. The results also show that the truck failure rates are consistently higher than the car failure rates.

Despite the low failure rates, substantial HC emission benefits from repair are possible due to the relatively large emission contribution that a vehicle with a bad or missing gas cap can make. Unfortunately, the current tests are functional type tests and do not produce emission level type information. Thus, the size of the HC emission reductions cannot be determined from the data in a manner analogous to the exhaust emission tests.

Figure 3.16 shows the percent of gas cap failures which are not retested or repaired. The percentages are quite high considering that the repair is quite simple, obvious, and inexpensive. The likely reason for the high rates is that many of these vehicles also failed the exhaust test, and the owner is not retesting the vehicle because of exhaust emission component problems and perhaps sold the vehicle outside the program, put it up on blocks, scrapped it, etc. Some evidence for this can be seen in the 1981 through 1984 truck non-repaired rates in Figure 3.16. These seem to follow the same shape as the exhaust failure rates rather than the gas cap only failure rates as one would expect.

3.6 Seasonal Differences in Emission Levels

Figures 3.17, 3.18 and 3.19 show the mean monthly HC, CO, and NOx initial test emission levels for cars. Each figure has four curves. The one labelled HC240 (or CO240 or NOx240) represents the results from the full 239 second test. The figure labelled HC30 are the results from the first 30 seconds of the IM240 test. The Phase 1 and Phase 2 curves represent the first 93 seconds, and the remaining 146 seconds, respectively. The individual Phase 1 and 30 second results are shown because they are generally more or less suseptible to cold start or other preconditioning problems than the full IM240 test. Thus, they may exhibit more or less variation due to seasonal ambient temperature changes than the full IM240 test.

Examination of the three figures shows that the average HC emissions show the least amount of variation by month or season of the year. In fact, for HC emissions there appears to be no definite pattern of increasing or decreasing emission levels versus month of test. On the other hand, the CO emissions show a more definite trend of higher CO emissions during the summer and lower emissions during the winter. The brief 30 second test results are particularily dramatic. The opposite results are obtained from the NOx curves which shows slight increases during the winter months and decreases during the summer months.

The exact reasons for the seasonal emission patterns cannot be conclusively determined from these data. However, the most likely explanation includes higher canister vapor loading and purge during the hot summer months which can lead to: richer air/fuel ratios, CO emission spikes, and potentially lower NOx emissions. The seasonal emission trends may exist despite a seemingly opposite seasonal pattern in gasoline volatility. While winter RVP is higher than summer RVP, the temperature effect may be stronger than the RVP effect. The level of oxygen in the fuel or the RVP (volatility) of the fuel maybe changing on a monthly or seasonal basis, and should have an impact on emission levels. Another contributing factor may be that oxygenated fuel use in the winter lowers fleetwide CO emissions. The limitations of the NOx correction factor may also have a seasonal effect on the NOx emissions. The correction factor is only applicable to temperatures at or under 86F. However, many summer months in Arizona have temperatures which exceed 86F, and in these cases the correction for 86F is used. Thus, this may have an effect on the reported NOx emissions and failure rates.

One explanation which does not seem consistent with the data is that the vehicles are poorly preconditioned for their emission test in the wintertime because of colder ambient temperatures and more vehicles in cold start mode immediately prior to their emission test. For example, at some busy stations during the end or beginning of the month testing rush, some vehicles may idle longer in line prior to their test. If the ambient temperatures are lower, then catalysts and engines may also be at slightly lower temperatures, and produce higher emission levels at the beginning of the test. If this were the case, and if its emission effect were substantial, the CO emission levels (particularily the 30 second test) should rise during the winter and fall during the summer. The opposite actually occurs.

3.7 Comparison of Emission Levels of Carbureted Versus Fuel Injected Vehicles

A propertiary VIN decoder purchased from Radian Corporation was used to decode the individual vehicle VINs and segregate them into either fuel injected or carbureted classifications. The vehicles were also segregated into car and truck groups. This process permitted an investigation of whether fuel metering type has a significant effect on average emission levels.

Figures 3.20 and 3.21 show the distribution of fuel metering type for cars and trucks. The results show the clear and overwhelming penetration of fuel injection technology into the fleet between 1981 and 1990. For cars, fuel injection technology had completely penetrated by 1990 and for trucks it was 1991. The crossover points occurred during the 1984 through 1985 time frame with cars penetrating slightly earlier, but trucks penetrating slighly faster once they had begun. Unfortunately, the Radian VIN decoder does not reliably decode the vehicles into finer fuel injection category such as ported fuel injection and throttle body fuel injection. However, it is believed that the vast majority of the late model vehicles are ported fuel injection types while many of the early 1980's fuel injection cars were throttle body technology. It should also be noted that the first fuel injected models to be introduced and the last carbureted model to be produced may have been peculiar cases. Thus, the comparison between fuel injected and carbureted is best for the model years with a reasonable percentage of each. Sample sizes are also better for these middle model years.

Figures 3.22 and 3.23 show surprizingly similar HC emission results between fuel injection and carbureted vehicle technology during the years in which overlap of the two technologies existed. For cars the agreement between fuel injection and carbureted average emission levels is nearly identical. The difference is larger for trucks; however, even this difference is typically only about 0.5 g/mi HC with carbureted trucks being higher. The results are somewhat surprising because it has long been assumed that fuel injection technology was clearly superior to carbureted technology in terms of HC emission control.

Figures 3.24 and 3.25 show the comparison between fuel injected vehicles and carbureted vehicles in terms of CO emissions. The results for both cars and trucks show sizeable differences between the two types with the fuel injected vehicles having lower emissions. The differences are particularily striking for trucks except for the 1990 model year which had a comparatively small sample of trucks. The sizeable differences in average CO emissions between fuel injected and carbureted technologies are expected, and are likely due to the more precise fuel control offered by the ported fuel injected vehicles versus the carbureted .

The NOx emission results shown in Figure 3.26 and 3.27 are also generally similar for both fuel injected and carbureted cars and trucks for the model years with good numbers of both types.

top of page

4.0 Comparsion of Arizona I/M Results with MOBILE5/TECH5 Projections

The previous section presented the calculated emission reductions from the Arizona I/M program. This section compares the results from the Arizona IM240 data with those projected by the EPA emission models. This comparison is not intended be a formal, and rigorous evaluation of the Arizona I/M program as part of the SIP process. Instead, it should be viewed more as a qualitative comparison of the program versus the EPA models. It is as much an investigation of the realism of the EPA models as of the Arizona I/M program.

The most appropriate EPA emission models in which to compare the data against are the MOBILE5 and TECH5 series of models (i.e., MOBILE5a, MOBILE5b). MOBILE5 is a computer model which projects vehicle emission rates for a given broad location based on a set of input parameters. These inputs include registration and VMT distributions, types of fuel, average temperatures, average speeds, types of control programs, and many others. The output is an overall emission factor (projection) for a given area. The TECH5 model is a sub program of the MOBILE5 model. It is used to generate the individual model year emission rates and the I/M credits which are used in the MOBILE5 model. The TECH5 model is a flexible program that can more precisely model emission rates and I/M reductions than the overall MOBILE5 model. Its flexibility includes the ability to project individual model year emission factors for a range of different vehicle ages. This flexibility permited a more precise comparison to be made between the data collected over a fairly narrow time frame, and an EPA emission model. This is a flexibility which the MOBILE5 model does not currently allow. However, if real data is found to match TECH5 predictions, the match also will apply to MOBILE5.

Arizona's I/M program was modeled using the TECH5 model and the concept shown in Figure 4.1. The vertical axis/direction in Figure 4.1 is the average emission level in grams per mile, and the horizontial axis is time. This figure is a conceptual schematic depicting the change in an individual model year's average emission level as the result of an I/M cycle. The points which are shown as an illustrative example are for the HC pollutant, the 1985 model year, and two vehicle age inputs (10 and 11 years old in 1995). The 1985 model year was chosen because it shows a larger I/M benefit than some of the more recent model years.

Separate TECH5 model runs for two ages (i.e., 10 and 11 years) were done because the typical new car model year begins in October, and cars are sold throughout the year. Thus, some vehicles in the sample may be almost one year older than others. For simplicity it was assumed that all of the cars in Arizona sample were originally sold in the month that they were tested, and that the ones tested in June through September had age (n) and those tested between October and December were age (n+1). The average result of the two ages is a weighted average based on number of vehicles in the two groups in the Arizona database.

Point A in Figure 4.1 represents a particular model year (in this case the 1985 model year) as it would have existed on January 1, 1995, if no I/M program existed. The higher emission levels from point A to point C reflect the deterioration of the particular model year over the one year period. Point B in the figure represents the model year as it exists as the result of its I/M program that was in effect at that time. The particular I/M program which was in operation was a loaded / idle test on 1981 and later vehicles. Point C in the figure represents the particular model years hypothetical emission level one year later as it would have existed if no I/M program existed. Point D is the predicted before IM240 emission level for the particular model year. It cannot be obtained directly from the TECH5 model; however, it was calculated by projecting a line from Point B, and assuming that the slope is the same as the slope of the line from Point A to Point C (this consistency in deterioration rates between I/M and Non I/M is a fundamental assumption of the TECH5 model in calculating I/M program benefits). Point E was computed by performing a TECH5 run using IM240 assumptions. It is the final IM240 emission level for a particular model year after inspection and repairs.

Creating this conceptual schematic is necessary because the actual Arizona IM240 program results are available only at Point D and Point E, and the comparison can only be made between these points. Clearly no actual data will ever be available at Points A and C (No I/M cases) since the Arizona I/M program has run continuously since 1977. Point D represents the initial test results of the vehicles as they come into the IM240 process. Point E represents the results afterwards (after failing vehicles are repaired to pass the IM240 test). The actual I/M reductions obtained from the data for a given model year should therefore be similar to the difference between Point D and Point E in the TECH5 output.

For the comparisons shown in Figure 4.2, 4.3 and 4.4, the output from the TECH5 model was adjusted to in-use conditions present at the I/M lanes by applying a multiplicative factor. The adjustments included a speed factor to correct the FTP based TECH5 model speed from 19.6 MPH to 30 mph (the average speed of the IM240 cycle). The FTP ambient temperature range of 60 to 86 F with an average of 75 F was also corrected to an average of 90 F to account for the higher ambient temperatures in Phoenix, Arizona. Also, the vehicle start operating modes of the FTP were also adjusted to account for the non-start mode conditions under which the IM240 test in Arizona is conducted. Finally, the FTP indolene fuel based emissions were adjusted to account for the natural in-use fuel properties. The adjustments factors were created from the ratios of successive MOBILE5 outputs which included and excluded the above parameters .

Figures 4.2 through 4.4 show the comparison of Arizona emission levels with TECH5 emission levels at Points D and E for cars only. For example, in Figure 4.2 the curve labeled Tech5 Base represents Point D, Tech5 I/M represents Point E, AZ Base represents the Arizona data results at Point D, and AZ I/M represents the Arizona data results at Point E. The difference between a Base curve and its corresponding I/M curve is the I/M benefit of the first cycle of the IM240 program.

The HC and CO results in Figure 4.2 and 4.3 suggest fairly good agreement between the adjusted TECH5 model output and the Arizona data results. For HC, the benefits are similar in magnitude, and gradually increase with earlier model years. For example, they range from virtually zero in the late model years to about 20 - 25 percent for the early 1980's vehicles. Some differences are apparent in a few model years during the mid-1980s with the TECH5 model predicting larger I/M benefits than the data suggests. The average before and after I/M HC emission levels based on the data and predicted by the TECH5 model are also generally similar. Most of the differences occur during the mid to late 1980s model years with the TECH5 model generally over- predicting HC emission levels. The overall I/M reductions weighted by TECH5 travel fractions (same weighting were used on the predictions based on the data and TECH5 projections) based on the data and the TECH5 model are also quite similar. For example, for cars, the data indicates an overall 14.3% benefit while the TECH5 model predicts a 16.9% overall benefit. In absolute terms, the reductions are 0.08 g/mi HC from the data, and 0.09 g/mi HC from the TECH5 model

The agreement between the TECH5 model and the Arizona data is even better for CO than for HC. In this case, both the TECH5 model and the data analysis estimate the I/M benefits at 16.2 % for cars. The difference in emission levels and benefits is very similar for all model years except the ones in the early 1980's. For these model years, the CO emission levels based on the Arizona data are actually higher than the model predictions. Also, for some model years, the I/M CO benefits calculated from the data are also slightly larger than the benefits predicted by the TECH5 model. In absolute terms, the differences between the data and the TECH5 are more substantial with the reduction based on the data at 1.24 g/mi CO, and the TECH5 reduction at 1.08 g/mi CO.

More sizeable differences are apparent for NOx emissions. These differences are most apparent in the older pre-1990 cars. In this case, the emission levels from the Arizona database are considerably lower than the predicted results from the model. It is only with the late model (1990 and later vehicles does the agreement seem good. The overall I/M NOx benefits for cars are also smaller than those predicted by the TECH5 model. For example, the data analysis indicates a 7.6% reduction while the TECH5 model predicts a 16.7% reduction. Much of this difference occurs in the early and mid 1980s model years. For the 1990 and later model years, the NOx benefits from both methods (TECH5 and the Arizona data) seem to be very small. In absolute terms, the reductions are also substantial with the data producing a 0.08 g/mi Nox reduction, and the TECH5 model predicting a 0.20 g/mi Nox reduction.

top of page

5.0 Comments from Peer Review

Prior to the final release of this report, a substantially final Draft version was released for review to various individuals and organizations. This section summarizes substantial written comments by the reviewers. To the extent possible these comments and suggestions were incorporated into the final version of this report.

Written comments were provided by only one reviewer - Dr. Robert Slott. They are provided below:

Per your request, here are some possible reasons why the use of I/M data to evaluate an I/M program may lead to an overestimation. (re: The 1/14/97 Draft Arizona I/M Emission Reduction Benefit Estimate)

The sample is only from the population of gasoline vehicles taking the test. Vehicles that are driving in the non-attainment area but are not part of the I/M program will not be included in the sample. Non-program vehicles could be transients (e.g., tourists) or vehicles that regularly drive in the area (e.g., vehicles registered out of the control area).
The sample may not be representative for the fleet being estimated. A sample of 7,647 vehicles may not be sufficiently large to be a random sample of highly skewed distributions. The random nature of the sample should be checked against a variety of fleet characteristics that could influence I/M performance (pass/fail, model year, vehicle type, engine and emission system technology type, zip code (a surrogate of vehicle owner wealth)). Initial emissions from the 7,647 sample could be checked against the initial emissions of the (14,422 - 7,647) randomly selected vehicles tested too late to see the response for failed vehicles. A non-representative sample could lead either to over- or underestimation of the emissions from the fleet required to take the I/M test.
High emitting vehicles show test-to-test variability. If vehicles that fail have test scores that are not test-to-test reproducible, then re-testing a failed vehicle without repair (or with ineffective repair) leads to an overestimation of the benefit of the I/M program. Test-to-test variability for a dynamometer test may be due to operator variability.
The I/M program benefit will be overestimated when a winter fuel is used during one part of the year in order to reduce CO (e.g., in Colorado and Arizona). Vehicles that fail on the summer fuel, and subsequently pass on the winter fuel, will have an enhanced benefit. They may pass without an effective repair. The amount of overestimation should be subject to estimation if the emissions' levels and times of failure and passing, and an estimated grams/mile emissions benefit of the winter fuel are available.
Any analysis that does not compare an "in-program" fleet with an equivalent "out-of-program" fleet, credits ongoing maintenance for the "in-program" fleet as arising from the "program." This will overestimate the benefit of the program.
The benefits of the program are apparently based on a two year time between inspection and maintenance. If the average time to repair is significant (longer than say two weeks), then the benefit of the program should be reduced accordingly.
Most owners primary wish is to "pass the test" rather than to "maintain their vehicle" or "clean the air." Owners may attempt to influence the test result in a way that is not characteristic of vehicle performance during normal driving (e.g., special gasoline additives just prior to taking the test). Owner behavior to attempt to pass the test would result in a greater number of vehicles passing compared than deserve to pass.
The Arizona analysis properly monitored each failed vehicle. Not all analyses do. The number and emissions for each category should be recorded: (1) repaired and later passed the test (with the length of time between first failure and ultimate passing), (2) repaired but still not passing the test and then waived, (3) known to be scrapped, (4) known to be sold or registered outside the control area, (5) disappeared off the radar screen. Extrapolating data only from vehicles that were known to have been failed and repaired to pass, will overestimate the benefit of a program. This analysis calculated that number separately as an upper limit to the benefits of the program.