Atmospheric Modeling and Analysis Research
Probabilistic Model Evaluation
When weighing the societal benefits of different air quality management strategies, policy-makers need quantitative information about the relative risks and likelihood for success of different options to guide their decisions. A key component in such a decision support system is an air quality model that can estimate not only a single "best-estimate" but also a credible range of values to reflect uncertainty in the model predictions. Probabilistic evaluation of the Community Multiscale Air Quality (CMAQ) models seeks to answer these questions:
- How do we quantify our uncertainty in model inputs and parameterizations?
- How do we propagate this uncertainty to the predicted model outputs?
- How do we communicate our level of confidence in the model-predicted values in a way that is valuable and useful to decision-makers?
To address these questions, EPA has deployed a combination of deterministic air quality models and statistical methods to derive probabilistic estimates of air quality. For example, an ensemble of deterministic simulations is often used to account for different sources of uncertainty in the modeling system (e.g. emissions or meteorological inputs, boundary conditions, parameterization of chemical or physical processes).
A challenge with ensemble approaches is that chemical transport models require significant input data and computational resources to complete a single simulation. EPA has applied the Decoupled Direct Method (CMAQ-DDM-3D) to create a reduced form version of the CMAQ model which can then be used to generate large member ensembles while avoiding major computational costs of running the regional air quality model multiple times.
EPA has also used hierarchical Bayesian statistical methods to calibrate the reduced form model based on observed pollutant levels. The statistical model is used to produce observation-based estimates for bias and uncertainty in model input parameters to the final model output metric of interest. This helsp EPA improve confidence in making air quality management decisions.
These approaches provide a posterior probability distribution of pollutant concentration at any given location and time. The full probability distribution can be used in several ways, such as estimating a range of likely, or "highly probable", concentration values, or estimating the probability of exceeding a given threshold value of a particular pollutant.
For example, the figure below shows the estimated probability of exceeding an ozone threshold concentration of 75 parts per billion over the Southeastern U.S., for current conditions (top) and with a 45 percebt reduction in nitrogen oxide (NOx) emissions from mobile sources (bottom).

Spatial plots of fourth highest summer 2005 maximum daily 8 hour (MD8) ozone concentration in parts per billion under current conditions (top left) and after a 45 percent reduction in mobile NOx emissions (top right). Bottom row shows the posterior probability of the fourth highest MD8 ozone value exceeding a standard of 75 parts per billion before (bottom left) and after (bottom right) the reduction in mobile NOx. Ozone monitoring locations are shown in black.
Contacts: Kristen Foley, Sergey Napelenok, Rob Pinder
Related Publications:
- K.M. Foley, B.J. Reich, S.L. Napelenok. Bayesian analysis of a reduced-form air quality model, , Environmental Science and Technology, in review.
- B.J. Reich, D. Cooley, K.M. Foley, S.L. Napelenok, B.A. Shaby. Extreme value analysis for evaluating ozone control strategies, Journal of the American Statistical Association, in review.
- R.W. Pinder, R.C. Gilliam, K.W. Appel, S.L. Napelenok, K.M. Foley, A.B. Gilliland. Efficient Probabilistic Estimates of Surface Ozone Concentration Using an Ensemble of Model Configurations and Direct Sensitivity Calculations, Environmental Science and Technology, 43(7):2388-2393, (2009).
- K. M. Foley, R.W. Pinder, S.L. Napelenok, and H.C. Frey. Probabilistic Estimates of Ozone Concentrations from an Ensemble of CMAQ Simulations
, poster for Models-3 Users' Conference, Chapel Hill, October 2008.
