Multi-criteria Integrated Resource Assessment (MIRA)
You will need the free Adobe Reader to view some of the files on this page. See EPA's PDF page to learn more.
On this page
- What is MIRA?
- Tools in the MIRA Toolbox
- Six Steps of the MIRA Process
- Learn More
MIRA is an approach that facilitates stakeholder engagement for collaborative multi-objective decision making. MIRA is designed to facilitate and support an inclusive, explicit, transparent, iterative learning-based decision process.
The process is used to:
- Create an Environmental Index (using many indicators)
- Rank environmental elements such as counties, watersheds or alternative decision options (using many indicators)
What kinds of analytical problems can MIRA handle?
- Policy/decision problems with many and diverse stakeholder interests
- Mix of quantitative and qualitative data/information, including chemical and non-chemical stressors and human health and ecosystem/ecosystem services
- Data/model uncertainties as well as decision uncertainty
- Policy/decision problems that utilize highly technical, expert information as well as stakeholder values
- Rank ordering and comparing things
MIRA is designed on these decision analytic principles:
- Consideration of Stakeholders
- Criteria and their metrics
- Significance of the data metrics
- Value judgments
Tame vs. Wicked Problems
- Have a right answer
- Have objective measures of success
- Are fact-driven
- Have a knowable truth (even if we don’t know it now or if there is uncertainty about it).
Wicked problems are:
- Multi-dimensional, i.e., Hard to define because of diverse stakeholder perspectives,
- Have no single right answer; only better or worse conditions
- Have solutions often have to be discovered through learning
Simple way to think about tame vs. wicked:
- Resolving tame problems helps us understand what is happening in the environmental system.
- Resolving wicked problems means we are deciding what we want to do within the environmental system.
Environmental problems are complex and deciding what to do about them is complicated by diverse stakeholder perspectives, a combination of science, economic and socially relevant data/information, and sometimes, regulatory constraints. Defining the problem and finding the necessary common ground in which to make policy decisions are difficult. This is a wicked problem.
What’s wicked got to do with it?
All decision making problems are wicked problems. Environmental decision making problems typically also contain tame problem components (i.e., we want to use scientific data in decision making). Knowing which components are tame vs. wicked in the environmental decision making problem flags those components that are best resolved with mathematical and scientific data and models (elegant solutions) vs. those components that are best resolved with facilitated stakeholder discussion and the application of values (clumsy solutions).
Examples of Wicked Environmental Problems: Global climate change, Sustainability, Environmental risk management
Tame Problems are those that can be clearly defined and agreed to by all stakeholders and have a clear right answer. A math problem is an example of a tame problem. Tame problems can be very complex. Going to Mars is an example of a tame but complex problem.
Environmental Solutions: MIRA facilitates "clumsy" solutions
Clumsy solutions are those that exhibit the following features:
Elegant solutions address tame problems. Examples of elegant solutions are the answer to an algebra problem, the experimental results from a toxicological study, or the engineering design of the Mars landing vehicle.
Learn more about how clumsy solutions work from Vermeij et al. 2006. Clumsy Solutions for a Complex World: The Case of Climate Change. Public Administration 84 (4): 817-843 and about how MIRA facilitates clumsy solutions in Applications.
How the MIRA modules connect with each other and with outside information:
Data Collection Manager
- allows users to store, sort, and retrieve data such as source emissions, demographics, and environmental quality values
Geostatistical Indicators Module
- allows users to take spatial fields (i.e., maps) and collapse them into a single number in order to compare one map to another in making a decision
Programmatic and Budget Decision Analysis Module
- provides a way to organize all decision criteria (i.e., indicators), include expert opinions and include what's important to stakeholders in making a decision
- outputs from Cause and Effect Models (from EPA and non-EPA sources) become inputs into MIRA; connects science with decision making (via the MIRA Decision Analysis Module)
- Mechanistic Models:
- Explicitly include the mechanisms or processes between the state variables; unlike empirical models. The parameters in mechanistic models should be supported by data and have real-world interpretations (EPA, 2009b). Examples are Fate and Transport models like Air Quality Models (AERMOD, CMAQ) and Water Quality Models (HSPF, SWAT)
- Empirical Models:
- Rely upon the observed relationships among experimental data typically because information about the underlying mechanism is not understood or available. These can be thought of as ‘best-fit’ models whose parameters may or may not have real-world interpretation. Examples are probabilistic models like regression models and exposure models.
- Integrated Modeling:
- A systems analysis-based approach to environmental assessment. It includes a set of interdependent science based components (models, data, and assessment methods) that together form the basis for constructing an appropriate modeling system. The constructed modeling system is capable of simulating the environmental stressor-response relationships relevant to a well specified problem statement. (Integrated Modeling for Integrate Environmental Decision Making, EPA100/R-08/010) An example is integrated environmental modeling Technologies (iemTechnologies)developed by EPA’s National Exposure Research Laboratory (NERL). It is a suite of tools for integrated environmental modeling, including Data for Environmental Modeling (D4EM), Framework for Risk Analysis for Multimedia Environmental Systems (FRAMES), and Supercomputer for Model Uncertainty and Sensitivity Evaluation (SuperMUSE).
- Data for Environmental Modeling (D4EM):
- D4EM interfaces National, Regional, State, Local and user defined databases with modeling systems, such as FRAMES.
- Framework for Risk Analysis for Multimedia Environmental Systems (FRAMES):
- FRAMES allows models to communicate with each other, facilitating the passage of data, resulting in the simulation of complex environmental processes.
- Supercomputer for Model Uncertainty and Sensitivity Evaluation (SuperMUSE):
- SuperMUSE combines a software and hardware to allows analysts to select one or more components of the MIRA-FRAMES architecture depending on their analytical needs and also give them the capability to perform model uncertainty and/or sensitivity analysis.
- Initialize/Define the Context
- Initialize/Define the Criteria/Narratives/Data and Organization of Indicators applicable to the Context
- Initialize/Apply significance to the data
- Initialize/Apply values
- Learning via iteration
- Understanding the consequences of each choice (i.e., decision uncertainty)
Select the number to the right of each application or keyword to jump to its corresponding reference.
|Creating an Environmental Index||7, 8|
|Ranking Environmental Elements
||1, 5, 10
2, 4, 5, 9, 10
2, 3, 11
Find Applications by Keywords
|Air Toxics||4, 5, 9, 10|
|Budget prioritization||5, 10|
|Clumsy solutions||9, 23|
|Decision Analysis/Decision Making||15, 16, 17, 18, 19, 20, 22, 23|
|Economics||1, 2, 3, 23|
|Ecosystem||5, 10, 20|
|Energy||1, 5, 10|
|Environmental Condition||2, 5, 10, 12, 21, 22|
|Indicators||1, 2, 5, 6, 10, 12, 14, 20, 21|
|Multi-pollutant||4, 5, 10|
|Ozone||2, 4, 5, 6, 9, 10, 12, 14, 23|
|PM2.5||4, 5, 9, 10|
|Public Health||4, 5, 6, 9, 10, 12, 14, 22|
|Risk Management||4, 5, 10, 12, 20, 22, 23|
|Stakeholders||1, 3, 4, 5, 6, 10, 12, 14, 15, 16, 17, 18, 19, 20, 23|
|Sustainability||5, 10, 20|
|Uncertainty||2, 3, 11, 21, 22, 23|
|Watershed Condition||5, 10|
- Using Environmental, Energy, Climate, Economic and Social Indicators to Evaluate Selected Crop Fertilizer Practices in the Chesapeake Bay Watershed (PDF) (1 pg, 533K)
- Air Quality Data - A Methodology to Assess Optimal Ozone Monitoring Network Design (PDF) (21 pp, 766K)
- Environmental Policy Analysis:How Much Uncertainty is Too Much and How Do We Know? (PDF) (29 pp, 1.3MB)
- Philadelphia Air Toxics Study: Evaluation of Risk Management Options Using MIRA (PDF) (1 pg, 83K)
- Trans-Disciplinary Learning: A Case Study Linking Science to Budgets (PDF) (18 pp, 1.3K)
- Evaluating Ozone Nonattainment: Technical Support Document for the 8-hour Ozone Designations 11-Factor Analysis
- Technical Support Document Cover Memo (PDF) (27 pp, 137K)
- Errata Sheet to Technical Support Document (PDF) (1 pg, 8.4K)
- Attachment A - Letter Discussing the MIRA Approach (PDF) (7 pp, 80K)
- Attachment B - 8-hour Ozone Nonattainment Designations - Comparison with EPA 120d Recommendations vs. State Responses (PDF) (6 pp, 52K)
- Attachment C - Memorandum of Agreement to Allow EPA to Grant the Separation of the Fredericksburg 8-Hour Ozone Nonattainment Area From the Washington, DC 8-Hour Ozone Nonattainment Area (PDF) (3 pp, 34K)
- Attachment D
- Charles City County, VA 2001 (PDF) (9 pp, 340K)
- Charles City County, VA 2002 (PDF) (9 pp, 375K)
- Charles City County, VA 2003 (PDF) (9 pp, 394K)
- Hampton Roads, VA 2001(PDF) (9 pp, 368K)
- Hampton County, VA 2002 (PDF) (9 pp, 347K)
- Hampton Roads, VA 2003 (PDF) (9 pp, 356K)
- Henrico County, VA 2001 (PDF) (9 pp, 336K)
- Henrico County, VA 2002 (PDF) (9 pp, 410K)
- Henrico County, VA 2003 (PDF) (9 pp, 369K)
- Attachment E
- Attachment F
- PM2.5 Public Health Community of Practice - Baltimore, MD (PDF) (19pp, 287K)
- Indicators and Implicit Weighting Scheme of the Hazard Ranking System (HRS) (PDF) (17pp, 78K)
Please contact your local library or EPA's Philadelphia library to obtain the unlinked documents listed below.
- Stahl, C.H. and A.J. Cimorelli. 2013. A Demonstration of the Necessity and Feasibility of Using a Clumsy Decision Analytic Approach on Wicked Environmental Problems. Integrated Environmental Assessment and Management 9(1): 17-30.
- Stahl, C.H., A.J. Cimorelli, C. Mazzarella and B. Jenkins. 2011. Toward Sustainability: A Case Study Demonstrating Transdiciplinary Learning Through the Selection and Use of Indicators in a Decision Making Process. Integrated Environmental Assessment and Management 7(3): 483-498.
- Sanderson, H., C. H. Stahl, R. Irwin and M. D. Rogers. 2005. Reflections on uncertainty in risk assessment and risk management by the Society for Environmental Toxicology and Chemistry's (SETAC) precautionary principle workgroup. Water Science and Technology 52: 73-79.
- Cimorelli, Alan J. and Cynthia H. Stahl. 2005. Tackling the Dilemma of the Science-Policy Interface in Environmental Policy Analysis . Bulletin of Science, Technology, and Society 25: 46-52.
- Stahl, Cynthia H. and Alan J. Cimorelli. 2005. How Much Uncertainty is Too Much and How Do We Know? A Case Example of the Assessment of Ozone Monitoring Network Options. Risk Analysis 25: 1109-1120.
- Stahl, Cynthia H., Cristina Fernandez and Alan J. Cimorelli. April 15, 2004. Technical Support Document for the Region III 8-hour Ozone Designations 11-Factor Analysis. Philadelphia, PA: U.S. Environmental Protection Agency, Region III.
- Stahl, Cynthia H. 2003. Multi-criteria Integrated Resource Assessment (MIRA): A New Decision Analytic Approach to Inform Environmental Policy Analysis. University of Delaware. For the degree of Doctor of Philosophy.
- Stahl, Cynthia H., Alan J. Cimorelli and Alice H. Chow. 2002. "A New Approach to Environmental Decision Analysis: Multi-criteria Integrated Resource Assessment (MIRA)" (PDF) (17 pp, 146K) , Bulletin of Science, Technology, and Society 22: 443-459.
- EPA Office of the Inspector General (August 15, 2002), "Consistency and Transparency in Determination of EPA's Anticipated Ozone Designations" (PDF) (36 pp, 115K), Report No. 2002-S-00016
- Cimorelli, Alan J., Cynthia H. Stahl, Alice H. Chow and Cristina Fernandez. June 1999. Decision Consequence Model (DCM): Integrating Environmental Data and Analysis Into Real-Time Decision Making paper presented to the Air and Waste Management Association Conference in St. Louis, Missouri.
- Stahl, Cynthia H., Hong-Jin Kim, Alan J. Cimorelli and Alice H. Chow. 1999. Decision Consequence Model (DCM): Integrating Environmental, Social, Political and Economic Impact Assessment into Real-Time Decision Making. Proceedings of the 19th Annual Meeting of the International Association for Impact Assessment, Glasgow, Scotland
- Stahl, Cynthia H. and Todd S. Bridges. 2013. "Fully baked" sustainability using decision analytic principles and ecosystem services . Integrated Environmental Assessment and Management 9(4): 551-553.
- Cimorelli, Alan J. and Cynthia H. Stahl. 2014. Avoiding "Proofiness": Addressing uncertainty in environmental characterization . Integrated Environmental Assessment and Management 10(1): 141-142.
- U.S. EPA, Office of the Science Advisor, Risk Assessment Forum. July 2014. Risk Assessment Forum White Paper: Probabilistic Risk Assessment Methods and Case Studies (PDF) (98pp., 5.2MB), EPA 100/R-14/004.
- Cynthia H. Stahl. 2014. Out of the Land of Oz: the importance of tackling wicked environmental problems without taming them. Environmental Systems and Decisions 34 (4):473-477.
Cynthia Stahl (firstname.lastname@example.org)
Decision Analysis Module
Office of Environmental Information & Analysis (3EA10)
Environmental Assessment & Innovation Division
US EPA Region 3
1650 Arch St.
Philadelphia, PA 19103-2029
Janet Kremer (email@example.com)
Model Outputs / Integrated Modeling
Office of Environmental Information and Analysis (3EA10)
Environmental Assessment & Innovation Division
US EPA Region 3
1650 Arch St.
Philadelphia, PA 19103-2029