CADDIS Volume 1: Stressor Identification
Step 5: Identify Probable Causes
On this page
- Evaluating the quantity and quality of evidence
- Types of evidence that use multiple types of evidence
- Evaluating consistency and credibility
- Summarizing the compelling evidence
- Summarizing the strength of evidence for each candidate cause
Types of evidence
You can be confident in deciding that a candidate cause did or did not lead to the impairment when all the evidence has high quality and makes sense qualitatively and quantitatively. When the supporting data are few or of poor quality, confidence is low. When the some types of evidence weaken and others strengthen the case for a candidate cause, uncertainty is greater and a determination is difficult. However, even when based on poor or minimal information, the causal analysis will still be useful as a screening assessment to reduce the number of candidate causes or to identify data needs. Therefore, "don't let the perfect be the enemy of the good!" Instead, use all of the evidence that you have to make what inferences you can.
This section describes how to evaluate the body of evidence for each candidate cause and how to identify probable, uncertain, and unlikely causes. When evaluating the evidence for a candidate cause, the quantity and quality of each type of evidence is evaluated separately; then consistency and credibility of the entire body of evidence is evaluated. Use of this consistent and transparent process will result in more defensible conclusions.
Evaluating the quality and quantity of the data and evidence derived from the data is essential for a confident assessment. You have been evaluating data quality and selecting the highest quality data available throughout the process. The quality and quantity of data and evidence influence the scores that were assigned during steps 3 and 4. You may wish to review the background on scoring for a refresher of the rationale for scoring the lines of evidence, and view an example summary scoring table. Then, while weighing all of the arguments for and against a candidate cause, be sure that information about the quality of the data and evidence have been captured.
- High-quality data are always superior to questionable data or data of uncertain origin. You may choose to exclude or discount poor-quality data. However, if you do, first consider if the low quality data can help you to identify the type and quality of new data that would improve the assessment.
- Increasing the number of types or pieces of evidence decreases the likelihood that any one faulty study or data set will mislead you. Likewise, increasing the quantity of data increases confidence in the conclusions of a particular study or your own calculations and thus increases the quality of the evidence.
Here is a list of characteristics to consider when evaluating quantity and quality of data and evidence. (For more detail concerning data quality, refer to the supplemental pages on quantifying uncertainty and assuring data quality).
- Number of pieces of evidence,
- Number of types of evidence evaluated,
- Quality of the data,
- Quantity of the data,
- Proper sampling design,
- Relevance of the data from elsewhere to the case at hand, and
- Distribution of the data across candidate causes.
All of the evidence from the case and from elsewhere is summarized for each candidate cause in terms of its consistency and in terms of mechanistic explanations of any inconsistencies (Table 5.1).
The consistency and credibility of the overall argument is just as important as the quality and quantity of data and evidence marshaled to support the case. Whereas the quality and quantity of each piece of evidence was evaluated individually, now the types of evidence are considered together. Do they tell a consistent story? When the candidate cause is consistently supported or weakened by many types of evidence, the confidence in the argument for or against the cause increases. The number of types of evidence makes a difference. It is unlikely to find eight different types of evidence all supporting a cause by chance. In contrast, consistent support for a cause by only one or two types of evidence could easily occur by chance alone. Sometimes there is a reasonable explanation for why a type of evidence does not agree with the rest of the evidence. So, if inconsistent evidence can be explained by a mechanistic, conceptual, or mathematical model, then the confidence in the argument for a candidate cause increases.
Evaluate consistency by bringing together the summary tables produced in Steps 3 and 4. Evaluate each specific effect individually. Although this makes for a complicated summary, it is important to do because different candidate causes may be eliciting different effects. Resist the temptation to add up the scores. Adding the scores erroneously implies that each type of evidence is equally important and is equitable only if the same types of evidence are available across all candidates. Further, the symbols are not units. Consider a candidate cause with two types of evidence, each with a score of +, giving a sum of ++ (1+1=2), and another with three types of evidence with scores of +++, ++ and - - - (3+2-3=2). Both sum to 2, but the triple negative score may be strong enough to refute the candidate cause! Instead, please use the scoring tables to identify the most compelling pieces of evidence and to develop an overall sense of the case for each candidate cause. See the Consistency of Evidence and Explanation of Evidence pages for more detailed discussion of these concepts.
After scoring the body of evidence for consistency, make a preliminary evaluation of the potential for the candidate cause to have led to each specific effect. A strong case is based on evidence that demonstrates four or five characteristics of causal relationships using many types and pieces of evidence. The investigator records the most compelling evidence for or against each candidate cause. This evidence will be used to convince stakeholders and decision-makers.
Although there are fifteen types of evidence, they can be usefully thought of as potentially supporting the six characteristics of causal relationships listed in Table 5.2 below. Confidence in a cause is increased if the supporting evidence addresses all six characteristics. Bear in mind, however, that it is not necessary that you demonstrate all six characteristics to satisfy the decision-makers and stakeholders involved in the case.
|Characteristics of Causal Relationship||Principle|
|Co-occurrence||The cause co-occurs with the unaffected entity in space and time.|
|Sufficiency||The intensity, frequency, and duraction of the cause are adequate and the entity is susceptible to produce the type and magnitude of the effect.|
|Time order||The cause precedes its effects.|
|Alteration||The entity is changed by the interaction with the cause.|
|Interaction||The cause physically interacts with the entity in a way that induces the effect.|
|Preceding causation||Each causal relationship is a result of a larger web of cause and effect relationships.|
Tables 5.3 and 5.4 list the characteristics of causal relationships supported by different types of evidence.
Table 5.5 summarizes options for categorizing the status of each candidate cause after the evidence is weighed. These results are used to compare across causes as described in the Compare Evidence Among Causes part of Step 5.
|Cause refuted by indisputable evidence||Refuted|
|Cause of impairment identified by diagnostic symptoms||Diagnosed|
|Cause of impairment refuted by diagnostic symptoms||Refuted|
|All evidence supports the case for the cause, evidence for three or four characteristics of causal relationships||Probable|
|All evidence weakens the case for the cause, evidence against three or four characteristics of causal relationships||Unlikely|
|All evidence supports the case for the cause, evidence for only one or two characteristics of causal relationships||Probable with low confidence|
|All evidence weakens the case for the cause, evidence against one or two characteristics of causal relationships||Unlikely with low confidence|
|Some evidence supports and some weakens the case for the cause||Unlikely with low confidence|
|Insufficient evidence to make a determination||Additional information required|
Next, in Compare Evidence Among Causes, candidate causes are compared to determine if there is just one probable cause, more than one probable cause, or some other conclusion.
Previous: Overview | Next: Compare Evidence Among Causes | Step-by-Step Guide Introduction