Benchmark Dose Software (BMDS)
IV. APPLICATION OF BMDS
Navigation
- Cancer Bioassay Data
- Study or Studies Selected
- Rational (Study/Endpoint): The rationale for study selection and endpoint selection, while important components of any comprehensive write-up of a BMD calculation, are beyond the scope of this quantitative example.
- tumor type: hepatocellular adenoma or carcinoma
- test animal: B6C3F1 mouse, female
- route of exposure: gavage
- study: NTP, 1985
- Dose-Response Data
- Dose Response Model Chosen: The multistage model was used because it is considered the default model for cancer bioassy data; application of other dichotomous models is encouraged for comparison purposes, but will not be described here for the sake of simplicity. Since a well described and documented model was used (BMDS Multistage v.2.1) details of the parameter estimation technique do not need to be described. Less used published models might require a more complete discussion of parameter estimation techniques (e.g. maximum likelihood, least squares, generalized estimating, equations).
- Model: Multistage, extra risk - A second-degree (i.e., n-1) multistage model
- Model Form: background + (1-background)* [1-EXP(-betal*dose^2)]
- Results
- Parameter Estimates
- AIC=158.688
- p-value=1
- Chi-Square =0
- Residuals=0
- Choice of BMR: A BMR of 10% extra risk was used, as is typical for standard cancer bioassay data.
- Computation of BMD BMD (ED10 ) = 2.91 mg/kg/day
- Computation of BMDL BMDL (LED10; 95% confidence limit estimated by likelihood profile) = 1.25 mg/kg/day
- Graphics
- Study Selected
Same study, but a first-degree multistage model is fit to the data to see if a more parsimonious model can also provide an adequate fit. - Dose Response Model Chosen
- Model: Multistage extra risk - a first degree multistage model;
- Model form: background + (1-background) * [1-EXP(-beta1*dose^1)];
- Results
- Parameter Estimates
- AIC = 157.272
- p-value = 0.4446
- chi-square=0.57
- Residuals
- BMR=10%
- BMD (ED10 ) = 1.88 mg/kg/day
- BMDL (LED10 ; 95% confidence limit estimated by likelihood profile) = 1.20 mg/kg/day
- Graphics
- Problematic Data Sets
- Improving Model Fit
- Relaxing Parameter Constraints
- Altering the Dataset (e.g., Dropping High Doses)
- Choosing Alternative Initial Parameter Values
If a model does not appear to fit the data (low p value), it may be possible to improve the model fit by choosing different starting values for the parameters associated with the model. This can be done by manually initializing parameter values in the advanced mode of BMDS. For more details and an example of the use of this model fitting technique see the latest draft of the EPA Technical Guidance Document. - Examples: Comparing Models
- Global Goodness-of-Fit Measures for Quantal (Dichotomous) Models
- Hierarchically related models can also be compared by log (likelihood) values presented in the BMDS Analysis of Deviance Tables.
- Unrelated models can be compared using Akaike Information Coefficient (AIC), and similar measures. AIC is defined as - 2 log L + 2 p, where log L is the log likelihood of the fit, and p is the number of parameters estimated. Smaller values indicate better fits. For example, the following data would suggest the use of the Log-Probit model.
- Global Measures for Continuous Models - The BMDS continuous models use a sequence of likelihood ratio tests to evaluate goodness-of-fit, instead of the chi-square tests used for dichotomous data.
- BMDS addresses the following questions in evaluating Continuous Model fit (assuming that the variance is modeled; following is from output):
- Test 1: Does response and/or variances differ among Dose levels?
- Test 2: Are Variances Homogeneous?
- Test 3: Are Variances Adequately Modeled? (A2 vs. A3)
- Test 4: Does the Model for the Mean Fit?
If the variance is not modeled (i.e., use selected "constant variance"); Test 3 does not apply.
- Local Measures of Fit
- Scaled residuals, for example, chi-squared residuals. The following table provides data from a multistage model run.
- Visual Comparison
Under EPA's proposed 1996 Guidelines for Carcinogen Risk Assessment, quantitative risk estimates from cancer bioassay data are typically calculated by modeling the data in the observed range to estimate a BMDL for a BMR of 10% extra risk, which is generally at the low end of the observable range for standard cancer bioassay data. This BMDL then serves as the "point of departure" for linear extrapolation or a nonlinear quantitative approach, as warranted by the mode of action of the carcinogen. This example uses data from EPA's 1988 Health and Environmental Effects Document for Dibromochloromethane and reflects the reporting requirement outlined earlier in Section II.E.
EXAMPLE 1
| Administered Dose (mg/kg/day) | Human Equivalent Dose (mg/kg/day) | Tumor Incidence |
|---|---|---|
| 0 | 0 | 6/50 |
| 50 | 2.83 | 10/49 |
| 100 | 5.67 | 19/50 |
| Parameter | Estimate (MLEs) | Std. Error |
|---|---|---|
| background | 0.12 | 0.132665 |
| beta (1) | 0.00930036 | 0.141898 |
| beta (2) | 0.00925286 | 0.0246904 |
Figure 1 - Second Degree Multistage Model
EXAMPLE 2
| Parameter | Estimate (MLEs) | Std. Error |
|---|---|---|
| background | 0.111488 | 0.120556 |
| beta (1) | 0.0559807 | 0.0391492 |
| Dose | Est. Prob. | Expected | Observed | Size | Chi^2 Residuals |
|---|---|---|---|---|---|
| 0.0000 | 0.1115 | 5.574 | 6 | 50 | 0.086 |
| 2.8300 | 0.2417 | 11.842 | 10 | 49 | -0.205 |
| 5.6700 | 0.3531 | 17.657 | 19 | 50 | 0.118 |
Choice of BMR and Computation of BMD/BMDL
Figure 2 - First Degree Multistage Model
The AIC is lower for the first-degree model suggesting that this is the preferred model; however, because the multistage model is really a family of k-degree models, a likelihood ratio test can be used to evaluate whether the improvement in fit afforded by estimating additional parameters is justified. In this case, the log likelihood for the second-degree model was -76.3439 and for the first-degree model was -76.6361. Thus twice the absolute difference in the log likelihoods is less than 3.84, i.e., a Chi-square with one degree of freedom (i.e., 2-1), suggesting that the first- degree multistage model is not significantly different from the second-degree model. Under the recommendations of the benchmark dose guidance, the more parsimonious first-degree model would be generally preferred. Final judgement on this may be subject to endpoint-specific guidance.
References
NTP (National Toxicology Program). 1985. Toxicology and carcinogenesis studies of chlorodibromomethane (CAS No. 124-48-1) in F344/N rats and B6C3F1 mice (gavage studies). NTP Tech. Report Series No. 282. NTIS PB 86-166675.
U.S. EPA. 1988. Health and Environmental Effects Document for Dibromochloromethane. Prepared by the Office of Health and Environmental Assessment, Environmental Criteria and Assessment Office, Cincinnati, OH. ECAO-CIN- GO40.
It is preferable to have studies with one or more doses near the level of the BMR to give a better estimate of the BMD, and thus, a shorter confidence interval. Studies in which all the dose levels show changes compared with control values (i.e., there is no NOAEL) are readily useable in BMD analyses, unless the lowest response level is much higher than the BMR. Data sets in which only one dose elicits an increased response (Problem Data Set #1), or data sets in which all non-control doses have essentially the same increased response level (Problem Data Set #2) provide limited information about the dose-response, since the most of the range of response from background to maximum occurs somewhere between two doses.
When this latter situation arises in quantal data, it is tempting to use a model like the Weibull with no restrictions on the power parameter, because such models can reach a plateau of less than 100%. However, this can result in seriously distorted BMDs, because the model predictions jump rapidly from background levels to the maximum level (see the appendix of the draft 2000 BMD Technical Guidance for a detailed example of this situation). In both situations, other models could be found that force the BMD to be anywhere between that extreme and the lowest dose that elicits a response. Thus the computed BMD will depend solely on the model selected, and goodness of fit provides no help in selecting among the possibilities. These data provide little useful information about dose-response; the ideal solution is to collect further data in the dose-range missed by the studies in hand.
There are three basic ways to improve the fit of a given BMDS model to a data set, (1) relax all restrictions on model parameters, (2) alter the data set (e.g., by dropping high doses) and (3) trying alternative initial parameter values. As can be seen from the previous example under Problematic Data Sets and the example below, the first solution can lead to unacceptable dose-response curves.
In Polynomial and Multistage models:
Beta (ß)parameters can be constrained to be either greater than or less than or equal to 0, to keep the slope of the dose-response monotonic.
If ßs in a polynomial model are allowed to change signs, unacceptable curvature can result.
Figure 3 - Unrestricted Multistage Model Run
It is sometimes acceptable to alter the data set to achieve an acceptable model fit. An example is for data from satorable effects. The response from saturable effects may "plateau" or flatten out at high doses. Most models have a difficult time fitting both the low dose and high dose regions of such data. Since the low dose region is usually the area of risk assessment concern, it is sometimes acceptable to drop high doses to achieve model fit, as long as there are enough data left to adequately define the low dose region.
Examples: Improving Model Fit
The Hill model (see equation below) is often used to fit data from saturable endpoints because it contains an asymptote term (v) that allows it to estimate a plateau level. Below is an example of a Hill model fit to data from a saturable endpoint. The P-value for this model ran (P=0.001777) suggests an inadequate model fit.
After dropping the high doses, an adequate fit is achieved (P=0.4577).
Note: BE CAREFUL! Extra Risk estimate is impacted in Hill Model when high doses are dropped. This is because Extra Risk for the continuous Hill Model is based on the Hill model estimate of the asymptote (v) level, which is highly dependent on the response at higher doses.
Examples: Analysis of Deviance Tables
| Model | Log (likelihood) | Deviance | Test DF | P-value |
|---|---|---|---|---|
| Full model | -44.6916 | |||
| Fitted model | -44.6917 | 1e-005 | 2 | 0.9975 |
| Reduced model | -138.379 | 187.375 | 3 | < .0001 |
| Model | Log (likelihood) | Deviance | Test DF | P-value |
|---|---|---|---|---|
| Full model | -44.6916 | |||
| Fitted model | -56.7536 | 24.1238 | 3 | < .0001 |
| Reduced model | -138.379 | 187.375 | 3 | < .0001 |
| Model | P | LL | AIC |
|---|---|---|---|
| Logistic | 2 | -36.1812 | 76.3624 |
| Log-Logistic | 2 | -35.9778 | 75.9556 |
| Probit | 2 | -36.0181 | 76.0362 |
| Log-Probit | 2 | -35.9518 | 75.9036 |
| Ms (3) | 1 | -40.0839 | 82.1678 |
| Ms (6) | 2 | -36.0098 | 76.0196 |
| Gamma | 2 | -35.9557 | 75.9114 |
| Weibull | 2 | -36.0064 | 76.0128 |
For more information on these tests see Continuous Model Text Output.
| Dose | Est._Prob. | Expected | Observed | Size | X2 Resid |
|---|---|---|---|---|---|
| 0.0000 | 0.0000 | 0.000 | 0 | 50 | 0.0 |
| 0.4500 | 0.3932 | 19.660 | 11 | 50 | -2.51 |
| 0.6500 | 0.7781 | 38.905 | 44 | 50 | 1.73 |
| 1.0000 | 0.9958 | 49.792 | 50 | 50 | 0.45 |
As a rule of thumb, residuals should be less than 2.0 in absolute value.
 Back to Previous Section |
 Forward to the Next Section |