Jump to main content or area navigation.

Contact Us

Benchmark Dose Software (BMDS)

IV. APPLICATION OF BMDS

  1. Cancer Bioassay Data
  2. Under EPA's proposed 1996 Guidelines for Carcinogen Risk Assessment, quantitative risk estimates from cancer bioassay data are typically calculated by modeling the data in the observed range to estimate a BMDL for a BMR of 10% extra risk, which is generally at the low end of the observable range for standard cancer bioassay data. This BMDL then serves as the "point of departure" for linear extrapolation or a nonlinear quantitative approach, as warranted by the mode of action of the carcinogen. This example uses data from EPA's 1988 Health and Environmental Effects Document for Dibromochloromethane and reflects the reporting requirement outlined earlier in Section II.E.

    EXAMPLE 1

    1. Study or Studies Selected
      1. Rational (Study/Endpoint): The rationale for study selection and endpoint selection, while important components of any comprehensive write-up of a BMD calculation, are beyond the scope of this quantitative example.
      2. tumor type: hepatocellular adenoma or carcinoma
      3. test animal: B6C3F1 mouse, female
      4. route of exposure: gavage
      5. study: NTP, 1985
      6. Example 2: Contacts for the Fish Tissue Study
        Administered Dose (mg/kg/day) Human Equivalent Dose (mg/kg/day) Tumor Incidence
        0 0 6/50
        50 2.83 10/49
        100 5.67 19/50
      7. Dose-Response Data
    2. Dose Response Model Chosen: The multistage model was used because it is considered the default model for cancer bioassy data; application of other dichotomous models is encouraged for comparison purposes, but will not be described here for the sake of simplicity. Since a well described and documented model was used (BMDS Multistage v.2.1) details of the parameter estimation technique do not need to be described. Less used published models might require a more complete discussion of parameter estimation techniques (e.g. maximum likelihood, least squares, generalized estimating, equations).
      1. Model: Multistage, extra risk - A second-degree (i.e., n-1) multistage model
      2. Model Form: background + (1-background)* [1-EXP(-betal*dose^2)]
      3. Results
        1. Parameter Estimates
        2. Parameter Estimates
          Parameter Estimate (MLEs) Std. Error
          background 0.12 0.132665
          beta (1) 0.00930036 0.141898
          beta (2) 0.00925286 0.0246904
        3. AIC=158.688
        4. p-value=1
        5. Chi-Square =0
        6. Residuals=0
    3. Choice of BMR: A BMR of 10% extra risk was used, as is typical for standard cancer bioassay data.
    4. Computation of BMD BMD (ED10 ) = 2.91 mg/kg/day
    5. Computation of BMDL BMDL (LED10; 95% confidence limit estimated by likelihood profile) = 1.25 mg/kg/day
    6. Graphics
    7. Figure 1 - Second Degree Multistage Model

      Figure 1 - Second Degree Multistage Model

      EXAMPLE 2

      1. Study Selected
        Same study, but a first-degree multistage model is fit to the data to see if a more parsimonious model can also provide an adequate fit.
      2. Dose Response Model Chosen
        1. Model: Multistage extra risk - a first degree multistage model;
        2. Model form: background + (1-background) * [1-EXP(-beta1*dose^1)];
        3. Results
          1. Parameter Estimates
          2. Parameter Estimates
            Parameter Estimate (MLEs) Std. Error
            background 0.111488 0.120556
            beta (1) 0.0559807 0.0391492
          3. AIC = 157.272
          4. p-value = 0.4446
          5. chi-square=0.57
          6. Residuals
          7. Dose Est. Prob. Expected Observed Size Chi^2 Residuals
            0.0000 0.1115 5.574 6 50 0.086
            2.8300 0.2417 11.842 10 49 -0.205
            5.6700 0.3531 17.657 19 50 0.118

        Choice of BMR and Computation of BMD/BMDL

      3. BMR=10%
      4. BMD (ED10 ) = 1.88 mg/kg/day
      5. BMDL (LED10 ; 95% confidence limit estimated by likelihood profile) = 1.20 mg/kg/day
      6. Graphics
      Plot - This plot shows the results of a first-degree multistage model run on the data described above. The model curve does not go directly through all data points, but provides a reasonably good fit to the data.  The BMD and BMDL described above for the first-degree model results are identified with green lines on the graph.

      Figure 2 - First Degree Multistage Model

      The AIC is lower for the first-degree model suggesting that this is the preferred model; however, because the multistage model is really a family of k-degree models, a likelihood ratio test can be used to evaluate whether the improvement in fit afforded by estimating additional parameters is justified. In this case, the log likelihood for the second-degree model was -76.3439 and for the first-degree model was -76.6361. Thus twice the absolute difference in the log likelihoods is less than 3.84, i.e., a Chi-square with one degree of freedom (i.e., 2-1), suggesting that the first- degree multistage model is not significantly different from the second-degree model. Under the recommendations of the benchmark dose guidance, the more parsimonious first-degree model would be generally preferred. Final judgement on this may be subject to endpoint-specific guidance.

      References

      NTP (National Toxicology Program). 1985. Toxicology and carcinogenesis studies of chlorodibromomethane (CAS No. 124-48-1) in F344/N rats and B6C3F1 mice (gavage studies). NTP Tech. Report Series No. 282. NTIS PB 86-166675.

      U.S. EPA. 1988. Health and Environmental Effects Document for Dibromochloromethane. Prepared by the Office of Health and Environmental Assessment, Environmental Criteria and Assessment Office, Cincinnati, OH. ECAO-CIN- GO40.

    Top of page


  3. Problematic Data Sets
  4. It is preferable to have studies with one or more doses near the level of the BMR to give a better estimate of the BMD, and thus, a shorter confidence interval. Studies in which all the dose levels show changes compared with control values (i.e., there is no NOAEL) are readily useable in BMD analyses, unless the lowest response level is much higher than the BMR. Data sets in which only one dose elicits an increased response (Problem Data Set #1), or data sets in which all non-control doses have essentially the same increased response level (Problem Data Set #2) provide limited information about the dose-response, since the most of the range of response from background to maximum occurs somewhere between two doses.

    Plot - This plot depicts the dose-response curve estimated by the Weibull model for Problematic Dataset #1.  It shows control and low dose data points that are at zero response and a high dose point with a very large response, causing the modelled dose-reponse to have an exagerated hockey stick form.
    Plot - This plot depicts the dose-response curve estimated by the Weibull model for Problematic Dataset #2.  It shows dose-response data for which response (fraction affected) rapidly increases with increasing dose.  The Weibull model curve fit (red line) has a virually infinite slope near the origin, and approaches zero near the final data point.

    When this latter situation arises in quantal data, it is tempting to use a model like the Weibull with no restrictions on the power parameter, because such models can reach a plateau of less than 100%. However, this can result in seriously distorted BMDs, because the model predictions jump rapidly from background levels to the maximum level (see the appendix of the draft 2000 BMD Technical Guidance for a detailed example of this situation). In both situations, other models could be found that force the BMD to be anywhere between that extreme and the lowest dose that elicits a response. Thus the computed BMD will depend solely on the model selected, and goodness of fit provides no help in selecting among the possibilities. These data provide little useful information about dose-response; the ideal solution is to collect further data in the dose-range missed by the studies in hand.

    Top of page


  5. Improving Model Fit
  6. There are three basic ways to improve the fit of a given BMDS model to a data set, (1) relax all restrictions on model parameters, (2) alter the data set (e.g., by dropping high doses) and (3) trying alternative initial parameter values. As can be seen from the previous example under Problematic Data Sets and the example below, the first solution can lead to unacceptable dose-response curves.

    1. Relaxing Parameter Constraints
    2. In Polynomial and Multistage models:

      Polynomial and Multistage formula

      Beta (ß)parameters can be constrained to be either greater than or less than or equal to 0, to keep the slope of the dose-response monotonic.

      If ßs in a polynomial model are allowed to change signs, unacceptable curvature can result.

      Plot - This plot depicts the dose-response curve estimated by the Multistage model when the beta coefficients of the model are allowed to be negative values.  The estimated best fit (red line) takes a meandering (up and down) nonmonotonic path through the four dose-response data points.

      Figure 3 - Unrestricted Multistage Model Run

    3. Altering the Dataset (e.g., Dropping High Doses)
    4. It is sometimes acceptable to alter the data set to achieve an acceptable model fit. An example is for data from satorable effects. The response from saturable effects may "plateau" or flatten out at high doses. Most models have a difficult time fitting both the low dose and high dose regions of such data. Since the low dose region is usually the area of risk assessment concern, it is sometimes acceptable to drop high doses to achieve model fit, as long as there are enough data left to adequately define the low dose region.

      Examples: Improving Model Fit

      The Hill model (see equation below) is often used to fit data from saturable endpoints because it contains an asymptote term (v) that allows it to estimate a plateau level. Below is an example of a Hill model fit to data from a saturable endpoint. The P-value for this model ran (P=0.001777) suggests an inadequate model fit.

      Hill Model Formula

       

      Plot - This plot depicts the dose-response curve estimated by the Hill model for a data set that contains a large number of dose groups that elicited low mean reponses and a few high dose groups that elicited much higher responses.  The response level was virtually the same for all of these high dose groups.  Because of the nature of the dose spacing for the exposed groups (smaller intervals for low dose groups), it is difficult to see how well the model fit the lower dose data.
      Hill Model Chart
      Plot - This plot depicts the dose-response curve estimated by the Hill model for the data set as above, but with the widely spaced high dose groups removed.  The model fit to this data is easier to view and seems improved.

      After dropping the high doses, an adequate fit is achieved (P=0.4577).

      Note: BE CAREFUL! Extra Risk estimate is impacted in Hill Model when high doses are dropped. This is because Extra Risk for the continuous Hill Model is based on the Hill model estimate of the asymptote (v) level, which is highly dependent on the response at higher doses.

    Top of page


  7. Choosing Alternative Initial Parameter Values
    If a model does not appear to fit the data (low p value), it may be possible to improve the model fit by choosing different starting values for the parameters associated with the model. This can be done by manually initializing parameter values in the advanced mode of BMDS. For more details and an example of the use of this model fitting technique see the latest draft of the EPA Technical Guidance Document.
  8.  


  9. Examples: Comparing Models
    1. Global Goodness-of-Fit Measures for Quantal (Dichotomous) Models
      1. Hierarchically related models can also be compared by log (likelihood) values presented in the BMDS Analysis of Deviance Tables.

      Examples: Analysis of Deviance Tables

      Weibull
      Model Log (likelihood) Deviance Test DF P-value
      Full model -44.6916      
      Fitted model -44.6917 1e-005 2 0.9975
      Reduced model -138.379 187.375 3 < .0001
      Quantal Quadratic
      Model Log (likelihood) Deviance Test DF P-value
      Full model -44.6916      
      Fitted model -56.7536 24.1238 3 < .0001
      Reduced model -138.379 187.375 3 < .0001
    2. Unrelated models can be compared using Akaike Information Coefficient (AIC), and similar measures. AIC is defined as - 2 log L + 2 p, where log L is the log likelihood of the fit, and p is the number of parameters estimated. Smaller values indicate better fits. For example, the following data would suggest the use of the Log-Probit model.
    3. Log-Probit model
      Model P LL AIC
      Logistic 2 -36.1812 76.3624
      Log-Logistic 2 -35.9778 75.9556
      Probit 2 -36.0181 76.0362
      Log-Probit 2 -35.9518 75.9036
      Ms (3) 1 -40.0839 82.1678
      Ms (6) 2 -36.0098 76.0196
      Gamma 2 -35.9557 75.9114
      Weibull 2 -36.0064 76.0128
    4. Global Measures for Continuous Models - The BMDS continuous models use a sequence of likelihood ratio tests to evaluate goodness-of-fit, instead of the chi-square tests used for dichotomous data.
      1. BMDS addresses the following questions in evaluating Continuous Model fit (assuming that the variance is modeled; following is from output):
        1. Test 1: Does response and/or variances differ among Dose levels?
        2. Test 2: Are Variances Homogeneous?
        3. Test 3: Are Variances Adequately Modeled? (A2 vs. A3)
        4. Test 4: Does the Model for the Mean Fit?

          If the variance is not modeled (i.e., use selected "constant variance"); Test 3 does not apply.

        5. For more information on these tests see Continuous Model Text Output.

    5. Local Measures of Fit
      1. Scaled residuals, for example, chi-squared residuals. The following table provides data from a multistage model run.
      Goodness of Fit
      Dose Est._Prob. Expected Observed Size X2 Resid
      0.0000 0.0000 0.000 0 50 0.0
      0.4500 0.3932 19.660 11 50 -2.51
      0.6500 0.7781 38.905 44 50 1.73
      1.0000 0.9958 49.792 50 50 0.45

      As a rule of thumb, residuals should be less than 2.0 in absolute value.

    6. Visual Comparison
    7. Plot - On the left is a plot showing a multistage model fit through four data points.  This data is not fit well by the multistage model, which estimates a shallow S shaped curve that goes between most of the data points.  On the right, a Weibull model predicted curve is shown that goes through each one of these same data points.  The Weibull curve has more of an accentuated S shape.

BMDS Quiz

Previous
Back to Previous Section
Next
Forward to the Next Section

Research Home | Search EPA | Satisfaction Questionnaire

Jump to main content.