predint

In many pharmaceutical and biomedical applications such as assay validation, assessment of historical control data or the detection of anti-drug antibodies, prediction intervals are of use. The package predint provides functions to calculate bootstrap calibrated prediction intervals for one or more future observation8s) based on overdispersed binomial data, overdispersed poisson data, as well as data that is modeled by linear random effects models fitted with lme4::lmer(). The main functions are:

For all of these functions, it is assumed that the historical, as well as the actual (or future) data descend from the same data generating process.

Installation

You can install the released version of predint from CRAN with:

install.packages("predint")

And the development version from GitHub with:

# install.packages("devtools")
devtools::install_github("MaxMenssen/predint")

Example

The following examples are based on the scenario described in Menssen and Schaarschmidt 2019: Based on historical control data for the mortality of male B6C3F1-mice obtained in long term studies at the National Toxicology Program (NTP 2017), prediction intervals (PI) can be computed in order to validate the observed mortality of actual (or future) control groups.

On the one hand, a PI for one future observation can be computed in order to validate the outcome of one actual (or future) untreated control group that is compared with several groups treated with the compound of interest.

On the other hand, in some cases it might be useful to validate the outcome of several control groups obtained from different trials simultaneously.

Similarly to Menssen and Schaarschmidt 2019, it is assumed that the data is overdispersed binomial. Hence, we will use the quasi_bin_pi() function in the following two examples.

Evaluation of one future control group

In this scenario, it is of interest to validate the control group of an actual (or future) study that is comprised of 30 mice instead of 50 mice as in the historical data. For this purpose a single prediction interval for one future observation is computed.

# load predint
library(predint)

# Data set 
# see Table 1 of the supplementary material of Menssen and Schaarschmidt 2019
dat_real <- data.frame("dead"=c(15, 10, 12, 12, 13, 11, 19, 11, 14, 21),
                       "alive"=c(35, 40, 38, 38, 37, 39, 31, 39, 36, 29))

# PI for one future control group comprised of 30 mice
pi_m1 <- quasi_bin_pi(histdat=dat_real, 
                      newsize=30,
                      traceplot = FALSE, 
                      alpha=0.05)
pi_m1
#>   total hist_prob quant_calib pred_se    lower    upper
#> 1    30     0.276    1.034365     5.6 2.487555 14.07245

The historical binomial probability of success (historical mortality rate) is 0.276, the bootstrap calibrated coefficient is 1.03437 and the standard error of the prediction is 5.6. The lower limit of the bootstrap calibrated asymptotic prediction interval is 2.48755 and its upper limit is given by 14.07245.

If the mortality is lower than 2.48755 it can be treated as unusual low. Consequently, mean comparisons between the control and the treatment groups might result in too many differences that are considered as significant and the compound of interest might be treated as more hazardous than it actually is.

On the other hand, the compound of interest might be treated as less hazardous if the mortality in the untreated control group is unusual high. This might be the case, if its mortality exceeds 14.07245.

Evaluation of several control groups

If a prediction interval for several future observations (in this case several control groups from several trails) is needed, their group sizes can be defined by newsize, eg. like newsize=c(50, 30, 30, 30).

pi_m4 <- quasi_bin_pi(histdat=dat_real,
                      newsize=c(50, 30, 30, 30), 
                      traceplot = FALSE,
                      alpha=0.05)
pi_m4
#>   total hist_prob quant_calib  pred_se    lower    upper
#> 1    50     0.276    1.288018 8.854377 2.395406 25.20459
#> 2    30     0.276    1.288018 5.600000 1.067102 15.49290
#> 3    30     0.276    1.288018 5.600000 1.067102 15.49290
#> 4    30     0.276    1.288018 5.600000 1.067102 15.49290

In this case, the untreated control group that contains 50 animals is in line with the historical control data if its mortality falls between 2.39541 and 25.20459. Similarly, the control groups that contain 30 animals are in line with the historical knowledge if their mortality ranges between 1.0671 and 15.4929.

References

Menssen, M., Schaarschmidt, F.: Prediction intervals for all of M future observations based on linear random effects models. Statistica Neerlandica. 2021. DOI: 10.1111/stan.12260

Menssen M, Schaarschmidt F.: Prediction intervals for overdispersed binomial data with application to historical controls. Statistics in Medicine. 2019;38:2652-2663. DOI:10.1002/sim.8124

NTP 2017: Tables of historical controls: pathology tables by route/vehicle., Accessed May 17, 2017.