Power with simglm

2017-07-24

Power Analysis with simglm

The simglm package allows the ability to conduct a power analysis through simulation. This will be particularly helpful with multilevel models and generalized linear models. To show the process, we will start with basic regression models.

Single Level Power Analysis

Let’s look at a simple single level regression example to get started:

fixed <- ~ 1 + act + diff + numCourse + act:numCourse
fixed_param <- c(0.5, 1.1, 0.6, 0.9, 1.1)
cov_param <- list(dist_fun = c('rnorm', 'rnorm', 'rnorm'),
                  var_type = c("single", "single", "single"),
                  opts = list(list(mean = 0, sd = 2),
                              list(mean = 0, sd = 2),
                              list(mean = 0, sd = 1)))
n <- 150
error_var <- 20
with_err_gen <- 'rnorm'
pow_param <- c('(Intercept)', 'act', 'diff', 'numCourse')
alpha <- .01
pow_dist <- "t"
pow_tail <- 2
replicates <- 100
power_out <- sim_pow(fixed = fixed, fixed_param = fixed_param, 
                     cov_param = cov_param, n = n, error_var = error_var,
                     with_err_gen = with_err_gen, data_str = "single",
                     pow_param = pow_param, alpha = alpha,
                     pow_dist = pow_dist, pow_tail = pow_tail, 
                     replicates = replicates)

Much of the output here is the same from the sim_reg function. The additional arguments, pow_param represents the terms to conduct a power analysis for and must be a subset of the fixed argument, alpha represents the per term level of significance, pow_dist represents the sampling distribution to refer to, either ‘z’ or ‘t’, pow_tail represents whether a one or two tailed hypothesis is being tested, and replicates represents the number of simulations to conduct. Note, to do a power analysis for the intercept, ‘(Intercept)’ must be used. By default, if pow_param is not specified power is conducted for all terms.

Finally, looking at the output from the above call:

power_out
var avg_test_stat sd_test_stat power num_reject num_repl data
(Intercept) 1.392625 0.8884058 0.10 10 100 1.000000e+00, 2.000000e+00, 3.000000e+00, 4.000000e+00, 5.000000e+00, 6.000000e+00, 7.000000e+00, 8.000000e+00, 9.000000e+00, 1.000000e+01, 1.100000e+01, 1.200000e+01, 1.300000e+01, 1.400000e+01, 1.500000e+01, 1.600000e+01, 1.700000e+01, 1.800000e+01, 1.900000e+01, 2.000000e+01, 2.100000e+01, 2.200000e+01, 2.300000e+01, 2.400000e+01, 2.500000e+01, 2.600000e+01, 2.700000e+01, 2.800000e+01, 2.900000e+01, 3.000000e+01, 3.100000e+01, 3.200000e+01, 3.300000e+01, 3.400000e+01, 3.500000e+01, 3.600000e+01, 3.700000e+01, 3.800000e+01, 3.900000e+01, 4.000000e+01, 4.100000e+01, 4.200000e+01, 4.300000e+01, 4.400000e+01, 4.500000e+01, 4.600000e+01, 4.700000e+01, 4.800000e+01, 4.900000e+01, 5.000000e+01, 5.100000e+01, 5.200000e+01, 5.300000e+01, 5.400000e+01, 5.500000e+01, 5.600000e+01, 5.700000e+01, 5.800000e+01, 5.900000e+01, 6.000000e+01, 6.100000e+01, 6.200000e+01, 6.300000e+01, 6.400000e+01, 6.500000e+01, 6.600000e+01, 6.700000e+01, 6.800000e+01, 6.900000e+01, 7.000000e+01, 7.100000e+01, 7.200000e+01, 7.300000e+01, 7.400000e+01, 7.500000e+01, 7.600000e+01, 7.700000e+01, 7.800000e+01, 7.900000e+01, 8.000000e+01, 8.100000e+01, 8.200000e+01, 8.300000e+01, 8.400000e+01, 8.500000e+01, 8.600000e+01, 8.700000e+01, 8.800000e+01, 8.900000e+01, 9.000000e+01, 9.100000e+01, 9.200000e+01, 9.300000e+01, 9.400000e+01, 9.500000e+01, 9.600000e+01, 9.700000e+01, 9.800000e+01, 9.900000e+01, 1.000000e+02, 1.231631e-01, 1.015185e+00, 3.335872e+00, 2.734283e+00, 1.492696e+00, 2.411522e+00, 3.111529e+00, 2.077953e-01, 1.219154e+00, 4.670513e-01, 8.708074e-01, 2.716086e+00, 1.093651e-01, 2.136001e+00, 7.255452e-03, 4.228096e-01, 1.121071e+00, 4.693424e-01, 8.737528e-01, 1.871385e+00, 5.905905e-01, 6.861932e-01, 1.329521e+00, 1.555060e+00, 8.634885e-03, 2.633475e+00, 1.835047e+00, 7.508085e-01, 2.464074e+00, 1.633203e+00, 1.053962e-01, 2.618734e+00, 4.625169e-01, 2.427289e+00, 1.671426e+00, 1.547343e+00, 6.252814e-01, 1.437986e+00, 2.021563e+00, 6.658394e-01, 1.372006e+00, 5.752112e-02, 2.902603e-01, 1.819901e+00, 1.971435e+00, 7.231813e-01, 2.552168e+00, 1.003888e-01, 1.027278e+00, 7.212163e-01, 1.309004e+00, 1.058530e+00, 2.662172e+00, 1.787592e+00, 3.102849e+00, 1.619165e+00, 3.115036e-01, 1.979270e+00, 1.907173e+00, 1.262164e+00, 4.269494e-01, 1.811926e+00, 1.460034e+00, 1.454865e+00, 2.281602e+00, 1.341986e+00, 1.582841e-01, 2.928833e-01, 9.375961e-01, 1.831595e+00, 9.413729e-01, 2.409916e+00, 3.233469e+00, 1.776826e+00, 3.975173e+00, 1.066847e+00, 1.009276e+00, 1.933361e+00, 8.854488e-01, 2.211008e+00, 9.665321e-01, 1.570193e-01, 1.449125e-02, 1.141851e+00, 2.116078e+00, 1.221558e+00, 2.067840e+00, 1.768451e+00, 1.514531e+00, 7.446923e-01, 2.290444e+00, 1.647897e+00, 1.940440e+00, 2.396968e+00, 1.365358e+00, 1.793979e-02, 1.117442e+00, 1.893571e+00, 1.611923e+00, 4.061993e-01, 0.000000e+00, 0.000000e+00, 1.000000e+00, 1.000000e+00, 0.000000e+00, 0.000000e+00, 1.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 1.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 1.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 1.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 1.000000e+00, 0.000000e+00, 1.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 1.000000e+00, 0.000000e+00, 1.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00
act 5.840202 1.2581223 1.00 100 100 1.000000, 2.000000, 3.000000, 4.000000, 5.000000, 6.000000, 7.000000, 8.000000, 9.000000, 10.000000, 11.000000, 12.000000, 13.000000, 14.000000, 15.000000, 16.000000, 17.000000, 18.000000, 19.000000, 20.000000, 21.000000, 22.000000, 23.000000, 24.000000, 25.000000, 26.000000, 27.000000, 28.000000, 29.000000, 30.000000, 31.000000, 32.000000, 33.000000, 34.000000, 35.000000, 36.000000, 37.000000, 38.000000, 39.000000, 40.000000, 41.000000, 42.000000, 43.000000, 44.000000, 45.000000, 46.000000, 47.000000, 48.000000, 49.000000, 50.000000, 51.000000, 52.000000, 53.000000, 54.000000, 55.000000, 56.000000, 57.000000, 58.000000, 59.000000, 60.000000, 61.000000, 62.000000, 63.000000, 64.000000, 65.000000, 66.000000, 67.000000, 68.000000, 69.000000, 70.000000, 71.000000, 72.000000, 73.000000, 74.000000, 75.000000, 76.000000, 77.000000, 78.000000, 79.000000, 80.000000, 81.000000, 82.000000, 83.000000, 84.000000, 85.000000, 86.000000, 87.000000, 88.000000, 89.000000, 90.000000, 91.000000, 92.000000, 93.000000, 94.000000, 95.000000, 96.000000, 97.000000, 98.000000, 99.000000, 100.000000, 5.947670, 6.159428, 6.147245, 6.889259, 6.363673, 6.401857, 4.421749, 6.926521, 6.508154, 4.378296, 5.754215, 5.270337, 4.648253, 4.027228, 7.430860, 7.251967, 6.509676, 7.189328, 2.885124, 4.495954, 3.333572, 5.084268, 7.006596, 7.001379, 4.135902, 6.869233, 5.239403, 4.619238, 6.143228, 5.079264, 6.491153, 6.198664, 3.083975, 5.679510, 7.275143, 4.608002, 4.392325, 5.537767, 5.363446, 9.256853, 7.210669, 5.840080, 6.429128, 6.090944, 4.918382, 7.552769, 7.819727, 3.077540, 4.564615, 5.453359, 5.217217, 7.321558, 4.589195, 4.384718, 5.752689, 5.493638, 7.181384, 6.332733, 6.265324, 5.496577, 6.559263, 4.017908, 4.726331, 7.290584, 5.669358, 4.452695, 5.140460, 6.288419, 6.226798, 4.754369, 6.461967, 5.420548, 6.787143, 3.594282, 5.103062, 4.527902, 9.098629, 7.298148, 6.841422, 6.981316, 7.338670, 7.090082, 7.673588, 7.763870, 5.025206, 6.477789, 5.590632, 5.567587, 6.899849, 6.172093, 7.199162, 6.872905, 5.514317, 5.060642, 5.961003, 4.439611, 5.949906, 5.763915, 5.011938, 4.408860, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000
diff 3.325170 1.1004735 0.77 77 100 1.000000, 2.000000, 3.000000, 4.000000, 5.000000, 6.000000, 7.000000, 8.000000, 9.000000, 10.000000, 11.000000, 12.000000, 13.000000, 14.000000, 15.000000, 16.000000, 17.000000, 18.000000, 19.000000, 20.000000, 21.000000, 22.000000, 23.000000, 24.000000, 25.000000, 26.000000, 27.000000, 28.000000, 29.000000, 30.000000, 31.000000, 32.000000, 33.000000, 34.000000, 35.000000, 36.000000, 37.000000, 38.000000, 39.000000, 40.000000, 41.000000, 42.000000, 43.000000, 44.000000, 45.000000, 46.000000, 47.000000, 48.000000, 49.000000, 50.000000, 51.000000, 52.000000, 53.000000, 54.000000, 55.000000, 56.000000, 57.000000, 58.000000, 59.000000, 60.000000, 61.000000, 62.000000, 63.000000, 64.000000, 65.000000, 66.000000, 67.000000, 68.000000, 69.000000, 70.000000, 71.000000, 72.000000, 73.000000, 74.000000, 75.000000, 76.000000, 77.000000, 78.000000, 79.000000, 80.000000, 81.000000, 82.000000, 83.000000, 84.000000, 85.000000, 86.000000, 87.000000, 88.000000, 89.000000, 90.000000, 91.000000, 92.000000, 93.000000, 94.000000, 95.000000, 96.000000, 97.000000, 98.000000, 99.000000, 100.000000, 3.942008, 3.727646, 3.924645, 2.222407, 2.653269, 2.885995, 3.631742, 3.965794, 4.478406, 2.639620, 3.604584, 3.840768, 3.239922, 3.355853, 2.971578, 6.393224, 3.237566, 6.583254, 4.464333, 2.924558, 2.980149, 2.413479, 1.433822, 3.871813, 4.209652, 3.897456, 2.960736, 2.964873, 2.179200, 1.090522, 3.130264, 3.232035, 3.058378, 2.534010, 4.754778, 2.635493, 2.239310, 3.750200, 2.552899, 4.565168, 3.904809, 1.062074, 2.946547, 1.982177, 3.476161, 2.041504, 5.236145, 3.051297, 2.751387, 3.246099, 4.933680, 3.484648, 3.495016, 4.177142, 3.403595, 2.317536, 3.316226, 2.982870, 2.417219, 1.694381, 2.845460, 2.842751, 4.794759, 3.114840, 3.159012, 3.280620, 2.700518, 1.932019, 2.912567, 1.202180, 4.613861, 2.181990, 4.762867, 3.105339, 5.893031, 4.490667, 4.479736, 3.606179, 3.932688, 4.007335, 3.130955, 3.043311, 1.182578, 2.763721, 2.558110, 2.170958, 4.383377, 4.119811, 4.283352, 4.056048, 2.379595, 2.917338, 1.465063, 2.292975, 3.515337, 3.418043, 6.148337, 5.254182, 3.272870, 3.240700, 1.000000, 1.000000, 1.000000, 0.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 0.000000, 0.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 0.000000, 0.000000, 1.000000, 1.000000, 1.000000, 0.000000, 1.000000, 1.000000, 0.000000, 1.000000, 0.000000, 1.000000, 1.000000, 0.000000, 1.000000, 0.000000, 1.000000, 0.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 0.000000, 1.000000, 1.000000, 0.000000, 0.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 0.000000, 1.000000, 0.000000, 1.000000, 0.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 0.000000, 1.000000, 0.000000, 0.000000, 1.000000, 1.000000, 1.000000, 1.000000, 0.000000, 1.000000, 0.000000, 0.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000
numCourse 2.208270 0.9915482 0.34 34 100 1.00000000, 2.00000000, 3.00000000, 4.00000000, 5.00000000, 6.00000000, 7.00000000, 8.00000000, 9.00000000, 10.00000000, 11.00000000, 12.00000000, 13.00000000, 14.00000000, 15.00000000, 16.00000000, 17.00000000, 18.00000000, 19.00000000, 20.00000000, 21.00000000, 22.00000000, 23.00000000, 24.00000000, 25.00000000, 26.00000000, 27.00000000, 28.00000000, 29.00000000, 30.00000000, 31.00000000, 32.00000000, 33.00000000, 34.00000000, 35.00000000, 36.00000000, 37.00000000, 38.00000000, 39.00000000, 40.00000000, 41.00000000, 42.00000000, 43.00000000, 44.00000000, 45.00000000, 46.00000000, 47.00000000, 48.00000000, 49.00000000, 50.00000000, 51.00000000, 52.00000000, 53.00000000, 54.00000000, 55.00000000, 56.00000000, 57.00000000, 58.00000000, 59.00000000, 60.00000000, 61.00000000, 62.00000000, 63.00000000, 64.00000000, 65.00000000, 66.00000000, 67.00000000, 68.00000000, 69.00000000, 70.00000000, 71.00000000, 72.00000000, 73.00000000, 74.00000000, 75.00000000, 76.00000000, 77.00000000, 78.00000000, 79.00000000, 80.00000000, 81.00000000, 82.00000000, 83.00000000, 84.00000000, 85.00000000, 86.00000000, 87.00000000, 88.00000000, 89.00000000, 90.00000000, 91.00000000, 92.00000000, 93.00000000, 94.00000000, 95.00000000, 96.00000000, 97.00000000, 98.00000000, 99.00000000, 100.00000000, 2.67019939, 2.10147964, 2.98342123, 1.63557084, 2.89006840, 2.82770689, 3.54505821, 1.60504625, 0.94727657, 1.45469079, 2.13195529, 2.70399890, 2.78450066, 3.20475928, 2.88975335, 2.91710027, 1.84417910, 1.71843919, 4.21658424, 2.24528056, 0.02297102, 0.61897207, 1.72674944, 2.80074194, 1.30586599, 1.28721648, 2.38149716, 1.63783667, 3.34391175, 1.44562014, 0.88034450, 1.59932883, 1.65269000, 1.70226192, 3.11209157, 2.64051030, 3.27059448, 1.54052650, 2.06836957, 3.92199175, 0.41992288, 1.15103717, 3.32219317, 0.22536273, 2.95572747, 2.34908000, 1.28698531, 2.51397523, 1.50418234, 2.45661057, 2.19200134, 3.40133857, 0.25516321, 0.20464148, 1.80476070, 2.19769165, 2.80947482, 2.21611379, 1.75309921, 2.08552966, 2.02336735, 2.48336067, 3.45844512, 1.98296720, 2.59030870, 1.83958085, 2.39386717, 0.85839572, 1.62940990, 2.82202660, 3.80636470, 4.62508318, 2.53322454, 3.08787518, 2.76233719, 2.00774499, 1.31794178, 2.88099089, 2.46064047, 2.60400791, 2.44033504, 4.00218073, 0.58246601, 0.05900353, 4.37760192, 2.20997025, 2.98591618, 2.75455458, 1.92040591, 2.60936347, 2.35295697, 2.46709285, 1.78513469, 0.90368917, 2.02200261, 3.23238644, 4.33302076, 2.22099928, 0.95241867, 1.06550681, 1.00000000, 0.00000000, 1.00000000, 0.00000000, 1.00000000, 1.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 1.00000000, 1.00000000, 1.00000000, 1.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 1.00000000, 1.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 1.00000000, 1.00000000, 0.00000000, 1.00000000, 1.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 1.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000

The output contains the variable name, the average test statistic, the standard deviation of the test statistic, the power rate, the number of null hypotheses rejects, and the total number of replications. Increasing the number of replications would increase the precision of the power analysis, however may significantly increase the computational time.

Standardized Coefficients

By default, the simglm package uses unstandardized regression coefficients when doing the simulation. A way to use standardized coefficients however, would be to generate standardized variables. For example:

fixed <- ~ 1 + act + diff + numCourse + act:numCourse
fixed_param <- c(0.2, 0.4, 0.25, 0.7, 0.1)
cov_param <- list(dist_fun = c('rnorm', 'rnorm', 'rnorm'),
                  var_type = c("single", "single", "single"),
                  opts = list(list(mean = 0, sd = 1),
                              list(mean = 0, sd = 1),
                              list(mean = 0, sd = 1)))
n <- 150
error_var <- 1
with_err_gen <- 'rnorm'
pow_param <- c('(Intercept)', 'act', 'diff', 'numCourse')
alpha <- .01
pow_dist <- "t"
pow_tail <- 2
replicates <- 100
power_out <- sim_pow(fixed = fixed, fixed_param = fixed_param, 
                     cov_param = cov_param,n = n, error_var = error_var,
                     with_err_gen = with_err_gen, data_str = "single",
                     pow_param = pow_param, alpha = alpha,
                     pow_dist = pow_dist, pow_tail = pow_tail, 
                     replicates = replicates)
power_out
var avg_test_stat sd_test_stat power num_reject num_repl data
(Intercept) 2.486385 0.9087604 0.43 43 100 1.0000000, 2.0000000, 3.0000000, 4.0000000, 5.0000000, 6.0000000, 7.0000000, 8.0000000, 9.0000000, 10.0000000, 11.0000000, 12.0000000, 13.0000000, 14.0000000, 15.0000000, 16.0000000, 17.0000000, 18.0000000, 19.0000000, 20.0000000, 21.0000000, 22.0000000, 23.0000000, 24.0000000, 25.0000000, 26.0000000, 27.0000000, 28.0000000, 29.0000000, 30.0000000, 31.0000000, 32.0000000, 33.0000000, 34.0000000, 35.0000000, 36.0000000, 37.0000000, 38.0000000, 39.0000000, 40.0000000, 41.0000000, 42.0000000, 43.0000000, 44.0000000, 45.0000000, 46.0000000, 47.0000000, 48.0000000, 49.0000000, 50.0000000, 51.0000000, 52.0000000, 53.0000000, 54.0000000, 55.0000000, 56.0000000, 57.0000000, 58.0000000, 59.0000000, 60.0000000, 61.0000000, 62.0000000, 63.0000000, 64.0000000, 65.0000000, 66.0000000, 67.0000000, 68.0000000, 69.0000000, 70.0000000, 71.0000000, 72.0000000, 73.0000000, 74.0000000, 75.0000000, 76.0000000, 77.0000000, 78.0000000, 79.0000000, 80.0000000, 81.0000000, 82.0000000, 83.0000000, 84.0000000, 85.0000000, 86.0000000, 87.0000000, 88.0000000, 89.0000000, 90.0000000, 91.0000000, 92.0000000, 93.0000000, 94.0000000, 95.0000000, 96.0000000, 97.0000000, 98.0000000, 99.0000000, 100.0000000, 2.6819618, 1.3567156, 4.4298199, 0.9364108, 2.1471202, 4.1628945, 2.6710670, 1.7040291, 1.8015147, 2.2437295, 3.3020786, 2.5292257, 2.1196800, 1.2044445, 4.6105485, 1.4725588, 3.1168138, 1.3648670, 1.9260656, 1.1244986, 1.8086940, 3.0578521, 3.1856662, 3.8880459, 3.0115170, 4.8308792, 3.6298306, 3.9766337, 1.1327946, 1.9932712, 2.8109390, 2.1186935, 1.4895015, 2.5115555, 3.0150342, 1.8915021, 2.7030455, 3.7309223, 1.5968297, 3.1650678, 1.3581889, 1.9142930, 1.9690313, 2.6220169, 2.8878743, 2.1524023, 2.5253425, 2.1831766, 2.1047288, 1.4323191, 0.9017149, 2.3702695, 2.2327198, 3.1811978, 3.0599543, 2.2252463, 3.2005295, 3.2231291, 2.7553822, 3.2217707, 2.4989381, 2.1395491, 2.2125796, 1.2119741, 3.5953247, 3.1746922, 2.1358578, 2.6984015, 2.4448708, 1.4164882, 2.5567804, 1.5310228, 1.5054759, 1.8453238, 1.5722326, 2.5386026, 2.3502492, 1.2401953, 1.1074954, 3.7934449, 2.8452212, 2.1635871, 3.7170051, 0.8058752, 3.0177337, 3.6571529, 3.5962611, 2.4454078, 2.7230056, 2.5914453, 2.1391067, 0.9647296, 3.0276781, 1.9255897, 2.8342919, 4.7324269, 2.5432621, 2.8467653, 2.9639976, 3.5528085, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 0.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 0.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 1.0000000
act 4.757052 1.2075950 0.95 95 100 1.000000, 2.000000, 3.000000, 4.000000, 5.000000, 6.000000, 7.000000, 8.000000, 9.000000, 10.000000, 11.000000, 12.000000, 13.000000, 14.000000, 15.000000, 16.000000, 17.000000, 18.000000, 19.000000, 20.000000, 21.000000, 22.000000, 23.000000, 24.000000, 25.000000, 26.000000, 27.000000, 28.000000, 29.000000, 30.000000, 31.000000, 32.000000, 33.000000, 34.000000, 35.000000, 36.000000, 37.000000, 38.000000, 39.000000, 40.000000, 41.000000, 42.000000, 43.000000, 44.000000, 45.000000, 46.000000, 47.000000, 48.000000, 49.000000, 50.000000, 51.000000, 52.000000, 53.000000, 54.000000, 55.000000, 56.000000, 57.000000, 58.000000, 59.000000, 60.000000, 61.000000, 62.000000, 63.000000, 64.000000, 65.000000, 66.000000, 67.000000, 68.000000, 69.000000, 70.000000, 71.000000, 72.000000, 73.000000, 74.000000, 75.000000, 76.000000, 77.000000, 78.000000, 79.000000, 80.000000, 81.000000, 82.000000, 83.000000, 84.000000, 85.000000, 86.000000, 87.000000, 88.000000, 89.000000, 90.000000, 91.000000, 92.000000, 93.000000, 94.000000, 95.000000, 96.000000, 97.000000, 98.000000, 99.000000, 100.000000, 5.932645, 5.300486, 3.644256, 3.447407, 5.732709, 5.694157, 6.044051, 6.132008, 4.629185, 4.074080, 6.100453, 5.530253, 2.564724, 3.657076, 4.279095, 4.421487, 2.876103, 5.311908, 5.621521, 3.681829, 4.903637, 2.068218, 5.996526, 4.098367, 3.647612, 5.340600, 5.128651, 7.239580, 4.143861, 6.241739, 5.922549, 2.135472, 5.087640, 3.322292, 4.127872, 5.764332, 3.561432, 2.435970, 3.703256, 6.817298, 4.055868, 4.528201, 4.384087, 3.613490, 3.172354, 4.508970, 4.272974, 4.481108, 3.789640, 4.648814, 4.563944, 6.395522, 6.664750, 6.990570, 5.539910, 4.508656, 7.056396, 4.654328, 5.542578, 4.481510, 5.277691, 2.887833, 5.618564, 5.958218, 4.536403, 5.103463, 4.915224, 4.455143, 6.974375, 5.258990, 4.429898, 3.054716, 4.508361, 4.124335, 6.160977, 6.006881, 3.321218, 6.666894, 5.212040, 5.316527, 5.540850, 6.934193, 4.257560, 5.436849, 5.940302, 5.624838, 4.591772, 5.267102, 5.294592, 4.529309, 3.732313, 2.746848, 3.369945, 5.498842, 3.881298, 3.252968, 2.485450, 5.632449, 3.523733, 4.160202, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 0.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 0.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 0.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 0.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 0.000000, 1.000000, 1.000000, 1.000000
diff 3.015957 1.0263456 0.66 66 100 1.0000000, 2.0000000, 3.0000000, 4.0000000, 5.0000000, 6.0000000, 7.0000000, 8.0000000, 9.0000000, 10.0000000, 11.0000000, 12.0000000, 13.0000000, 14.0000000, 15.0000000, 16.0000000, 17.0000000, 18.0000000, 19.0000000, 20.0000000, 21.0000000, 22.0000000, 23.0000000, 24.0000000, 25.0000000, 26.0000000, 27.0000000, 28.0000000, 29.0000000, 30.0000000, 31.0000000, 32.0000000, 33.0000000, 34.0000000, 35.0000000, 36.0000000, 37.0000000, 38.0000000, 39.0000000, 40.0000000, 41.0000000, 42.0000000, 43.0000000, 44.0000000, 45.0000000, 46.0000000, 47.0000000, 48.0000000, 49.0000000, 50.0000000, 51.0000000, 52.0000000, 53.0000000, 54.0000000, 55.0000000, 56.0000000, 57.0000000, 58.0000000, 59.0000000, 60.0000000, 61.0000000, 62.0000000, 63.0000000, 64.0000000, 65.0000000, 66.0000000, 67.0000000, 68.0000000, 69.0000000, 70.0000000, 71.0000000, 72.0000000, 73.0000000, 74.0000000, 75.0000000, 76.0000000, 77.0000000, 78.0000000, 79.0000000, 80.0000000, 81.0000000, 82.0000000, 83.0000000, 84.0000000, 85.0000000, 86.0000000, 87.0000000, 88.0000000, 89.0000000, 90.0000000, 91.0000000, 92.0000000, 93.0000000, 94.0000000, 95.0000000, 96.0000000, 97.0000000, 98.0000000, 99.0000000, 100.0000000, 2.5547534, 3.4657252, 1.1027616, 3.9451339, 3.8797226, 1.5274803, 2.8699772, 0.1540064, 3.6986368, 3.3893953, 2.9839064, 3.9044215, 3.0161798, 3.9985390, 3.9926078, 0.9718192, 4.5085791, 3.6063807, 1.6385006, 3.9980299, 1.2487897, 3.7674967, 3.9592223, 2.7620160, 3.3104240, 2.0826640, 4.7631833, 4.1588486, 3.8146896, 4.0944881, 2.3176759, 3.9958452, 2.2712010, 3.3461618, 3.2854423, 2.5672952, 3.1628674, 2.2868210, 3.4386533, 5.1066331, 2.3780834, 2.2205196, 3.5330775, 2.1002840, 4.2179786, 1.6584703, 4.1573810, 2.0541991, 2.4868308, 3.0113331, 3.0353839, 3.7609687, 4.4356831, 4.2638435, 2.5351491, 3.6144575, 2.7919604, 2.4405783, 1.8665532, 1.8134434, 2.6166064, 3.7426709, 1.7388730, 1.2792187, 3.9157308, 0.1169043, 4.8821753, 4.8240221, 2.9998797, 2.2862787, 2.6735922, 3.7099683, 4.5629935, 2.1477693, 2.7781436, 1.2764934, 1.8417749, 3.0386543, 3.1966966, 2.5351076, 3.2218186, 2.8313236, 3.5334079, 3.2204695, 3.1514998, 3.1239576, 3.8842488, 2.7135266, 2.4042300, 4.3517336, 4.4731265, 3.2894377, 1.4334149, 3.7328088, 2.2781470, 2.9374507, 2.7127734, 2.5426915, 3.5541189, 2.7187755, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 0.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000
numCourse 8.197547 1.2958690 1.00 100 100 1.000000, 2.000000, 3.000000, 4.000000, 5.000000, 6.000000, 7.000000, 8.000000, 9.000000, 10.000000, 11.000000, 12.000000, 13.000000, 14.000000, 15.000000, 16.000000, 17.000000, 18.000000, 19.000000, 20.000000, 21.000000, 22.000000, 23.000000, 24.000000, 25.000000, 26.000000, 27.000000, 28.000000, 29.000000, 30.000000, 31.000000, 32.000000, 33.000000, 34.000000, 35.000000, 36.000000, 37.000000, 38.000000, 39.000000, 40.000000, 41.000000, 42.000000, 43.000000, 44.000000, 45.000000, 46.000000, 47.000000, 48.000000, 49.000000, 50.000000, 51.000000, 52.000000, 53.000000, 54.000000, 55.000000, 56.000000, 57.000000, 58.000000, 59.000000, 60.000000, 61.000000, 62.000000, 63.000000, 64.000000, 65.000000, 66.000000, 67.000000, 68.000000, 69.000000, 70.000000, 71.000000, 72.000000, 73.000000, 74.000000, 75.000000, 76.000000, 77.000000, 78.000000, 79.000000, 80.000000, 81.000000, 82.000000, 83.000000, 84.000000, 85.000000, 86.000000, 87.000000, 88.000000, 89.000000, 90.000000, 91.000000, 92.000000, 93.000000, 94.000000, 95.000000, 96.000000, 97.000000, 98.000000, 99.000000, 100.000000, 9.046935, 6.937249, 8.455761, 8.588930, 8.105211, 7.799362, 6.505621, 6.677047, 8.315007, 9.109716, 7.080121, 7.784049, 8.233323, 6.585757, 8.262034, 11.230813, 7.463113, 7.552839, 6.330806, 7.516805, 11.743456, 10.440063, 10.227436, 8.823039, 6.765888, 9.175599, 8.658078, 8.608331, 6.729597, 9.046263, 10.391565, 8.831172, 8.044310, 7.314063, 7.199986, 6.481523, 9.628677, 7.092911, 9.482460, 7.946784, 8.635501, 8.305891, 6.959263, 9.047011, 8.519898, 6.768883, 8.480345, 7.419703, 7.709503, 6.351193, 8.046994, 9.606824, 7.209925, 6.333270, 6.542661, 8.346570, 6.360060, 7.572347, 8.204034, 5.795366, 8.699420, 7.247256, 9.361192, 9.355025, 7.235052, 8.775566, 7.024227, 8.854206, 8.559113, 6.501599, 7.228218, 8.639976, 10.116918, 8.071644, 6.712043, 6.453267, 6.686131, 7.926526, 8.395255, 9.277677, 8.869460, 8.845709, 9.678938, 7.516587, 9.170201, 8.089476, 7.867614, 7.834399, 8.296610, 9.109583, 8.815661, 13.510831, 7.122821, 8.591774, 8.923541, 8.644041, 7.488864, 9.184330, 7.808675, 10.838328, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000

Varying Arguments

fixed <- ~ 1 + act + diff + numCourse + act:numCourse
fixed_param <- c(0.5, 1.1, 0.6, 0.9, 1.1)
cov_param <- list(dist_fun = c('rnorm', 'rnorm', 'rnorm'), 
                  var_type = c("single", "single", "single"),
                  opts = list(list(mean = 0, sd = 2),
                              list(mean = 0, sd = 2),
                              list(mean = 0, sd = 1)))
n <- NULL
error_var <- NULL
with_err_gen <- 'rnorm'
pow_param <- c('(Intercept)', 'act', 'diff', 'numCourse')
alpha <- .01
pow_dist <- "t"
pow_tail <- 2
replicates <- 10
terms_vary <- list(n = c(20, 40, 60, 80, 100), error_var = c(5, 10, 20),
                   fixed_param = list(c(0.5, 1.1, 0.6, 0.9, 1.1), 
                                      c(0.6, 1.1, 0.6, 0.9, 1.1)),
                cov_param = list(list(dist_fun = c('rnorm', 'rnorm', 'rnorm'),
                                       mean = c(0, 0, 0), sd = c(2, 2, 1), 
                                  var_type = c("single", "single", "single")),
                                  list(dist_fun = c('rnorm', 'rnorm', 'rnorm'),
                                       mean = c(0.5, 0, 0), sd = c(2, 2, 1), 
                                  var_type = c("single", "single", "single"))
                                  )
                   )
power_out <- sim_pow(fixed = fixed, fixed_param = fixed_param, 
                     cov_param = cov_param,
                     n = n, error_var = error_var, with_err_gen = with_err_gen, 
                     data_str = "single", pow_param = pow_param, alpha = alpha,
                     pow_dist = pow_dist, pow_tail = pow_tail, 
                     replicates = replicates, terms_vary = terms_vary)

Model Misspecification

It is also possible to specify a model different than the generating model for evaluation of power. This may be useful if it is thought another variable is important in explaining variation, however due to design issues is not possible to collect this variable. As a result, there would likely be additional variation due to this variable that will not be able to be explained. Building this into the generating model can help provide additional information about the impact on power.

An example using single level power analysis is shown below.

fixed <- ~ 1 + act + diff + numCourse + act:numCourse
fixed_param <- c(0.5, 1.1, 0.6, 0.9, 1.1)
cov_param <- list(dist_fun = c('rnorm', 'rnorm', 'rnorm'),
                  var_type = c("single", "single", "single"),
                  opts = list(list(mean = 0, sd = 2),
                              list(mean = 0, sd = 2),
                              list(mean = 0, sd = 1)))
n <- 150
error_var <- 20
with_err_gen <- 'rnorm'
pow_param <- c('(Intercept)', 'act', 'diff', 'numCourse')
alpha <- .01
pow_dist <- "t"
pow_tail <- 2
replicates <- 100

lm_fit_mod <- sim_data ~ 1 + act + diff

power_out <- sim_pow(fixed = fixed, fixed_param = fixed_param, 
                     cov_param = cov_param, n = n, error_var = error_var,
                     with_err_gen = with_err_gen, data_str = "single",
                     pow_param = pow_param, alpha = alpha,
                     pow_dist = pow_dist, pow_tail = pow_tail, 
                     replicates = replicates, lm_fit_mod = lm_fit_mod)
power_out
var avg_test_stat sd_test_stat power num_reject num_repl data
(Intercept) 1.280522 0.9273867 0.09 9 100 1.00000000, 2.00000000, 3.00000000, 4.00000000, 5.00000000, 6.00000000, 7.00000000, 8.00000000, 9.00000000, 10.00000000, 11.00000000, 12.00000000, 13.00000000, 14.00000000, 15.00000000, 16.00000000, 17.00000000, 18.00000000, 19.00000000, 20.00000000, 21.00000000, 22.00000000, 23.00000000, 24.00000000, 25.00000000, 26.00000000, 27.00000000, 28.00000000, 29.00000000, 30.00000000, 31.00000000, 32.00000000, 33.00000000, 34.00000000, 35.00000000, 36.00000000, 37.00000000, 38.00000000, 39.00000000, 40.00000000, 41.00000000, 42.00000000, 43.00000000, 44.00000000, 45.00000000, 46.00000000, 47.00000000, 48.00000000, 49.00000000, 50.00000000, 51.00000000, 52.00000000, 53.00000000, 54.00000000, 55.00000000, 56.00000000, 57.00000000, 58.00000000, 59.00000000, 60.00000000, 61.00000000, 62.00000000, 63.00000000, 64.00000000, 65.00000000, 66.00000000, 67.00000000, 68.00000000, 69.00000000, 70.00000000, 71.00000000, 72.00000000, 73.00000000, 74.00000000, 75.00000000, 76.00000000, 77.00000000, 78.00000000, 79.00000000, 80.00000000, 81.00000000, 82.00000000, 83.00000000, 84.00000000, 85.00000000, 86.00000000, 87.00000000, 88.00000000, 89.00000000, 90.00000000, 91.00000000, 92.00000000, 93.00000000, 94.00000000, 95.00000000, 96.00000000, 97.00000000, 98.00000000, 99.00000000, 100.00000000, 0.40061462, 1.15601011, 0.07951147, 2.83767049, 0.01986554, 1.30063726, 1.55207324, 2.54195390, 2.54208214, 1.18439099, 0.10074614, 0.66971905, 0.34605091, 0.93738318, 1.83458962, 1.62780849, 0.59341930, 1.66487772, 0.20824381, 0.22857387, 0.16667214, 1.17044479, 1.03539934, 0.64585468, 0.22977129, 0.98639836, 1.27582508, 2.53665281, 1.33577022, 0.27397964, 1.97367172, 1.59060563, 2.34866055, 1.58304308, 2.77077819, 1.31257214, 0.55055898, 1.49849685, 1.59505926, 1.67798685, 0.43168089, 1.65229204, 2.20150911, 2.18883347, 2.11985464, 0.82051429, 1.17448260, 0.96279943, 0.98664525, 3.78867834, 0.42274640, 0.54200856, 0.58592838, 0.34079041, 1.68053540, 0.70189791, 2.90379829, 0.76226000, 0.20767444, 0.21460206, 0.50974518, 0.63804172, 1.10210348, 2.03620645, 3.35197659, 0.35348282, 0.61521402, 1.72702077, 0.22230394, 1.28242876, 1.22789793, 1.86014874, 1.22729109, 2.84874606, 0.84999951, 1.18142412, 0.02703678, 3.86412467, 0.63558524, 1.30624216, 3.46871171, 1.45867941, 1.59654904, 1.00109449, 2.37569317, 1.78418514, 0.87776279, 2.33956385, 0.06431345, 0.11732469, 0.56590128, 0.21360532, 3.53007807, 1.40498273, 1.02663088, 1.12102029, 0.14287368, 0.70710817, 1.83345198, 2.48168133, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000
act 5.222484 1.3454179 0.98 98 100 1.000000, 2.000000, 3.000000, 4.000000, 5.000000, 6.000000, 7.000000, 8.000000, 9.000000, 10.000000, 11.000000, 12.000000, 13.000000, 14.000000, 15.000000, 16.000000, 17.000000, 18.000000, 19.000000, 20.000000, 21.000000, 22.000000, 23.000000, 24.000000, 25.000000, 26.000000, 27.000000, 28.000000, 29.000000, 30.000000, 31.000000, 32.000000, 33.000000, 34.000000, 35.000000, 36.000000, 37.000000, 38.000000, 39.000000, 40.000000, 41.000000, 42.000000, 43.000000, 44.000000, 45.000000, 46.000000, 47.000000, 48.000000, 49.000000, 50.000000, 51.000000, 52.000000, 53.000000, 54.000000, 55.000000, 56.000000, 57.000000, 58.000000, 59.000000, 60.000000, 61.000000, 62.000000, 63.000000, 64.000000, 65.000000, 66.000000, 67.000000, 68.000000, 69.000000, 70.000000, 71.000000, 72.000000, 73.000000, 74.000000, 75.000000, 76.000000, 77.000000, 78.000000, 79.000000, 80.000000, 81.000000, 82.000000, 83.000000, 84.000000, 85.000000, 86.000000, 87.000000, 88.000000, 89.000000, 90.000000, 91.000000, 92.000000, 93.000000, 94.000000, 95.000000, 96.000000, 97.000000, 98.000000, 99.000000, 100.000000, 7.897136, 5.066868, 6.343637, 5.137605, 5.945589, 3.783660, 3.974044, 5.714289, 5.329016, 5.893684, 3.807184, 5.034628, 6.266082, 4.266946, 4.596525, 4.075440, 4.500922, 5.198499, 4.114101, 6.086907, 5.465194, 5.932322, 2.787939, 6.610407, 5.045905, 4.034730, 3.214774, 6.519324, 4.746928, 2.944255, 4.515169, 7.185187, 3.767033, 6.691314, 5.024920, 6.169734, 5.584714, 6.085325, 4.359644, 4.482620, 5.470148, 4.807704, 6.357708, 4.068029, 4.502378, 5.098558, 4.111151, 2.973581, 4.628570, 7.598822, 7.159989, 3.049804, 5.702219, 6.210973, 6.480423, 3.828307, 5.157793, 7.591035, 6.968514, 5.673443, 7.054540, 5.774307, 6.861663, 4.730957, 5.080521, 3.367265, 4.713797, 7.850937, 3.934354, 1.993187, 6.513064, 3.937338, 5.108978, 7.142429, 3.907511, 5.049282, 4.272914, 4.923499, 3.160290, 6.358539, 7.472372, 4.979824, 4.428086, 5.299983, 5.688249, 4.868348, 7.554107, 5.017175, 5.964806, 2.256940, 5.026777, 4.048384, 5.755616, 8.398452, 7.122784, 4.651033, 5.432569, 3.043395, 6.178130, 5.686645, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 0.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 0.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000
diff 2.810335 0.8414746 0.57 57 100 1.0000000, 2.0000000, 3.0000000, 4.0000000, 5.0000000, 6.0000000, 7.0000000, 8.0000000, 9.0000000, 10.0000000, 11.0000000, 12.0000000, 13.0000000, 14.0000000, 15.0000000, 16.0000000, 17.0000000, 18.0000000, 19.0000000, 20.0000000, 21.0000000, 22.0000000, 23.0000000, 24.0000000, 25.0000000, 26.0000000, 27.0000000, 28.0000000, 29.0000000, 30.0000000, 31.0000000, 32.0000000, 33.0000000, 34.0000000, 35.0000000, 36.0000000, 37.0000000, 38.0000000, 39.0000000, 40.0000000, 41.0000000, 42.0000000, 43.0000000, 44.0000000, 45.0000000, 46.0000000, 47.0000000, 48.0000000, 49.0000000, 50.0000000, 51.0000000, 52.0000000, 53.0000000, 54.0000000, 55.0000000, 56.0000000, 57.0000000, 58.0000000, 59.0000000, 60.0000000, 61.0000000, 62.0000000, 63.0000000, 64.0000000, 65.0000000, 66.0000000, 67.0000000, 68.0000000, 69.0000000, 70.0000000, 71.0000000, 72.0000000, 73.0000000, 74.0000000, 75.0000000, 76.0000000, 77.0000000, 78.0000000, 79.0000000, 80.0000000, 81.0000000, 82.0000000, 83.0000000, 84.0000000, 85.0000000, 86.0000000, 87.0000000, 88.0000000, 89.0000000, 90.0000000, 91.0000000, 92.0000000, 93.0000000, 94.0000000, 95.0000000, 96.0000000, 97.0000000, 98.0000000, 99.0000000, 100.0000000, 3.8055255, 2.4122643, 3.4892421, 3.7250799, 3.6507544, 2.9173772, 2.9385129, 3.6427177, 2.9515993, 3.0227434, 2.2784755, 1.3608281, 3.9161685, 2.3531798, 2.5376141, 1.6025575, 3.9914539, 2.3561045, 3.7542375, 1.8418172, 1.8969971, 3.8117854, 3.0010787, 1.9729998, 3.7420293, 2.0778330, 1.9210697, 3.1182511, 1.9623310, 3.3407216, 4.0167310, 3.9690128, 3.3219117, 4.2265576, 1.7963981, 4.0546847, 3.3042188, 2.0647871, 2.6706515, 2.3384931, 2.7721642, 1.9165017, 4.2441097, 2.1516843, 2.3482831, 1.2813251, 3.0678119, 2.7024839, 1.9456785, 2.0021700, 2.2787234, 1.6447407, 2.8521665, 4.5167940, 3.6325208, 2.5596056, 3.5539022, 2.5406236, 2.9845527, 3.5813155, 2.7076241, 3.6144892, 1.8012809, 3.3878202, 5.1489430, 1.9148629, 2.9895113, 2.5889731, 2.9123213, 1.8854058, 2.8228786, 2.2741578, 2.6912977, 3.9802827, 1.6565134, 3.2719228, 0.6670528, 2.9405926, 4.5581068, 2.3408127, 2.6428122, 3.3552732, 1.9370077, 2.6534049, 1.6229114, 3.8418157, 3.0183553, 2.7008266, 3.2814582, 2.6170524, 2.3300945, 3.1895590, 1.7712503, 1.9401495, 2.3375046, 2.3864642, 4.0377411, 2.5860802, 2.0565755, 2.8383891, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 0.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 1.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 1.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 1.0000000
numCourse NA NaN NA NA 100 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA

You’ll notice, compared to above, the power is somewhat reduced for the other three fixed effects when not modeling the numCourse variable. You can also build in the lm_fit_mod argument into the design via the terms_vary argument.

fixed <- ~ 1 + act + diff + numCourse + act:numCourse
fixed_param <- c(0.5, 1.1, 0.6, 0.9, 1.1)
cov_param <- list(dist_fun = c('rnorm', 'rnorm', 'rnorm'),
                  var_type = c("single", "single", "single"),
                  opts = list(list(mean = 0, sd = 2),
                              list(mean = 0, sd = 2),
                              list(mean = 0, sd = 1)))
n <- 150
error_var <- 20
with_err_gen <- 'rnorm'
pow_param <- c('(Intercept)', 'act', 'diff', 'numCourse')
alpha <- .01
pow_dist <- "t"
pow_tail <- 2
replicates <- 100

terms_vary <- list(n = c(20, 40, 60, 80, 100), error_var = c(5, 10, 20),
                   fixed_param = list(c(0.5, 1.1, 0.6, 0.9, 1.1), 
                                      c(0.6, 1.1, 0.6, 0.9, 1.1)),
                lm_fit_mod = list(sim_data ~ 1 + act + diff, 
                                  sim_data ~ 1 + act)
)

power_out <- sim_pow(fixed = fixed, fixed_param = fixed_param, 
                     cov_param = cov_param, n = n, error_var = error_var,
                     with_err_gen = with_err_gen, data_str = "single",
                     pow_param = pow_param, alpha = alpha,
                     pow_dist = pow_dist, pow_tail = pow_tail, 
                     replicates = replicates, terms_vary = terms_vary)
power_out
## # A tibble: 240 x 11
## # Groups:   var, n, error_var, fixed_param [?]
##            var     n error_var         fixed_param
##         <fctr> <dbl>     <dbl>              <fctr>
##  1 (Intercept)    20         5 0.5,1.1,0.6,0.9,1.1
##  2 (Intercept)    20         5 0.5,1.1,0.6,0.9,1.1
##  3 (Intercept)    20         5 0.6,1.1,0.6,0.9,1.1
##  4 (Intercept)    20         5 0.6,1.1,0.6,0.9,1.1
##  5 (Intercept)    20        10 0.5,1.1,0.6,0.9,1.1
##  6 (Intercept)    20        10 0.5,1.1,0.6,0.9,1.1
##  7 (Intercept)    20        10 0.6,1.1,0.6,0.9,1.1
##  8 (Intercept)    20        10 0.6,1.1,0.6,0.9,1.1
##  9 (Intercept)    20        20 0.5,1.1,0.6,0.9,1.1
## 10 (Intercept)    20        20 0.5,1.1,0.6,0.9,1.1
## # ... with 230 more rows, and 7 more variables: lm_fit_mod <fctr>,
## #   avg_test_stat <dbl>, sd_test_stat <dbl>, power <dbl>,
## #   num_reject <dbl>, num_repl <dbl>, data <list>

Nested Data

Extending the power analysis to two level models is a straightforward addition.

fixed <- ~1 + time + diff + act + time:act
random <- ~1 + time
fixed_param <- c(0, 0.2, 0.1, 0.3, 0.05)
random_param <- list(random_var = c(7, 4), rand_gen = "rnorm")
cov_param <- list(dist_fun = c('rnorm', 'rnorm'),
                  var_type = c("level1", "level2"),
                  opts = list(list(mean = 0, sd = 1),
                              list(mean = 0, sd = 1)))
n <- 150
p <- 30
error_var <- 1
data_str <- "long"
pow_param <- c('time', 'diff', 'act')
alpha <- .01
pow_dist <- "z"
pow_tail <- 2
replicates <- 10
power_out <- sim_pow(fixed = fixed, random = random, 
                     fixed_param = fixed_param, 
                     random_param = random_param, cov_param = cov_param, 
                     k = NULL, n = n, p = p,
                     error_var = error_var, with_err_gen = "rnorm",
                     data_str = data_str, unbal = list(level2 = FALSE, level3 = FALSE),
                     pow_param = pow_param, 
                     alpha = alpha, pow_dist = pow_dist, pow_tail = pow_tail,
                     replicates = replicates)

A few notes about the sim_pow function in relation to nested data. First, the lmer function from the lme4 package is used to fit the models. When arima = TRUE, then the nlme package is used, but this is currently not supported. One note, the power simulation takes more computational time compared to the single level example.

The power output is identical to the single level model above:

power_out
var avg_test_stat sd_test_stat power num_reject num_repl data
act 1.199068 0.8455917 0.1 1 10 1.7804706, 0.7816410, 2.8604786, 0.6718963, 1.1378117, 1.2043643, 0.3539519, 2.2266216, 0.2676146, 0.7058292, 0.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000, 0.0000000
diff 6.810261 0.7410328 1.0 10 10 6.204323, 6.510491, 7.129845, 7.277809, 7.822506, 7.328319, 6.905194, 5.604418, 5.834302, 7.485405, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000, 1.000000
time 1.598097 0.8040690 0.2 2 10 1.1337604, 1.9734057, 2.7544328, 1.0466450, 1.4475744, 0.6762225, 2.8642617, 0.5552760, 1.4465758, 2.0828124, 0.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 0.0000000, 1.0000000, 0.0000000, 0.0000000, 0.0000000

Three Level Designs

fixed <- ~1 + time + diff + act + actClust + time:act
random <- ~1 + time
random3 <- ~ 1 + time
fixed_param <- c(4, 2, 6, 2.3, 7, 0)
random_param <- list(random_var = c(7, 4), rand_gen = 'rnorm')
random_param3 <- list(random_var = c(4, 2), rand_gen = 'rnorm')
cov_param <- list(dist_fun = c('rnorm', 'rnorm', 'rnorm'),
                  var_type = c("level1", "level2", "level3"),
                  opts = list(list(mean = 0, sd = 1.5),
                              list(mean = 0, sd = 4),
                              list(mean = 0, sd = 2)))
k <- 10
n <- 45
p <- 8
error_var <- 4
with_err_gen <- 'rnorm'
data_str <- "long"
pow_param <- c('time', 'diff', 'act', 'actClust')
alpha <- .01
pow_dist <- "z"
pow_tail <- 2
replicates <- 3
power_out <- sim_pow(fixed = fixed, random = random, random3 = random3,
                     fixed_param = fixed_param,
                     random_param = random_param, random_param3 = random_param3,
                     cov_param = cov_param,
                     k = k, n = n, p = p,
                     error_var = error_var, with_err_gen = "rnorm",
                     data_str = data_str, unbal = list(level2 = FALSE, level3 = FALSE),
                     pow_param = pow_param, alpha = alpha,
                     pow_dist = pow_dist, pow_tail = pow_tail, 
                     replicates = replicates)
power_out
var avg_test_stat sd_test_stat power num_reject num_repl data
act 63.786625 2.2518580 1 3 3 62.55067, 66.38581, 62.42340, 1.00000, 1.00000, 1.00000
actClust 17.804648 5.4796603 1 3 3 24.06992, 13.90624, 15.43778, 1.00000, 1.00000, 1.00000
diff 242.491518 9.3978039 1 3 3 252.7433, 240.4470, 234.2842, 1.0000, 1.0000, 1.0000
time 4.216052 0.3089218 1 3 3 4.198716, 3.916163, 4.533277, 1.000000, 1.000000, 1.000000

Generalized Power Analysis

fixed <- ~ 1 + act + diff
fixed_param <- c(0.1, 0.5, 0.3)
cov_param <- list(dist_fun = c('rnorm', 'rnorm'),
                  var_type = c("single", "single"),
                  opts = list(list(mean = 0, sd = 2),
                              list(mean = 0, sd = 4)))
n <- 50
pow_param <- c('(Intercept)', 'act', 'diff')
alpha <- .01
pow_dist <- "z"
pow_tail <- 2
replicates <- 10

power_out <- sim_pow_glm(fixed = fixed, fixed_param = fixed_param, 
                         cov_param = cov_param, 
                         n = n, data_str = "single", 
                         outcome_type = 'logistic',
                         pow_param = pow_param, alpha = alpha,
                         pow_dist = pow_dist, pow_tail = pow_tail, 
                         replicates = replicates)
power_out
var avg_test_stat sd_test_stat power num_reject num_repl data
(Intercept) 0.4781146 0.4327737 0.0 0 10 0.472603021, 0.583306239, 0.090192789, 0.069962478, 0.882679628, 1.198395241, 0.008760172, 0.220197541, 0.213077618, 1.041971181, 0.000000000, 0.000000000, 0.000000000, 0.000000000, 0.000000000, 0.000000000, 0.000000000, 0.000000000, 0.000000000, 0.000000000
act 2.4288803 0.6418409 0.3 3 10 2.163518, 2.281146, 3.117034, 1.123640, 2.421159, 3.407061, 2.504779, 1.936294, 2.911426, 2.422747, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 1.000000, 0.000000
diff 2.4156568 0.8158059 0.6 6 10 1.497130, 0.586447, 3.139853, 2.981695, 3.138462, 2.629135, 2.474835, 2.906578, 2.134430, 2.668005, 0.000000, 0.000000, 1.000000, 1.000000, 1.000000, 1.000000, 0.000000, 1.000000, 0.000000, 1.000000

Vary Arguments

fixed <- ~ 1 + act + diff
fixed_param <- c(0.1, 0.5, 0.3)
cov_param <- list(dist_fun = c('rnorm', 'rnorm'),
                  var_type = c("single", "single"),
                  opts = list(list(mean = 0, sd = 5),
                              list(mean = 0, sd = 8)))
n <- NULL
pow_param <- c('(Intercept)', 'act', 'diff')
alpha <- .01
pow_dist <- "z"
pow_tail <- 2
replicates <- 10
terms_vary <- list(n = c(20, 40, 60, 80, 100),
                   fixed_param = list(c(0.5, 0.1, 0.2), 
                                      c(0.6, 0.1, 0.2)))

power_out <- sim_pow_glm(fixed = fixed, fixed_param = fixed_param, 
                         cov_param = cov_param, 
                         n = n, data_str = "single", 
                         outcome_type = 'logistic',
                         pow_param = pow_param, alpha = alpha,
                         pow_dist = pow_dist, pow_tail = pow_tail, 
                         replicates = replicates, terms_vary = terms_vary)
## Warning: glm.fit: algorithm did not converge
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred

## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
power_out
## # A tibble: 30 x 9
## # Groups:   var, n [?]
##            var     n fixed_param avg_test_stat sd_test_stat power
##         <fctr> <dbl>      <fctr>         <dbl>        <dbl> <dbl>
##  1 (Intercept)    20 0.5,0.1,0.2     1.2532056    0.5452253   0.0
##  2 (Intercept)    20 0.6,0.1,0.2     0.6418442    0.6165474   0.0
##  3 (Intercept)    40 0.5,0.1,0.2     1.6335148    0.7007165   0.1
##  4 (Intercept)    40 0.6,0.1,0.2     1.4600937    0.6766618   0.1
##  5 (Intercept)    60 0.5,0.1,0.2     1.2774841    0.8152388   0.0
##  6 (Intercept)    60 0.6,0.1,0.2     1.8918550    0.7959895   0.2
##  7 (Intercept)    80 0.5,0.1,0.2     1.3396153    0.9382061   0.1
##  8 (Intercept)    80 0.6,0.1,0.2     2.2675546    1.2499084   0.4
##  9 (Intercept)   100 0.5,0.1,0.2     1.9007975    0.9094621   0.1
## 10 (Intercept)   100 0.6,0.1,0.2     2.2761739    0.8134314   0.4
## # ... with 20 more rows, and 3 more variables: num_reject <dbl>,
## #   num_repl <dbl>, data <list>