Provided a distribution of risk in a population, this function calculates the exact AUC of a model that produces the risk estimates. For example, a logistic regression model built with a normal linear predictor yields logit-normal distributed predicted risks. The AUC from the logistic regression model is the same as the AUC estimated from the distribution of the predicted risks, independent of the outcome. This method for AUC calculation is useful for simulation studies where the predicted risks are a mixture of two distributions. The exact prevalence of the outcome can easily be calculated, along with the exact AUC of the model.

auc_density(density, cut.points = seq(from = 0, to = 1, by = 0.001), ...)

Arguments

density

a function name that describes the continuous probability density function of the risk from 0 to 1.

cut.points

sequence of points in [0, 1] where the sensitivity and specificity are calculated. More points lead to a more precise estimate of the AUC. Default is seq(from = 0, to = 1,by = 0.001).

...

arguments for the function specified in density. For example, dbeta(x, shape1=1, shape2=1) has need for two additional arguments to specify the density function (shape1 and shape2).

Value

Returns a list sensitivity and specificity at each cut point, the expected value or mean risk, and the AUC associated with the distribution.

Author

Daniel D Sjoberg sjobergd@mskcc.org

Examples

auc_density(density = dbeta, shape1 = 1, shape2 = 1)
#> $cut.point
#>    [1] 0.000 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009 0.010 0.011
#>   [13] 0.012 0.013 0.014 0.015 0.016 0.017 0.018 0.019 0.020 0.021 0.022 0.023
#>   [25] 0.024 0.025 0.026 0.027 0.028 0.029 0.030 0.031 0.032 0.033 0.034 0.035
#>   [37] 0.036 0.037 0.038 0.039 0.040 0.041 0.042 0.043 0.044 0.045 0.046 0.047
#>   [49] 0.048 0.049 0.050 0.051 0.052 0.053 0.054 0.055 0.056 0.057 0.058 0.059
#>   [61] 0.060 0.061 0.062 0.063 0.064 0.065 0.066 0.067 0.068 0.069 0.070 0.071
#>   [73] 0.072 0.073 0.074 0.075 0.076 0.077 0.078 0.079 0.080 0.081 0.082 0.083
#>   [85] 0.084 0.085 0.086 0.087 0.088 0.089 0.090 0.091 0.092 0.093 0.094 0.095
#>   [97] 0.096 0.097 0.098 0.099 0.100 0.101 0.102 0.103 0.104 0.105 0.106 0.107
#>  [109] 0.108 0.109 0.110 0.111 0.112 0.113 0.114 0.115 0.116 0.117 0.118 0.119
#>  [121] 0.120 0.121 0.122 0.123 0.124 0.125 0.126 0.127 0.128 0.129 0.130 0.131
#>  [133] 0.132 0.133 0.134 0.135 0.136 0.137 0.138 0.139 0.140 0.141 0.142 0.143
#>  [145] 0.144 0.145 0.146 0.147 0.148 0.149 0.150 0.151 0.152 0.153 0.154 0.155
#>  [157] 0.156 0.157 0.158 0.159 0.160 0.161 0.162 0.163 0.164 0.165 0.166 0.167
#>  [169] 0.168 0.169 0.170 0.171 0.172 0.173 0.174 0.175 0.176 0.177 0.178 0.179
#>  [181] 0.180 0.181 0.182 0.183 0.184 0.185 0.186 0.187 0.188 0.189 0.190 0.191
#>  [193] 0.192 0.193 0.194 0.195 0.196 0.197 0.198 0.199 0.200 0.201 0.202 0.203
#>  [205] 0.204 0.205 0.206 0.207 0.208 0.209 0.210 0.211 0.212 0.213 0.214 0.215
#>  [217] 0.216 0.217 0.218 0.219 0.220 0.221 0.222 0.223 0.224 0.225 0.226 0.227
#>  [229] 0.228 0.229 0.230 0.231 0.232 0.233 0.234 0.235 0.236 0.237 0.238 0.239
#>  [241] 0.240 0.241 0.242 0.243 0.244 0.245 0.246 0.247 0.248 0.249 0.250 0.251
#>  [253] 0.252 0.253 0.254 0.255 0.256 0.257 0.258 0.259 0.260 0.261 0.262 0.263
#>  [265] 0.264 0.265 0.266 0.267 0.268 0.269 0.270 0.271 0.272 0.273 0.274 0.275
#>  [277] 0.276 0.277 0.278 0.279 0.280 0.281 0.282 0.283 0.284 0.285 0.286 0.287
#>  [289] 0.288 0.289 0.290 0.291 0.292 0.293 0.294 0.295 0.296 0.297 0.298 0.299
#>  [301] 0.300 0.301 0.302 0.303 0.304 0.305 0.306 0.307 0.308 0.309 0.310 0.311
#>  [313] 0.312 0.313 0.314 0.315 0.316 0.317 0.318 0.319 0.320 0.321 0.322 0.323
#>  [325] 0.324 0.325 0.326 0.327 0.328 0.329 0.330 0.331 0.332 0.333 0.334 0.335
#>  [337] 0.336 0.337 0.338 0.339 0.340 0.341 0.342 0.343 0.344 0.345 0.346 0.347
#>  [349] 0.348 0.349 0.350 0.351 0.352 0.353 0.354 0.355 0.356 0.357 0.358 0.359
#>  [361] 0.360 0.361 0.362 0.363 0.364 0.365 0.366 0.367 0.368 0.369 0.370 0.371
#>  [373] 0.372 0.373 0.374 0.375 0.376 0.377 0.378 0.379 0.380 0.381 0.382 0.383
#>  [385] 0.384 0.385 0.386 0.387 0.388 0.389 0.390 0.391 0.392 0.393 0.394 0.395
#>  [397] 0.396 0.397 0.398 0.399 0.400 0.401 0.402 0.403 0.404 0.405 0.406 0.407
#>  [409] 0.408 0.409 0.410 0.411 0.412 0.413 0.414 0.415 0.416 0.417 0.418 0.419
#>  [421] 0.420 0.421 0.422 0.423 0.424 0.425 0.426 0.427 0.428 0.429 0.430 0.431
#>  [433] 0.432 0.433 0.434 0.435 0.436 0.437 0.438 0.439 0.440 0.441 0.442 0.443
#>  [445] 0.444 0.445 0.446 0.447 0.448 0.449 0.450 0.451 0.452 0.453 0.454 0.455
#>  [457] 0.456 0.457 0.458 0.459 0.460 0.461 0.462 0.463 0.464 0.465 0.466 0.467
#>  [469] 0.468 0.469 0.470 0.471 0.472 0.473 0.474 0.475 0.476 0.477 0.478 0.479
#>  [481] 0.480 0.481 0.482 0.483 0.484 0.485 0.486 0.487 0.488 0.489 0.490 0.491
#>  [493] 0.492 0.493 0.494 0.495 0.496 0.497 0.498 0.499 0.500 0.501 0.502 0.503
#>  [505] 0.504 0.505 0.506 0.507 0.508 0.509 0.510 0.511 0.512 0.513 0.514 0.515
#>  [517] 0.516 0.517 0.518 0.519 0.520 0.521 0.522 0.523 0.524 0.525 0.526 0.527
#>  [529] 0.528 0.529 0.530 0.531 0.532 0.533 0.534 0.535 0.536 0.537 0.538 0.539
#>  [541] 0.540 0.541 0.542 0.543 0.544 0.545 0.546 0.547 0.548 0.549 0.550 0.551
#>  [553] 0.552 0.553 0.554 0.555 0.556 0.557 0.558 0.559 0.560 0.561 0.562 0.563
#>  [565] 0.564 0.565 0.566 0.567 0.568 0.569 0.570 0.571 0.572 0.573 0.574 0.575
#>  [577] 0.576 0.577 0.578 0.579 0.580 0.581 0.582 0.583 0.584 0.585 0.586 0.587
#>  [589] 0.588 0.589 0.590 0.591 0.592 0.593 0.594 0.595 0.596 0.597 0.598 0.599
#>  [601] 0.600 0.601 0.602 0.603 0.604 0.605 0.606 0.607 0.608 0.609 0.610 0.611
#>  [613] 0.612 0.613 0.614 0.615 0.616 0.617 0.618 0.619 0.620 0.621 0.622 0.623
#>  [625] 0.624 0.625 0.626 0.627 0.628 0.629 0.630 0.631 0.632 0.633 0.634 0.635
#>  [637] 0.636 0.637 0.638 0.639 0.640 0.641 0.642 0.643 0.644 0.645 0.646 0.647
#>  [649] 0.648 0.649 0.650 0.651 0.652 0.653 0.654 0.655 0.656 0.657 0.658 0.659
#>  [661] 0.660 0.661 0.662 0.663 0.664 0.665 0.666 0.667 0.668 0.669 0.670 0.671
#>  [673] 0.672 0.673 0.674 0.675 0.676 0.677 0.678 0.679 0.680 0.681 0.682 0.683
#>  [685] 0.684 0.685 0.686 0.687 0.688 0.689 0.690 0.691 0.692 0.693 0.694 0.695
#>  [697] 0.696 0.697 0.698 0.699 0.700 0.701 0.702 0.703 0.704 0.705 0.706 0.707
#>  [709] 0.708 0.709 0.710 0.711 0.712 0.713 0.714 0.715 0.716 0.717 0.718 0.719
#>  [721] 0.720 0.721 0.722 0.723 0.724 0.725 0.726 0.727 0.728 0.729 0.730 0.731
#>  [733] 0.732 0.733 0.734 0.735 0.736 0.737 0.738 0.739 0.740 0.741 0.742 0.743
#>  [745] 0.744 0.745 0.746 0.747 0.748 0.749 0.750 0.751 0.752 0.753 0.754 0.755
#>  [757] 0.756 0.757 0.758 0.759 0.760 0.761 0.762 0.763 0.764 0.765 0.766 0.767
#>  [769] 0.768 0.769 0.770 0.771 0.772 0.773 0.774 0.775 0.776 0.777 0.778 0.779
#>  [781] 0.780 0.781 0.782 0.783 0.784 0.785 0.786 0.787 0.788 0.789 0.790 0.791
#>  [793] 0.792 0.793 0.794 0.795 0.796 0.797 0.798 0.799 0.800 0.801 0.802 0.803
#>  [805] 0.804 0.805 0.806 0.807 0.808 0.809 0.810 0.811 0.812 0.813 0.814 0.815
#>  [817] 0.816 0.817 0.818 0.819 0.820 0.821 0.822 0.823 0.824 0.825 0.826 0.827
#>  [829] 0.828 0.829 0.830 0.831 0.832 0.833 0.834 0.835 0.836 0.837 0.838 0.839
#>  [841] 0.840 0.841 0.842 0.843 0.844 0.845 0.846 0.847 0.848 0.849 0.850 0.851
#>  [853] 0.852 0.853 0.854 0.855 0.856 0.857 0.858 0.859 0.860 0.861 0.862 0.863
#>  [865] 0.864 0.865 0.866 0.867 0.868 0.869 0.870 0.871 0.872 0.873 0.874 0.875
#>  [877] 0.876 0.877 0.878 0.879 0.880 0.881 0.882 0.883 0.884 0.885 0.886 0.887
#>  [889] 0.888 0.889 0.890 0.891 0.892 0.893 0.894 0.895 0.896 0.897 0.898 0.899
#>  [901] 0.900 0.901 0.902 0.903 0.904 0.905 0.906 0.907 0.908 0.909 0.910 0.911
#>  [913] 0.912 0.913 0.914 0.915 0.916 0.917 0.918 0.919 0.920 0.921 0.922 0.923
#>  [925] 0.924 0.925 0.926 0.927 0.928 0.929 0.930 0.931 0.932 0.933 0.934 0.935
#>  [937] 0.936 0.937 0.938 0.939 0.940 0.941 0.942 0.943 0.944 0.945 0.946 0.947
#>  [949] 0.948 0.949 0.950 0.951 0.952 0.953 0.954 0.955 0.956 0.957 0.958 0.959
#>  [961] 0.960 0.961 0.962 0.963 0.964 0.965 0.966 0.967 0.968 0.969 0.970 0.971
#>  [973] 0.972 0.973 0.974 0.975 0.976 0.977 0.978 0.979 0.980 0.981 0.982 0.983
#>  [985] 0.984 0.985 0.986 0.987 0.988 0.989 0.990 0.991 0.992 0.993 0.994 0.995
#>  [997] 0.996 0.997 0.998 0.999 1.000
#> 
#> $sensitivity
#>    [1] 1.000000 0.999999 0.999996 0.999991 0.999984 0.999975 0.999964 0.999951
#>    [9] 0.999936 0.999919 0.999900 0.999879 0.999856 0.999831 0.999804 0.999775
#>   [17] 0.999744 0.999711 0.999676 0.999639 0.999600 0.999559 0.999516 0.999471
#>   [25] 0.999424 0.999375 0.999324 0.999271 0.999216 0.999159 0.999100 0.999039
#>   [33] 0.998976 0.998911 0.998844 0.998775 0.998704 0.998631 0.998556 0.998479
#>   [41] 0.998400 0.998319 0.998236 0.998151 0.998064 0.997975 0.997884 0.997791
#>   [49] 0.997696 0.997599 0.997500 0.997399 0.997296 0.997191 0.997084 0.996975
#>   [57] 0.996864 0.996751 0.996636 0.996519 0.996400 0.996279 0.996156 0.996031
#>   [65] 0.995904 0.995775 0.995644 0.995511 0.995376 0.995239 0.995100 0.994959
#>   [73] 0.994816 0.994671 0.994524 0.994375 0.994224 0.994071 0.993916 0.993759
#>   [81] 0.993600 0.993439 0.993276 0.993111 0.992944 0.992775 0.992604 0.992431
#>   [89] 0.992256 0.992079 0.991900 0.991719 0.991536 0.991351 0.991164 0.990975
#>   [97] 0.990784 0.990591 0.990396 0.990199 0.990000 0.989799 0.989596 0.989391
#>  [105] 0.989184 0.988975 0.988764 0.988551 0.988336 0.988119 0.987900 0.987679
#>  [113] 0.987456 0.987231 0.987004 0.986775 0.986544 0.986311 0.986076 0.985839
#>  [121] 0.985600 0.985359 0.985116 0.984871 0.984624 0.984375 0.984124 0.983871
#>  [129] 0.983616 0.983359 0.983100 0.982839 0.982576 0.982311 0.982044 0.981775
#>  [137] 0.981504 0.981231 0.980956 0.980679 0.980400 0.980119 0.979836 0.979551
#>  [145] 0.979264 0.978975 0.978684 0.978391 0.978096 0.977799 0.977500 0.977199
#>  [153] 0.976896 0.976591 0.976284 0.975975 0.975664 0.975351 0.975036 0.974719
#>  [161] 0.974400 0.974079 0.973756 0.973431 0.973104 0.972775 0.972444 0.972111
#>  [169] 0.971776 0.971439 0.971100 0.970759 0.970416 0.970071 0.969724 0.969375
#>  [177] 0.969024 0.968671 0.968316 0.967959 0.967600 0.967239 0.966876 0.966511
#>  [185] 0.966144 0.965775 0.965404 0.965031 0.964656 0.964279 0.963900 0.963519
#>  [193] 0.963136 0.962751 0.962364 0.961975 0.961584 0.961191 0.960796 0.960399
#>  [201] 0.960000 0.959599 0.959196 0.958791 0.958384 0.957975 0.957564 0.957151
#>  [209] 0.956736 0.956319 0.955900 0.955479 0.955056 0.954631 0.954204 0.953775
#>  [217] 0.953344 0.952911 0.952476 0.952039 0.951600 0.951159 0.950716 0.950271
#>  [225] 0.949824 0.949375 0.948924 0.948471 0.948016 0.947559 0.947100 0.946639
#>  [233] 0.946176 0.945711 0.945244 0.944775 0.944304 0.943831 0.943356 0.942879
#>  [241] 0.942400 0.941919 0.941436 0.940951 0.940464 0.939975 0.939484 0.938991
#>  [249] 0.938496 0.937999 0.937500 0.936999 0.936496 0.935991 0.935484 0.934975
#>  [257] 0.934464 0.933951 0.933436 0.932919 0.932400 0.931879 0.931356 0.930831
#>  [265] 0.930304 0.929775 0.929244 0.928711 0.928176 0.927639 0.927100 0.926559
#>  [273] 0.926016 0.925471 0.924924 0.924375 0.923824 0.923271 0.922716 0.922159
#>  [281] 0.921600 0.921039 0.920476 0.919911 0.919344 0.918775 0.918204 0.917631
#>  [289] 0.917056 0.916479 0.915900 0.915319 0.914736 0.914151 0.913564 0.912975
#>  [297] 0.912384 0.911791 0.911196 0.910599 0.910000 0.909399 0.908796 0.908191
#>  [305] 0.907584 0.906975 0.906364 0.905751 0.905136 0.904519 0.903900 0.903279
#>  [313] 0.902656 0.902031 0.901404 0.900775 0.900144 0.899511 0.898876 0.898239
#>  [321] 0.897600 0.896959 0.896316 0.895671 0.895024 0.894375 0.893724 0.893071
#>  [329] 0.892416 0.891759 0.891100 0.890439 0.889776 0.889111 0.888444 0.887775
#>  [337] 0.887104 0.886431 0.885756 0.885079 0.884400 0.883719 0.883036 0.882351
#>  [345] 0.881664 0.880975 0.880284 0.879591 0.878896 0.878199 0.877500 0.876799
#>  [353] 0.876096 0.875391 0.874684 0.873975 0.873264 0.872551 0.871836 0.871119
#>  [361] 0.870400 0.869679 0.868956 0.868231 0.867504 0.866775 0.866044 0.865311
#>  [369] 0.864576 0.863839 0.863100 0.862359 0.861616 0.860871 0.860124 0.859375
#>  [377] 0.858624 0.857871 0.857116 0.856359 0.855600 0.854839 0.854076 0.853311
#>  [385] 0.852544 0.851775 0.851004 0.850231 0.849456 0.848679 0.847900 0.847119
#>  [393] 0.846336 0.845551 0.844764 0.843975 0.843184 0.842391 0.841596 0.840799
#>  [401] 0.840000 0.839199 0.838396 0.837591 0.836784 0.835975 0.835164 0.834351
#>  [409] 0.833536 0.832719 0.831900 0.831079 0.830256 0.829431 0.828604 0.827775
#>  [417] 0.826944 0.826111 0.825276 0.824439 0.823600 0.822759 0.821916 0.821071
#>  [425] 0.820224 0.819375 0.818524 0.817671 0.816816 0.815959 0.815100 0.814239
#>  [433] 0.813376 0.812511 0.811644 0.810775 0.809904 0.809031 0.808156 0.807279
#>  [441] 0.806400 0.805519 0.804636 0.803751 0.802864 0.801975 0.801084 0.800191
#>  [449] 0.799296 0.798399 0.797500 0.796599 0.795696 0.794791 0.793884 0.792975
#>  [457] 0.792064 0.791151 0.790236 0.789319 0.788400 0.787479 0.786556 0.785631
#>  [465] 0.784704 0.783775 0.782844 0.781911 0.780976 0.780039 0.779100 0.778159
#>  [473] 0.777216 0.776271 0.775324 0.774375 0.773424 0.772471 0.771516 0.770559
#>  [481] 0.769600 0.768639 0.767676 0.766711 0.765744 0.764775 0.763804 0.762831
#>  [489] 0.761856 0.760879 0.759900 0.758919 0.757936 0.756951 0.755964 0.754975
#>  [497] 0.753984 0.752991 0.751996 0.750999 0.750000 0.748999 0.747996 0.746991
#>  [505] 0.745984 0.744975 0.743964 0.742951 0.741936 0.740919 0.739900 0.738879
#>  [513] 0.737856 0.736831 0.735804 0.734775 0.733744 0.732711 0.731676 0.730639
#>  [521] 0.729600 0.728559 0.727516 0.726471 0.725424 0.724375 0.723324 0.722271
#>  [529] 0.721216 0.720159 0.719100 0.718039 0.716976 0.715911 0.714844 0.713775
#>  [537] 0.712704 0.711631 0.710556 0.709479 0.708400 0.707319 0.706236 0.705151
#>  [545] 0.704064 0.702975 0.701884 0.700791 0.699696 0.698599 0.697500 0.696399
#>  [553] 0.695296 0.694191 0.693084 0.691975 0.690864 0.689751 0.688636 0.687519
#>  [561] 0.686400 0.685279 0.684156 0.683031 0.681904 0.680775 0.679644 0.678511
#>  [569] 0.677376 0.676239 0.675100 0.673959 0.672816 0.671671 0.670524 0.669375
#>  [577] 0.668224 0.667071 0.665916 0.664759 0.663600 0.662439 0.661276 0.660111
#>  [585] 0.658944 0.657775 0.656604 0.655431 0.654256 0.653079 0.651900 0.650719
#>  [593] 0.649536 0.648351 0.647164 0.645975 0.644784 0.643591 0.642396 0.641199
#>  [601] 0.640000 0.638799 0.637596 0.636391 0.635184 0.633975 0.632764 0.631551
#>  [609] 0.630336 0.629119 0.627900 0.626679 0.625456 0.624231 0.623004 0.621775
#>  [617] 0.620544 0.619311 0.618076 0.616839 0.615600 0.614359 0.613116 0.611871
#>  [625] 0.610624 0.609375 0.608124 0.606871 0.605616 0.604359 0.603100 0.601839
#>  [633] 0.600576 0.599311 0.598044 0.596775 0.595504 0.594231 0.592956 0.591679
#>  [641] 0.590400 0.589119 0.587836 0.586551 0.585264 0.583975 0.582684 0.581391
#>  [649] 0.580096 0.578799 0.577500 0.576199 0.574896 0.573591 0.572284 0.570975
#>  [657] 0.569664 0.568351 0.567036 0.565719 0.564400 0.563079 0.561756 0.560431
#>  [665] 0.559104 0.557775 0.556444 0.555111 0.553776 0.552439 0.551100 0.549759
#>  [673] 0.548416 0.547071 0.545724 0.544375 0.543024 0.541671 0.540316 0.538959
#>  [681] 0.537600 0.536239 0.534876 0.533511 0.532144 0.530775 0.529404 0.528031
#>  [689] 0.526656 0.525279 0.523900 0.522519 0.521136 0.519751 0.518364 0.516975
#>  [697] 0.515584 0.514191 0.512796 0.511399 0.510000 0.508599 0.507196 0.505791
#>  [705] 0.504384 0.502975 0.501564 0.500151 0.498736 0.497319 0.495900 0.494479
#>  [713] 0.493056 0.491631 0.490204 0.488775 0.487344 0.485911 0.484476 0.483039
#>  [721] 0.481600 0.480159 0.478716 0.477271 0.475824 0.474375 0.472924 0.471471
#>  [729] 0.470016 0.468559 0.467100 0.465639 0.464176 0.462711 0.461244 0.459775
#>  [737] 0.458304 0.456831 0.455356 0.453879 0.452400 0.450919 0.449436 0.447951
#>  [745] 0.446464 0.444975 0.443484 0.441991 0.440496 0.438999 0.437500 0.435999
#>  [753] 0.434496 0.432991 0.431484 0.429975 0.428464 0.426951 0.425436 0.423919
#>  [761] 0.422400 0.420879 0.419356 0.417831 0.416304 0.414775 0.413244 0.411711
#>  [769] 0.410176 0.408639 0.407100 0.405559 0.404016 0.402471 0.400924 0.399375
#>  [777] 0.397824 0.396271 0.394716 0.393159 0.391600 0.390039 0.388476 0.386911
#>  [785] 0.385344 0.383775 0.382204 0.380631 0.379056 0.377479 0.375900 0.374319
#>  [793] 0.372736 0.371151 0.369564 0.367975 0.366384 0.364791 0.363196 0.361599
#>  [801] 0.360000 0.358399 0.356796 0.355191 0.353584 0.351975 0.350364 0.348751
#>  [809] 0.347136 0.345519 0.343900 0.342279 0.340656 0.339031 0.337404 0.335775
#>  [817] 0.334144 0.332511 0.330876 0.329239 0.327600 0.325959 0.324316 0.322671
#>  [825] 0.321024 0.319375 0.317724 0.316071 0.314416 0.312759 0.311100 0.309439
#>  [833] 0.307776 0.306111 0.304444 0.302775 0.301104 0.299431 0.297756 0.296079
#>  [841] 0.294400 0.292719 0.291036 0.289351 0.287664 0.285975 0.284284 0.282591
#>  [849] 0.280896 0.279199 0.277500 0.275799 0.274096 0.272391 0.270684 0.268975
#>  [857] 0.267264 0.265551 0.263836 0.262119 0.260400 0.258679 0.256956 0.255231
#>  [865] 0.253504 0.251775 0.250044 0.248311 0.246576 0.244839 0.243100 0.241359
#>  [873] 0.239616 0.237871 0.236124 0.234375 0.232624 0.230871 0.229116 0.227359
#>  [881] 0.225600 0.223839 0.222076 0.220311 0.218544 0.216775 0.215004 0.213231
#>  [889] 0.211456 0.209679 0.207900 0.206119 0.204336 0.202551 0.200764 0.198975
#>  [897] 0.197184 0.195391 0.193596 0.191799 0.190000 0.188199 0.186396 0.184591
#>  [905] 0.182784 0.180975 0.179164 0.177351 0.175536 0.173719 0.171900 0.170079
#>  [913] 0.168256 0.166431 0.164604 0.162775 0.160944 0.159111 0.157276 0.155439
#>  [921] 0.153600 0.151759 0.149916 0.148071 0.146224 0.144375 0.142524 0.140671
#>  [929] 0.138816 0.136959 0.135100 0.133239 0.131376 0.129511 0.127644 0.125775
#>  [937] 0.123904 0.122031 0.120156 0.118279 0.116400 0.114519 0.112636 0.110751
#>  [945] 0.108864 0.106975 0.105084 0.103191 0.101296 0.099399 0.097500 0.095599
#>  [953] 0.093696 0.091791 0.089884 0.087975 0.086064 0.084151 0.082236 0.080319
#>  [961] 0.078400 0.076479 0.074556 0.072631 0.070704 0.068775 0.066844 0.064911
#>  [969] 0.062976 0.061039 0.059100 0.057159 0.055216 0.053271 0.051324 0.049375
#>  [977] 0.047424 0.045471 0.043516 0.041559 0.039600 0.037639 0.035676 0.033711
#>  [985] 0.031744 0.029775 0.027804 0.025831 0.023856 0.021879 0.019900 0.017919
#>  [993] 0.015936 0.013951 0.011964 0.009975 0.007984 0.005991 0.003996 0.001999
#> [1001] 0.000000
#> 
#> $specificity
#>    [1] 0.000000 0.001999 0.003996 0.005991 0.007984 0.009975 0.011964 0.013951
#>    [9] 0.015936 0.017919 0.019900 0.021879 0.023856 0.025831 0.027804 0.029775
#>   [17] 0.031744 0.033711 0.035676 0.037639 0.039600 0.041559 0.043516 0.045471
#>   [25] 0.047424 0.049375 0.051324 0.053271 0.055216 0.057159 0.059100 0.061039
#>   [33] 0.062976 0.064911 0.066844 0.068775 0.070704 0.072631 0.074556 0.076479
#>   [41] 0.078400 0.080319 0.082236 0.084151 0.086064 0.087975 0.089884 0.091791
#>   [49] 0.093696 0.095599 0.097500 0.099399 0.101296 0.103191 0.105084 0.106975
#>   [57] 0.108864 0.110751 0.112636 0.114519 0.116400 0.118279 0.120156 0.122031
#>   [65] 0.123904 0.125775 0.127644 0.129511 0.131376 0.133239 0.135100 0.136959
#>   [73] 0.138816 0.140671 0.142524 0.144375 0.146224 0.148071 0.149916 0.151759
#>   [81] 0.153600 0.155439 0.157276 0.159111 0.160944 0.162775 0.164604 0.166431
#>   [89] 0.168256 0.170079 0.171900 0.173719 0.175536 0.177351 0.179164 0.180975
#>   [97] 0.182784 0.184591 0.186396 0.188199 0.190000 0.191799 0.193596 0.195391
#>  [105] 0.197184 0.198975 0.200764 0.202551 0.204336 0.206119 0.207900 0.209679
#>  [113] 0.211456 0.213231 0.215004 0.216775 0.218544 0.220311 0.222076 0.223839
#>  [121] 0.225600 0.227359 0.229116 0.230871 0.232624 0.234375 0.236124 0.237871
#>  [129] 0.239616 0.241359 0.243100 0.244839 0.246576 0.248311 0.250044 0.251775
#>  [137] 0.253504 0.255231 0.256956 0.258679 0.260400 0.262119 0.263836 0.265551
#>  [145] 0.267264 0.268975 0.270684 0.272391 0.274096 0.275799 0.277500 0.279199
#>  [153] 0.280896 0.282591 0.284284 0.285975 0.287664 0.289351 0.291036 0.292719
#>  [161] 0.294400 0.296079 0.297756 0.299431 0.301104 0.302775 0.304444 0.306111
#>  [169] 0.307776 0.309439 0.311100 0.312759 0.314416 0.316071 0.317724 0.319375
#>  [177] 0.321024 0.322671 0.324316 0.325959 0.327600 0.329239 0.330876 0.332511
#>  [185] 0.334144 0.335775 0.337404 0.339031 0.340656 0.342279 0.343900 0.345519
#>  [193] 0.347136 0.348751 0.350364 0.351975 0.353584 0.355191 0.356796 0.358399
#>  [201] 0.360000 0.361599 0.363196 0.364791 0.366384 0.367975 0.369564 0.371151
#>  [209] 0.372736 0.374319 0.375900 0.377479 0.379056 0.380631 0.382204 0.383775
#>  [217] 0.385344 0.386911 0.388476 0.390039 0.391600 0.393159 0.394716 0.396271
#>  [225] 0.397824 0.399375 0.400924 0.402471 0.404016 0.405559 0.407100 0.408639
#>  [233] 0.410176 0.411711 0.413244 0.414775 0.416304 0.417831 0.419356 0.420879
#>  [241] 0.422400 0.423919 0.425436 0.426951 0.428464 0.429975 0.431484 0.432991
#>  [249] 0.434496 0.435999 0.437500 0.438999 0.440496 0.441991 0.443484 0.444975
#>  [257] 0.446464 0.447951 0.449436 0.450919 0.452400 0.453879 0.455356 0.456831
#>  [265] 0.458304 0.459775 0.461244 0.462711 0.464176 0.465639 0.467100 0.468559
#>  [273] 0.470016 0.471471 0.472924 0.474375 0.475824 0.477271 0.478716 0.480159
#>  [281] 0.481600 0.483039 0.484476 0.485911 0.487344 0.488775 0.490204 0.491631
#>  [289] 0.493056 0.494479 0.495900 0.497319 0.498736 0.500151 0.501564 0.502975
#>  [297] 0.504384 0.505791 0.507196 0.508599 0.510000 0.511399 0.512796 0.514191
#>  [305] 0.515584 0.516975 0.518364 0.519751 0.521136 0.522519 0.523900 0.525279
#>  [313] 0.526656 0.528031 0.529404 0.530775 0.532144 0.533511 0.534876 0.536239
#>  [321] 0.537600 0.538959 0.540316 0.541671 0.543024 0.544375 0.545724 0.547071
#>  [329] 0.548416 0.549759 0.551100 0.552439 0.553776 0.555111 0.556444 0.557775
#>  [337] 0.559104 0.560431 0.561756 0.563079 0.564400 0.565719 0.567036 0.568351
#>  [345] 0.569664 0.570975 0.572284 0.573591 0.574896 0.576199 0.577500 0.578799
#>  [353] 0.580096 0.581391 0.582684 0.583975 0.585264 0.586551 0.587836 0.589119
#>  [361] 0.590400 0.591679 0.592956 0.594231 0.595504 0.596775 0.598044 0.599311
#>  [369] 0.600576 0.601839 0.603100 0.604359 0.605616 0.606871 0.608124 0.609375
#>  [377] 0.610624 0.611871 0.613116 0.614359 0.615600 0.616839 0.618076 0.619311
#>  [385] 0.620544 0.621775 0.623004 0.624231 0.625456 0.626679 0.627900 0.629119
#>  [393] 0.630336 0.631551 0.632764 0.633975 0.635184 0.636391 0.637596 0.638799
#>  [401] 0.640000 0.641199 0.642396 0.643591 0.644784 0.645975 0.647164 0.648351
#>  [409] 0.649536 0.650719 0.651900 0.653079 0.654256 0.655431 0.656604 0.657775
#>  [417] 0.658944 0.660111 0.661276 0.662439 0.663600 0.664759 0.665916 0.667071
#>  [425] 0.668224 0.669375 0.670524 0.671671 0.672816 0.673959 0.675100 0.676239
#>  [433] 0.677376 0.678511 0.679644 0.680775 0.681904 0.683031 0.684156 0.685279
#>  [441] 0.686400 0.687519 0.688636 0.689751 0.690864 0.691975 0.693084 0.694191
#>  [449] 0.695296 0.696399 0.697500 0.698599 0.699696 0.700791 0.701884 0.702975
#>  [457] 0.704064 0.705151 0.706236 0.707319 0.708400 0.709479 0.710556 0.711631
#>  [465] 0.712704 0.713775 0.714844 0.715911 0.716976 0.718039 0.719100 0.720159
#>  [473] 0.721216 0.722271 0.723324 0.724375 0.725424 0.726471 0.727516 0.728559
#>  [481] 0.729600 0.730639 0.731676 0.732711 0.733744 0.734775 0.735804 0.736831
#>  [489] 0.737856 0.738879 0.739900 0.740919 0.741936 0.742951 0.743964 0.744975
#>  [497] 0.745984 0.746991 0.747996 0.748999 0.750000 0.750999 0.751996 0.752991
#>  [505] 0.753984 0.754975 0.755964 0.756951 0.757936 0.758919 0.759900 0.760879
#>  [513] 0.761856 0.762831 0.763804 0.764775 0.765744 0.766711 0.767676 0.768639
#>  [521] 0.769600 0.770559 0.771516 0.772471 0.773424 0.774375 0.775324 0.776271
#>  [529] 0.777216 0.778159 0.779100 0.780039 0.780976 0.781911 0.782844 0.783775
#>  [537] 0.784704 0.785631 0.786556 0.787479 0.788400 0.789319 0.790236 0.791151
#>  [545] 0.792064 0.792975 0.793884 0.794791 0.795696 0.796599 0.797500 0.798399
#>  [553] 0.799296 0.800191 0.801084 0.801975 0.802864 0.803751 0.804636 0.805519
#>  [561] 0.806400 0.807279 0.808156 0.809031 0.809904 0.810775 0.811644 0.812511
#>  [569] 0.813376 0.814239 0.815100 0.815959 0.816816 0.817671 0.818524 0.819375
#>  [577] 0.820224 0.821071 0.821916 0.822759 0.823600 0.824439 0.825276 0.826111
#>  [585] 0.826944 0.827775 0.828604 0.829431 0.830256 0.831079 0.831900 0.832719
#>  [593] 0.833536 0.834351 0.835164 0.835975 0.836784 0.837591 0.838396 0.839199
#>  [601] 0.840000 0.840799 0.841596 0.842391 0.843184 0.843975 0.844764 0.845551
#>  [609] 0.846336 0.847119 0.847900 0.848679 0.849456 0.850231 0.851004 0.851775
#>  [617] 0.852544 0.853311 0.854076 0.854839 0.855600 0.856359 0.857116 0.857871
#>  [625] 0.858624 0.859375 0.860124 0.860871 0.861616 0.862359 0.863100 0.863839
#>  [633] 0.864576 0.865311 0.866044 0.866775 0.867504 0.868231 0.868956 0.869679
#>  [641] 0.870400 0.871119 0.871836 0.872551 0.873264 0.873975 0.874684 0.875391
#>  [649] 0.876096 0.876799 0.877500 0.878199 0.878896 0.879591 0.880284 0.880975
#>  [657] 0.881664 0.882351 0.883036 0.883719 0.884400 0.885079 0.885756 0.886431
#>  [665] 0.887104 0.887775 0.888444 0.889111 0.889776 0.890439 0.891100 0.891759
#>  [673] 0.892416 0.893071 0.893724 0.894375 0.895024 0.895671 0.896316 0.896959
#>  [681] 0.897600 0.898239 0.898876 0.899511 0.900144 0.900775 0.901404 0.902031
#>  [689] 0.902656 0.903279 0.903900 0.904519 0.905136 0.905751 0.906364 0.906975
#>  [697] 0.907584 0.908191 0.908796 0.909399 0.910000 0.910599 0.911196 0.911791
#>  [705] 0.912384 0.912975 0.913564 0.914151 0.914736 0.915319 0.915900 0.916479
#>  [713] 0.917056 0.917631 0.918204 0.918775 0.919344 0.919911 0.920476 0.921039
#>  [721] 0.921600 0.922159 0.922716 0.923271 0.923824 0.924375 0.924924 0.925471
#>  [729] 0.926016 0.926559 0.927100 0.927639 0.928176 0.928711 0.929244 0.929775
#>  [737] 0.930304 0.930831 0.931356 0.931879 0.932400 0.932919 0.933436 0.933951
#>  [745] 0.934464 0.934975 0.935484 0.935991 0.936496 0.936999 0.937500 0.937999
#>  [753] 0.938496 0.938991 0.939484 0.939975 0.940464 0.940951 0.941436 0.941919
#>  [761] 0.942400 0.942879 0.943356 0.943831 0.944304 0.944775 0.945244 0.945711
#>  [769] 0.946176 0.946639 0.947100 0.947559 0.948016 0.948471 0.948924 0.949375
#>  [777] 0.949824 0.950271 0.950716 0.951159 0.951600 0.952039 0.952476 0.952911
#>  [785] 0.953344 0.953775 0.954204 0.954631 0.955056 0.955479 0.955900 0.956319
#>  [793] 0.956736 0.957151 0.957564 0.957975 0.958384 0.958791 0.959196 0.959599
#>  [801] 0.960000 0.960399 0.960796 0.961191 0.961584 0.961975 0.962364 0.962751
#>  [809] 0.963136 0.963519 0.963900 0.964279 0.964656 0.965031 0.965404 0.965775
#>  [817] 0.966144 0.966511 0.966876 0.967239 0.967600 0.967959 0.968316 0.968671
#>  [825] 0.969024 0.969375 0.969724 0.970071 0.970416 0.970759 0.971100 0.971439
#>  [833] 0.971776 0.972111 0.972444 0.972775 0.973104 0.973431 0.973756 0.974079
#>  [841] 0.974400 0.974719 0.975036 0.975351 0.975664 0.975975 0.976284 0.976591
#>  [849] 0.976896 0.977199 0.977500 0.977799 0.978096 0.978391 0.978684 0.978975
#>  [857] 0.979264 0.979551 0.979836 0.980119 0.980400 0.980679 0.980956 0.981231
#>  [865] 0.981504 0.981775 0.982044 0.982311 0.982576 0.982839 0.983100 0.983359
#>  [873] 0.983616 0.983871 0.984124 0.984375 0.984624 0.984871 0.985116 0.985359
#>  [881] 0.985600 0.985839 0.986076 0.986311 0.986544 0.986775 0.987004 0.987231
#>  [889] 0.987456 0.987679 0.987900 0.988119 0.988336 0.988551 0.988764 0.988975
#>  [897] 0.989184 0.989391 0.989596 0.989799 0.990000 0.990199 0.990396 0.990591
#>  [905] 0.990784 0.990975 0.991164 0.991351 0.991536 0.991719 0.991900 0.992079
#>  [913] 0.992256 0.992431 0.992604 0.992775 0.992944 0.993111 0.993276 0.993439
#>  [921] 0.993600 0.993759 0.993916 0.994071 0.994224 0.994375 0.994524 0.994671
#>  [929] 0.994816 0.994959 0.995100 0.995239 0.995376 0.995511 0.995644 0.995775
#>  [937] 0.995904 0.996031 0.996156 0.996279 0.996400 0.996519 0.996636 0.996751
#>  [945] 0.996864 0.996975 0.997084 0.997191 0.997296 0.997399 0.997500 0.997599
#>  [953] 0.997696 0.997791 0.997884 0.997975 0.998064 0.998151 0.998236 0.998319
#>  [961] 0.998400 0.998479 0.998556 0.998631 0.998704 0.998775 0.998844 0.998911
#>  [969] 0.998976 0.999039 0.999100 0.999159 0.999216 0.999271 0.999324 0.999375
#>  [977] 0.999424 0.999471 0.999516 0.999559 0.999600 0.999639 0.999676 0.999711
#>  [985] 0.999744 0.999775 0.999804 0.999831 0.999856 0.999879 0.999900 0.999919
#>  [993] 0.999936 0.999951 0.999964 0.999975 0.999984 0.999991 0.999996 0.999999
#> [1001] 1.000000
#> 
#> $mu
#> [1] 0.5
#> 
#> $auc
#> [1] 0.833333
#>