Within the sample Comparison of prediction performances of models and sub-models. Application to Alzheimer disease

Abstract : Abstract Our objective is to compare the predictive ability of several nested models. It stems from the following problem in epidemiology : the occurrence of a certain disease is to be predicted to happen within a fixed period of time thanks to the values of a number of items measured on the observed patients. It may happen that one or several items, proved to be relevant for the best fitting model, have a non significant contribution to the prediction of who is at risk of developing the disease. The indices we use to compare the respective predictive ability of two models are the Integrated Discrimination Improvement (IDI) and the BRier’s score Improvement (BRI). Estimation of the models and their relative IDI and BRI are conducted on the same sample, and their respective asymptotic properties are proved. We apply the results to Alzheimer disease on a French cohort.