Minimum sample size calculations for external validation of a clinical prediction model with a time-to-event outcome.

Riley, Richard D.; Collins, Gary S.; Ensor, Joie; Archer, Lucinda; Booth, Sarah; Mozumder, Sarwar I.; Rutherford, Mark J.; van Smeden, Maarten; Lambert, Paul C.; Snell, Kym I. E.

Abstract

Previous articles in Statistics in Medicine describe how to calculate the sample size required for external validation of prediction models with continuous and binary outcomes. The minimum sample size criteria aim to ensure precise estimation of key measures of a model's predictive performance, including measures of calibration, discrimination, and net benefit. Here, we extend the sample size guidance to prediction models with a time-to-event (survival) outcome, to cover external validation in datasets containing censoring. A simulation-based framework is proposed, which calculates the sample size required to target a particular confidence interval width for the calibration slope measuring the agreement between predicted risks (from the model) and observed risks (derived using pseudo-observations to account for censoring) on the log cumulative hazard scale. Precise estimation of calibration curves, discrimination, and net-benefit can also be checked in this framework. The process requires assumptions about the validation population in terms of the (i) distribution of the model's linear predictor and (ii) event and censoring distributions. Existing information can inform this; in particular, the linear predictor distribution can be approximated using the C-index or Royston's D statistic from the model development article, together with the overall event risk. We demonstrate how the approach can be used to calculate the sample size required to validate a prediction model for recurrent venous thromboembolism. Ideally the sample size should ensure precise calibration across the entire range of predicted risks, but must at least ensure adequate precision in regions important for clinical decision-making. Stata and R code are provided.

Citation

Riley, R. D., Collins, G. S., Ensor, J., Archer, L., Booth, S., Mozumder, S. I., …Snell, K. I. E. (2022). Minimum sample size calculations for external validation of a clinical prediction model with a time-to-event outcome. Statistics in Medicine, 41(7), 1280-1295. https://doi.org/10.1002/sim.9275

• Journal Article Type: Article
• Acceptance Date: Nov 16, 2021
• Online Publication Date: Dec 16, 2021
• Publication Date: Mar 30, 2022
• Journal: Statistics in Medicine
• Print ISSN: 0277-6715
• Publisher: Wiley
• Volume: 41
• Issue: 7
• Pages: 1280-1295
• DOI: https://doi.org/10.1002/sim.9275
• Keywords: calibration; external validation; prediction model; sample size; time-to-event & survival data
• Publisher URL: https://onlinelibrary.wiley.com/doi/10.1002/sim.9275