Determining sample size for progression criteria using hypothesis testing in pragmatic pilot RCTs

Lewis, Martyn; Bromley, Kieran; McCray, Gareth; Sutton, Christopher; Lancaster, Gillian

Abstract

Introduction: The current CONSORT guidelines for reporting of pilot
trials do not recommend hypothesis testing of clinical outcomes on
the basis of a pilot trial being under-powered to detect such differences and this being the aim of the main trial. It states that primary evaluation should focus on descriptive analysis of feasibility or process outcomes (e.g. recruitment, adherence, treatment fidelity).
Whilst the argument for not testing clinical outcomes is justifiable,
the same does not necessarily apply to feasibility/process outcomes,
where differences may be large and detectable with small sample
sizes. Moreover, there remains much ambiguity around sample size
for pilot trials.

Methods: Many pilot trials adopt a ‘traffic light’ system for evaluating
progression to the main trial determined by a set of criteria set up a
priori. We can set up a hypothesis-testing approach focused around
this system that tests against being in the RED zone (unacceptable
outcome) based on an expectation of being in the GREEN zone (acceptable outcome).
Results. For example, in relation to treatment fidelity, if we assume
the upper boundary of the RED zone is 40% and the lower bound of
the GREEN zone is 70% (designating unacceptable/acceptable treatment fidelity, respectively), the sample size required for analysis
given 90% power and one-sided 5% alpha would be n=22 (intervention group alone). A larger sample size would increase the power to reject the RED signal in favour of potential progression.
Discussion: In general, more than one key process outcomes are
assessed for progression to a main trial; a composite approach would
be to appraise the rules of progression across all these key outcomes.
This methodology begins to provide a formal framework for
hypothesis-testing and sample size indication around process outcome evaluation for pilot RCTs

Citation

Lewis, M., Bromley, K., McCray, G., Sutton, C., & Lancaster, G. Determining sample size for progression criteria using hypothesis testing in pragmatic pilot RCTs. Presented at 5th International Clinical Trials Methodology Conference (ICTMC 2019), Brighton, UK

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 5th International Clinical Trials Methodology Conference (ICTMC 2019)
• Online Publication Date: Oct 22, 2019
• Publication Date: Oct 22, 2019
• Deposit Date: Jun 21, 2023
• Volume: 20
• Pages: S1
• Public URL: https://keele-repository.worktribe.com/output/502668
• Publisher URL: https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-019-3688-6