Friday, January 13, 2012

Checklist for reviewing pilot studies

Pilot Study Checklist

At least one of the following reasons to conduct the study applies
·      Study administration
·      Data management
·      Scientific

Aims and objectives are clearly stated

Collected data are consistent with goals

No statistical hypothesis is tested

Sample size is justified (not necessarily in a statistical sense)

The way in which collected will be used in the design of a larger study has been addressed

This study will answer the question on whether a full scale trial/experiment is worth pursuing

Criteria that will determine continuation of a larger study are specified

Other recommendations and thoughts

Distinguish between
·      Toy problems for learning experience
·      Identifying feasibility answers
·      Getting “preliminary data” (why?)

Always identify
·      What the goal of the study is
·      How it will be determined whether the goal is met
·      What steps need to be taken to meet the goal

What is innovative or novel.

A poorly designed study can be used as an example to train investigators in designing experiments.

Example of a badly designed study

Prior to a larger clinical trial, a pilot study involving 10 patients is proposed. The goal of the project is to investigate how well patients will tolerate wearing a new ambulatory heart monitor while receiving an experimental medication. Data will be downloaded from the monitors and will be analyzed using a t-test procedure for comparison of post-treatment heart rate to pre-treatment heart rate.

·      The main goal was to assess tolerability but there is no measure defined for it. No data to address the main goal is collected.
·      There is an implicit hypothesis tested.
·      Sample size is not justified and may be driven by the desire to detect a difference.

If the goal is to estimate tolerability, where tolerability is defined as a dichotomous variable. Then the investigator needs to describe a criterion for unacceptable tolerability. The closer the intolerability is to zero, the larger the sample size needs to be.

Suggest a two tiered review. Statistician need only review protocols that have scientific merit.


Moore CG, Carter R, Nietert PJ, Stewart PW (2011). Recommendations for planning pilot studies in clinical and translational research. Clinical and Translational Science, 4(5): 332-337.

No comments:

Post a Comment

Subscribe via email

Enter your email address:

Delivered by FeedBurner