In a case study examined to look at Multiple Imputation (MI) in clinical trials, comparing Active to Placebo treatment (at Weeks 2, 4, 6 and 12 of the trial) in adolescents with acne, drop outs were common. The primary endpoint was the number of lesions at Week 12. The factors believed to affect the propensity to be missing included age, side effects and lack of efficacy, and thus missing data patterns differ between groups.
It is common for datasets of this type to be analysed using an analysis of covariance (ANCOVA) of last observation carried forward (LOCF) data. Multiple imputation (MI) methods can be programmed in PROC MI in SAS Version 9.3 offering an alternative method to deal with missing data; we explore the MI process, compare results with LOCF ANCOVA and a mixed models repeated measures (MMRM) and ask is it worth the effort?
A simulation of 1000 data sets was carried out by removing data randomly from a completer dataset (N=131) using propensity scores based on the pattern of missing data observed in the full dataset (N=153). Least squares (LS) means and differences were estimated with standard errors (SE). Boxplots are presented of the bias and relative SE from MI compared to LOCF ANCOVA and a MMRM approach without imputation of data; these are relative to the ANCOVA on the completer dataset.
We focus on the least biased of several methods of MI tested: Predictive Mean Matching (PMM) which imputes values by sampling from k observed data points closest to a regression predicted value where the regression parameters are sampled from a posterior distribution. The total variance of combined ANCOVA results (see Figure 1) is calculated from the average within-imputation (W) and between-imputation variance (B). , 
Figure 1: Flow chart of Multiple Imputation Process
MMRM is the least biased and LOCF the most biased of the three methods (Figure 2). Relative SEs were largest for PMM (Figure 3).
Figure 2: Bias in LS Means of Estimate
Figure 3: Relative standard error of difference in treatment means
Both figures show distribution from 1000 simulations (data were removed randomly based on propensity scores; the propensity model included age, side effect of pain after treatment and efficacy measured by lesion counts). Bias and relative standard errors are relative to the completer dataset.
The Food and Drug Administration (FDA) were critical of the use of LOCF in Phase 3; it assumes no trend of response over time resulting in bias and a distorted covariance structure. All methods in PROC MI and MMRM make the assumption that data are Missing at Random (MAR). PROC MI has useful functionality in summarising the missing data patterns.
MI is complex to define a priori as there are many details to consider (see Figure 1) and additional data processing steps are necessary. The PMM method of imputation has the advantage over alternative MI methods in that no bounds, rounding or post-imputation manipulation is required to give plausible imputed lesion counts.
Sensitivity analyses can investigate a range of delta (δ) values added to imputed values to explore the robustness of conclusions to imputation.
Relative SEs were generally greater than 1 for all methods, this is to be expected given the loss of approximately 15% of data from the completer dataset by using the propensity scores in the simulation of missing values. The SEs from MI techniques incorporate an additional component (B) to account for the uncertainty in the imputation, whereas LOCF ignores this uncertainty. However, the resulting SE from MI is appreciably larger than that from MMRM, and thus this MI method has less power.
MI is complex to define and computationally intensive and thus would need to have substantial benefits to be worth the effort for a primary analysis. We found PMM to have less power than MMRM without reducing bias. Therefore, we recommend: MMRM as the primary analysis; use of PROC MI to investigate the sensitivity (delta method); and avoiding LOCF. Further work could investigate scenarios such as data not being MAR, varying k and whether the default burn-in of 20 in PMM is sufficient.
 SAS/STAT(R) 12.1 User's Guide, "The MIANALYZE Procedure, Combining Inferences from Imputed Data Sets," [Online]. Available:
 D. Rubin, Multiple Imputation for Nonresponse in Surveys, New York: John Wiley & Sons, 1987.
This article was featured as a poster at the PSI 2014 annual conference.