Estimands – What you need to know

Over the last few years, Estimands have become an increasingly hot topic for statisticians within our industry. In August 2017, the first draft of The International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) E9 (R1) addendum was released for consultation. The addendum was entitled ʽEstimands and sensitivity analysis in clinical trialsʼ and is expected to be finalised in the coming months. Pharmaceutical companies, regulatory agencies and clinical research organisation (CROs) are working in implementing this addendum into their working practices. It is a time of learning and change for all.

What Are Estimands?

Clinical trials are conducted to measure the effects of a treatment and provide evidence of the safety and efficacy of the treatment under investigation. Trials are generally designed to have balanced treatment groups of patients that will complete the trial as per the protocol, providing complete data. However, in reality, this is often not the case. Some patients may use alternative treatments during the course of the trial, others may stop treatment or switch to an alternative, and there may be deaths. These are known as intercurrent events.
The addendum aims to ensure that these events are taken into account during the planning and conduct of clinical trials. An estimand is a way to explicitly state how intercurrent events will be dealt with and ensure the objective of the trial, the design, data collection, and analyses are consistent with this approach. It defines in detail what needs to be estimated to address a specific scientific question of interest. This will add precision to the research question that is being asked with the aim of improving the quality of clinical research.

For a more detailed discussion on what Estimands are listen to our statistical consultants discuss the topic in the podcast below:

The Four Elements of the Estimands Framework

An estimand has 4 attributes: population, the variable, the specification of how to account for intercurrent events, and the population level summary, as detailed below:

• The population is the target population for the research question. This is generally the population of people who would be eligible for the trial. For example, patients with severe hayfever, aged 18 and over. This is the wider population of potential patients, rather than the population of patients recruited into the trial.
• The variable is the endpoint that will be obtained from all patients. For example, a laboratory measurement, or change from baseline of a specific measurement, a response or non-response to treatment, time to a clinical event of importance, etc. Such endpoints have typically been clearly defined in protocols prior to the addendum, so no change to current practice is required.
• An intercurrent event is one that occurs after treatment initiation and either precludes the observation of the variable, or affects its interpretation. For example, a patient requires rescue medication during a clinical trial, but continues with the trial post-rescue medication. In this case the measurements of the variable following rescue medication will have been affected by the taking of the medication. The rescue medication may be ignored, in which case the trial is assessing the efficacy of the trial treatment regardless of the need for rescue medication. Alternatively, data post-rescue medication could be excluded from the analyses, in which case the trial is assessing the efficacy of the trial medication until rescue medication is required. How the rescue medication is treated will change the research question being asked. It is therefore important that the handling of intercurrent events is carefully considered and justified within the protocol. The addendum outlines 5 strategies for addressing intercurrent events. These strategies will be discussed later.
• The population –level summary is the variable on which the comparison between treatments will be based. This could be the difference in mean change from baseline between groups, or a ratio of one group compared to the other in a response rate. Prior to the addendum the population level summary would typically be defined in the statistical section of the analyses plan. Using the new framework this information will be included within the estimand section of the protocol.

These 4 attributes together describe the estimand. They are interrelated and so must be considered together.

Source: Estimands and Sensitivity Analyses: What’s in the new ICH E9 Addendum?
Chrissie Fletcher, 1 Day PSI Meeting on Estimands, 27th Sept 2017

Treatment Strategies for Intercurrent Events

As mentioned above, the way in which intercurrent events are handled needs careful consideration. Five strategies are defined in the addendum. These strategies can be used independently or in combination.

• Treatment Policy strategy.
• Composite strategy.
• Hypothetical strategy.
• Principal Stratum strategy.
• While on Treatment strategy.

Treatment Policy

Under the Treatment Policy, whether an intercurrent event has occurred or not is irrelevant, the data will be collected and analysed regardless. For example, if a patient took rescue medication, or discontinued from the trial, the data post the event would be included in the analyses. This policy reflects the intention‑to‑treat (ITT) principle and is intended to measure the effectiveness of the treatment in ‘real world’ conditions where patients will not adhere perfectly to the protocol when taking medication. The intercurrent events would need to be identified in the protocol, and following an event such as trial treatment discontinuation, the patient would be required to remain in the clinical trial so that the endpoint could be measured until the end of the treatment period. This policy is not appropriate for an intercurrent event such as death, for which the measurement would not exist after the event.

The advantage of this policy is that it reflects more closely the real-life situation, but the disadvantage is that the intercurrent events will increase the variability of the data, making a statistically significant treatment effect less likely. It may also be difficult to obtain endpoint measurements from patients who have discontinued trial treatment, as they may have little motivation to remain in the trial.

Figure 1: Treatment Policy

Composite Strategy

For the composite strategy the occurrence of the intercurrent event is incorporated into the endpoint. For example, if patients are assessed as being responders or non‑responders, then the use of rescue medication could be combined with this to give a composite endpoint of responder without the use of rescue medication. So patients who respond, but use rescue medication would be considered as non‑responders. For a numerical endpoint, an unfavourable value could be imputed if a rescue medication was used.

This approach has been used in studies prior to the framework, for example imputing patients to be non‑responders if they took a prohibited concomitant medication during the treatment period. However, the rules for deriving the endpoint would have been included in the statistical section of the RAP, rather than prominently in the protocol.

Figure 2: Composite Strategy

Hypothetical Strategy

Under the hypothetical strategy, the aim of the trial is to estimate the treatment effect in the hypothetical situation where the intercurrent event did not occur. For example, if the intercurrent event was trial discontinuation, what would the endpoint measurement have been if the patient had not discontinued? The measurement is hypothetical and so needs to be estimated, and the statistical methodology for such estimation needs to be clearly defined.

This approach must be justified in the protocol, and the hypothetical situation clearly defined. Where patients discontinue treatment because an alternative treatment has become available, or they have moved away from the trial centre, this strategy may be reasonable. However, if patients were to discontinue treatment due to an adverse event (AE), it would not be appropriate to consider the hypothetical case. As in the real‑life situation, such patients would not continue taking the treatment, which is important information when considering how effective the treatment is.

This strategy has the advantage that some of the variability in the data will be removed, and so there may be more likelihood of observing a statistically significant treatment effect. However, it has the disadvantage of using imputed data based on assumptions, and is not representing the ‘real’ world as closely as other strategies.

Figure 3: Hypothetical Strategy

Principal Stratum Strategy

With the Principal Stratum Strategy, the aim is to estimate the treatment effect within the stratum of patients for whom the intercurrent event did not occur. The stratum of patients cannot be identified prior to the start of the trial because it is not possible to tell who will have the intercurrent event.

As an example, imagine a trial treatment that causes vomiting in a proportion of patients, but for those who do not experience vomiting the treatment is generally well tolerated. In this case the intercurrent event is discontinuation of the trial treatment due to the AE of vomiting. Whether or not the treatment is efficacious among these patients is irrelevant as they would not wish to continue taking the medication. The aim of the trial, therefore, is to estimate the treatment effect in those patients who did not experience the intercurrent event.

There are certain scenarios where this approach is very useful, but it will need to be carefully justified in the protocol.

Figure 4: Principal Stratum Strategy

While on Treatment Strategy

This strategy considers measurements of the endpoint up until the time of the intercurrent event. This approach is relevant where the endpoint is measured at multiple time points. The strategy uses the measurements up until the time of the event, and the trial would be designed in such a way as to stop taking measurements post the event.

This strategy would not be appropriate in many cases where an intercurrent event may well be indicative that the medication was no longer going to be of benefit. However, in a scenario where a treatment was intended to give pain relief for a terminal illness, the While on Treatment strategy is justified.

Figure 5: While On Treatment Strategy

Constructing Estimands

When constructing estimands, the first stage is to identify the intercurrent events that are likely to occur in this trial setting. For each intercurrent event, the treatment strategy needs to be decided. A combination of strategies can be used if appropriate. The whole trial team should be involved in this process; it is not just a statistical issue. Justification of the strategies used should be included in the protocol.

Estimands can then be written using the 4 attributes discussed earlier: population, endpoint, how to handle intercurrent events, and population level summary. The trial, data collection, and analyses are designed in line with the estimands, to ensure that the scientific questions posed will be addressed by the trial. Any assumptions made should be clearly defined.

Sensitivity and Supplementary Analyses

Any assumptions made by the statistical methodology should be outlined in the statistical analyses plan. Sensitivity analyses should be planned to test the robustness of these assumptions. Supplementary analyses can be used to investigate additional estimands.

Missing Data

The estimand framework will impact how missing data are defined and methodology used to deal with missing data. For example, if a patient discontinues trial medication, then under Treatment Policy, the measurements following discontinuation are required, and if not collected would be considered missing. Under the hypothetical strategy the measurements will not be collected, but should be replaced with the hypothetical measurements expected if that patient had not discontinued the trial medication. However, under While on Treatment, those measurements would be of no interest, and therefore not considered missing data, as the intention was not to collect them.

Where missing data are imputed the assumptions used for the imputation must be clearly defined in the SAP. Sensitivity analyses should be used to assess the assumptions.

Summary

Most of the considerations that have been described above would be part of the planning of a clinical trial under current practice. However, much of the detail would be included in the SAP, rather than upfront in the protocol, and the justification for the approach used, and assumptions behind it are seldom included.

The framework in the addendum provides a structure to ensure that the intercurrent events are identified and appropriately dealt with in the trial design and analyses. The estimands section of the protocol will bring the information to the forefront and make it easier for all concerned to understand the research question being asked, and to ensure the trial has been conducted in line with that.

Quanticate's statistical consultants are among the leaders in their respective areas and have supported several clients with Estimands. Clients have the ability to choose expertise from a range of consultants to match their needs. If you require more information or support with Estimands or other services please Submit a RFI and member of our team will be in touch with you shortly.

Related Blogs:

• An Introduction to Estimands [Podcast]
• An Introduction to Missing Data in Clinical Trials
• Multiple Imputation for Handling Missing Data in Clinical Trials

REFERENCES

The International Council of Technical Requirements for Pharmaceuticals for Human Use (ICH)
http://www.ich.org/ichnews/press-releases/view/article/ich-releases-finalised-draft-guidelines-andpublishes-working-party-membership-for-first-time.html

Strategies for composite estimands in confirmatory clinical trials: Examples from trials in nasal 
polyps and steroid reduction - https://onlinelibrary.wiley.com/doi/full/10.1002/pst.1909

Bayesian inference for a principal stratum estimand to assess the treatment effect in a subgroup characterized by post-randomization events - https://arxiv.org/pdf/1809.03741.pdf