Establishing robust metrics at the start of a study, and systems to monitor them consistently, should be integral to all work that a Clinical Research Organisation (CRO) performs in any sponsor vendor relationship. In this context, ‘clinical metrics’ is a broad term for measurable indicators, while key performance indicators, or KPIs, are the smaller subset linked most directly to strategic priorities, governance, or scorecard reporting.
A lot of performance metrics need to be “balanced” between what is controllable by the CRO and what is controllable by the sponsor. Therefore, the metrics that are incorporated into a contract need to be carefully chosen. Ideally it should be a small and accurate set of performance metrics that will meet the customer’s needs and enforce efficient information flow. That said, contact governance is only one use case. Performance metrics are also used more broadly within trial operations to understand whether processes are working as intended and where intervention is needed.
When defining what metrics will be collected, it is critical to carefully consider the reasons and benefits, as the collection of too many metrics usually proves disadvantageous. Metrics should enable a team to increase productivity, work smarter, and make better decisions. They should also measure success of the sponsor as well as the vendor because delays or deviations from the original plan can be caused by both.
What are Clinical Trial Performance Metrics and KPIs?
A metric is simply a measurement. In clinical trials, metrics are used to assess whether a process, team, site, or study activity is performing efficiently and effectively. A KPI is a metrics that has been selected because it matters most to oversight and decision-making. Not every metric needs to be a KPI, and treating every measurement as a KPI usually creates noise rather than clarity.
Practically, teams usually use clinical trial performance metrics in three ways:
-
To monitor timeliness and throughput
- To assess quality, compliance, or risk
- To understand efficiency, cost, or operational burden
Challenges of Metrics Development
By considering the following, it is possible to avoid many of the pitfalls associated with metric development:
- Too many metrics create excessive overheads and bureaucracy.
- Complex metrics are difficult to explain and translation into actions can be difficult.
- Data should be accurate and complete. If not, they should not be collected.
- The collection of metrics should not affect the outcome; it is not uncommon for people to act in a way contrary to the best interests of the business in order to make baseline or captured data more acceptable.
- Be consistent; the same items should be measured in the same way for all studies. For example, definitions should be consistent across different clinical studies and programmes.
- Not all that counts can be counted and not all that can be counted counts.
Defining Clinical Trial Metrics
In order for metrics to be useful, they should be clearly defined at the outset. For ease of tracking they should be named, standardised, and grouped in a way that reflects how the study is managed in practice. Common groupings include start-up and cycle time metrics, recruitment and retention metrics, quality and protocol adherence metrics, data timeliness metrics, and cost or resource-use metrics. Although almost anything can be measured, when defining metrics it is important to ensure that effort is spent measuring things that can be changed and which make a difference to the overall success of a project.
A metric should be used as an instrument to measure the effectiveness of a process. That means the team should be clear on why a metric is being collected, who will use it, what comparator makes it meaningful, and what action should follow if it moves out of range.
Before any measurements begin, it is important to consider whether fewer metrics could be measured to achieve the same endpoint. Careful consideration of the data collected will not only allow for process change, but also the definition of more effective metrics in the future.
When defining metrics, the user should consider whether the indicator measured is ‘leading’ or ‘lagging’. For clinical trials, these terms can be considered to relate to whether the end user will use the metric to identify opportunities in the current trial or to identify opportunities in future trials, respectively.
Consideration should be given to exactly how the metric will be measured. Additional analysis may be performed on a ‘for cause’ basis after consideration of whether/why a metric is important and to further define which processes should be examined if the metric is out of range.
The reporting frequency should be clearly defined; it is important to gather sufficient data to allow accurate and meaningful analysis but to not collect data too frequently. The target, or range you are trying to achieve, should also feature in metrics design.
Clinical Trial Performance Metrics Across the Study Lifecycle
Clinical trial performance metrics are usually most useful when mapped to where the study actually is. In start-up, teams often focus on activation and contracting timelines. During recruitment and conduct, attention typically shifts to the operational measures that best show whether the trial is running as planned. Near closeout, the focus usually moves to data finalisation and reporting readiness.
Example Metrics Collected in Clinical Trials
| Measure | Category | Indicator | Vendor | Sponsor | Site |
|
Final approved protocol to final approved Case Report Form (CRF) |
Time | LAGGING | X |
||
| Protocol approval to first site activated | Time | LEADING | X | ||
| Final protocol approval to first patient first visit (FPFV ; all sites) | Time | LEADING | X | ||
| Final CRF/electronic CRF (eCRF) to database (DB) Live | Time | LAGGING | X | ||
| CRFs received to data entry complete (paper) | Time | LEADING | X | ||
| Patient visits complete to eCRF data entered (eCRF) | Time | LEADING | X | ||
| Number of queries per 100 CRF pages | Quality | LAGGING | X | ||
| Receipt of query response to DB update time | Time | LEADING | X | ||
| Visits according to clinical monitoring plan | Quality | LAGGING | X | ||
| Last patient last visit (LPLV) to DB Lock | Time | LAGGING | X | ||
| DB lock to final Tables, Figures and Listings (TFLs) | Time | LAGGING | X | ||
| DB lock to final Clinical Study Report (CSR) | Time | LAGGING | X | ||
| Sponsor initiated scope changes | Cost | LAGGING | X | ||
| Clinical Research Organisation (CRO) initiated scope changes | Cost | LAGGING | X | ||
| Sponsor satisfaction - metrics generated per company procedures | Quality | LAGGING | X | ||
| Invoice payment timelines | Cost | LAGGING | X | ||
| Costs incurred relative to study progress (e.g. cash flow, performance) | Cost | LEADING | X |
LEADING: End user will use metric to identify opportunities in current trial
LAGGING: End user will use metric to identify opportunities in future trial
How to Use and Manage the Data
The collection and presentation of metrics should be kept simple. Data should be collected at regular intervals and reviewed individually and collectively. A traffic light approach on a dashboard is a useful way of monitoring performance against expectations and for monitoring standards and targets for improvement. Simple dashboards and scoreboards can be useful because they make drift visible quickly, but they only work well when thresholds, definitions, and reporting sources are agreed in advance. Some organisations also benchmark results against historical studies or internal comparators so that a raw number is not interpreted without context.
Specific Metrics Collected in Clinical Trials: Example 1
| Title | Category | Indicator |
|
Final protocol approval to first patient |
Time | LAGGING |
| Definition | ||
|
The total number of calendar days from the date of final approved protocol release to CRO to the date first visit for all sites (i.e. all sites screened on patient) Displayed as range from lowest to highest |
||
| Additional Analysis on a 'for cause' basis | Reporting frequency | Target |
| Analysis of the reasons for delay include timelines for ethics committee/signed site agreements and Competent Authority approval. Monitoring resources, protocol amendments, site contract issues identifies sites that potentially may not be used for future studies | Twice-monthly during site selection phase |
+ 4 weeks across the study (Green) Within 4-8 weeks across the study (Yellow) >8 weeks across the study (Red) |
LAGGING: End user will use metric to identify opportunities in future trial
Specific Metrics Collected in Clinical Trials: Example 2
| Title | Category | Indicator |
| Receipt of query response to database update time | Time | LEADING |
|
Definition Displayed as range from lowest to highest |
||
| Additional Analysis on a 'for cause' basis | Reporting frequency | Target |
|
Cycle time in excess of target indicated less than optimum processes Work that is not prioritised or passed between too many staff members can cause delays |
Quarterly |
2-3 days (Green) 4-5 days (Yellow) >5 days (Red) |
LEADING: End user will use metric to identify opportunities in current trial
Clinical Trial Performance Metrics and Quality by Design
A useful current framing is to select metrics that help teams detect whether design and delivery choices are reducing avoidable problems that matter to study quality. In quality by design approaches, metrics are not collected for their own sake. They are used to understand whether important risks are being mitigated, whether trial complexity is creating avoidable burden, and whether operational issues such as protocol deviations, missed key assessments, early terminations, or major audit findings are moving in the right direction.
There are no universally mandated trial metrics for every organisation or every study, so neutral, consensus-based phrasing is usually the safest editorial posture.
Conclusion
Clinical trial performance metric collection and the process changes based upon findings, are often viewed differently by individuals within an organisation depending upon previous experience. With time and careful consideration invested at the outset, the right collection of performance metrics can lead to the identification of weaknesses in a process and the implementation of corrections to reduce time taken and associated costs and therefore improve quality. Without sufficient consideration at the outset, metrics collection and any subsequent actions can be onerous and have, at best, little benefit relative to the time and cost invested.
Defining clinical trial performance metrics clearly, with agreed definitions, owners, reporting cadence, and intended actions, can give great clarity to what progress reporting is required and the resulting information will be invaluable in helping to keep the project on track. It will also be very useful during a post-project analysis where lessons learned can be applied to the next project.
Quanticate has expensive experience across a range of outsourcing models depending on the preferred partnership with a vendor. As a biometric CRO our services range from statistical programming, biostatistical consulting, clinical data management, medical writing and pharmacovigilance. Within all our contracts and as part of our Coded to Care OATH, performance metrics are clearly defined and agreed with our clients to ensure high quality is delivered and client expectations are met. Please request a consultation if you would like to speak to a member of our team who can support your trial.
