Bias in Real-world Endpoints when Conducting External Control Arm Analyses

Tue, 21 Apr 2026 00:00:00 +0000

Background

When augmenting single-arm trials with real-world data (RWD) for external control arm analyses, biases may arise due to differences in how and when patients’ disease progression is assessed between the two settings. This is particularly worrisome in oncology studies, where time-to-event endpoints like progression-free survival (PFS) rely on strict progression evaluation criteria collected at fixed assessment schedules. On the other hand, RWD assessments may be less frequent or regimented, and determination of disease progression may be made based solely on unstructured clinical documentation or more flexible guidelines than the trial standards.

Ackerman et al (2026) "Quantitative bias analyses to address measurement error in time-to-event endpoints." American Journal of Epidemiology.

These types of endpoint measurement error may cause bias (misclassification and surveillance bias), such that real-world PFS estimates appear longer (or shorter) than expected under trial settings, thus limiting the ability to make apples-to-apples comparisons between a single-arm trial and its real-world comparator arm.

Key Findings and Takeaways

It is often infeasible in ECA analyses to construct RWD comparators with endpoints aligned to trial standards, but:

Simulation studies can demonstrate when no bias is present due to measurement misalignments.
Methods can be applied to quantify and reduce statistically meaningful measurement error bias.
Quantitative Bias Analyses can contextualize biased ECA findings when measurement error bias is not directly estimable.

Robust statistical methods can adequately align RWD endpoints towards trial standards and reduce biases for regulatory approval of ECAs.

Relevant Papers

Ackerman, B., Gan, R.W., Meyer, C.S., Zhang, Y., Wang, J.R., Hayden, J., et al. (2026). Quantitative bias analyses to address measurement error in time-to-event endpoints. American Journal of Epidemiology.

Edwards, J.K., Cole, S.R., Zivich, P.N., Ackerman, B., Napravnik, S., Henderson, H., Lash, T.L., Shook-Sa, B.E. (2026). Risk functions with outcome measurement error. Biostatistics.

Ackerman, B., Gan, R.W., Zhang, Y., Siddique, J., Roose, J., Lund, J.L., et al. (2025). Regression calibration for time-to-event outcomes: Mitigating bias due to measurement error in real-world endpoints. Epidemiologic Methods.

Ackerman, B., Gan, R.W., Meyer, C.S., Wang, J.R., Zhang, Y., Hayden, J., et al. (2024). Measurement error and bias in real‑world oncology endpoints when constructing external control arms. Frontiers in Drug Safety and Regulation.

This work was presented at the 2025 EMA Workshop on the use of external controls for evidence generation in regulatory decision-making

Extending Inferences from Trials to Target Populations

Fri, 21 Apr 2023 00:00:00 +0000

Background

Randomized trials are often considered the gold standard for estimating causal effects and evaluating efficacy of new therapeutics. However, patients recruited for clinical trials are often not representative of the target populations of interest, and thus, findings from trials may not generalize or transport from the recruited sample population to broader populations.

Generalizability refers to whether inferences extend from the source study population to the target population from which it was sampled. Transportability refers to whether inferences extend from the source study population to a different population (e.g., one may wish to transport findings from a trial conducted in the US to a population of patients in Europe).

Both generalizability and transportability depend on representativeness, or the similarity between the trial participants and the target population with respect to key baseline factors. More specifically, without representativeness on characteristics that moderate the treatment effect, then trial findings may not extend to the target population of interest.

Key Findings and Takeaways

While it may not be feasible to always recruit trial participants that reflect the intended target populations,

Evaluating representativeness on key factors that modify the treatment effect can inform how well trial findings generalize or transport to target populations of interest.
Statistical methods, such as propensity score-type weighting or doubly-robust outcome modeling, can be applied to extend trial inferences to broader populations, using external population data as a reference frame.
Finding suitable target population data can be challenging in practice, and may require further statistical adjustments when extending inferences (e.g., when using a nationally representative complex health survey).

Relevant Papers

Vuong, Q., Metcalfe, R.K., Ling, A., Ackerman, B., Inoue, K., Park, J.J.H. (2025). Systematic review of applied transportability and generalizability analyses: A landscape analysis. Annals of Epidemiology.

Gerke, T., Ackerman, B., Long, L., Baxi, S., Miksad, R., Adamson, B., et al. (2021). Representativeness of real world data: a framework for assessing oncology EHR-derived data. SSRN.

Ackerman, B., Lesko, C.R., Siddique, J., Susukida, R., Stuart, E.A. (2020). Generalizing randomized trial findings to a target population using complex survey population data. Statistics In Medicine.

Lesko, C.R., Ackerman, B., Webster-Clark, M., Edwards, J.K. (2020). Target validity: bringing treatment of external validity in line with internal validity. Current Epidemiology Reports.

Ackerman, B., Schmid, I., Rudolph, K.E., Seamans, M.J., Susukida, R., Mojtabai, R., Stuart, E.A. (2019). Implementing statistical methods for generalizing randomized trial findings to a target population. Addictive Behaviors.

Nguyen, T.Q., Ackerman, B., Schmid, I., Cole, S., Stuart, E.A. (2018). Sensitivity analyses for effect modifiers not observed in the target population when generalizing treatment effects from a randomized controlled trial: Assumptions, models, effect scales, data scenarios, and implementation details. PLoS One.

Research Portfolio on Ben Ackerman, PhD

Bias in Real-world Endpoints when Conducting External Control Arm Analyses

Background

Key Findings and Takeaways

Relevant Papers

Extending Inferences from Trials to Target Populations

Background

Key Findings and Takeaways

Relevant Papers