Sensitivity Analysis for Unobserved Effect Modification when Generalizing Findings from Randomized Trials to Target Populations

Jun 10, 2019 3:30 PM — 4:00 PM
Washington, DC

Event description: This half-day workshop will inform efforts by the interagency Federal Committee on Statistical Methodology (FCSM) to improve information shared with the public about how agencies combine data from multiple sources to produce estimates, analyses and other data products. The workshop will provide insight into a critical aspect of integration processes, namely, identifying and evaluating critical assumptions made during integration, and the possible impact on results when those assumptions are changed. At this workshop, we will explore existing examples of such work, toward creating a broader understanding of techniques and issues involved. We will also emphasize a set of broader reporting issues and recommendations to help the public understand the implications of the different assumptions that are involved when integrating data from multiple sources into a single product. We will cover issues with evaluation and reporting of situations where 1) all sources have completely overlapping variables and we have to select a source, 2) the sources have some overlap on common variables but information has to be filled in across cases or domains with imputation and other techniques like statistical matching, 3) the sources are all providing unique information that has to be filled in for cases or domains not common to the initial data sources – again, relying on imputation, statistical matching, etc. – and 4) the sources have similar but not non-identical variables that must be expressed with a common metric (sometimes referred to as data harmonization). Integration procedures, sensitivity analyses, and reporting standards will vary depending on the combinations of data sources: multiple sample surveys, sample surveys with non-probability data, sample surveys and administrative data, and non-probability data with administrative data.