| Title: |
Quantifying Sensitivity to Selection on Unobserved Covariates: Recasting the Coefficient of Proportionality within a Correlational Framework |
| Language: |
English |
| Authors: |
Kenneth Frank (ORCID 0000-0002-6116-5509); Qinyun Lin; Spiro Maroulis; Shimeng Dai, Contributor; Nicole Jess, Contributor; Hung-Chang Lin, Contributor; Yuqing Liu, Contributor; Sarah Maestrales, Contributor; Ellen Searle, Contributor; Jordan Tait, Contributor |
| Source: |
Grantee Submission. 2025. |
| Peer Reviewed: |
Y |
| Page Count: |
80 |
| Publication Date: |
2025 |
| Sponsoring Agency: |
Institute of Education Sciences (ED) |
| Contract Number: |
R305D220022 |
| Document Type: |
Speeches/Meeting Papers; Reports - Evaluative |
| Descriptors: |
Statistical Analysis; Correlation; Predictor Variables; Inferences |
| Abstract: |
Sensitivity analyses can inform evidence-based education policy by quantifying the hypothetical conditions necessary to change an inference. Perhaps the most prevalent index used for sensitivity analyses is Oster's (2019) Coefficient of Proportionality (COP). Oster's COP leverages changes in estimated effects and R[superscript 2] when observed covariates are added to a model to quantify how strong selection on "unobserved covariates" would have to be relative to on "observed covariates" to nullify an estimated effect. In this paper, we reconceptualize the COP as a function of unobserved covariates' correlations with the focal predictor (e.g., treatment) and with the outcome. Our correlation-based approach addresses recent critiques of Oster's COP while preserving the comparison of selection on unobserved covariates to selection on observed covariates. As importantly, our expressions do not depend on an analyst's subjective choice of covariates to include in a baseline model, reproduce the exact results from Ordinary Least Squares (OLS) estimates even in finite samples, can be adapted to a threshold for inference based on statistical significance, and can be directly calculated from conventionally reported quantities (e.g., estimated effect, standard error) through the Konfound packages in R or Stata. Thus, for most published studies in the social sciences our COP index can be easily applied and intuitively interpreted. |
| Abstractor: |
As Provided |
| IES Funded: |
Yes |
| Entry Date: |
2025 |
| Accession Number: |
ED674603 |
| Database: |
ERIC |