| Title: |
Disentangling Person-Dependent and Item-Dependent Causal Effects: Applications of Item Response Theory to the Estimation of Treatment Effect Heterogeneity |
| Language: |
English |
| Authors: |
Joshua B. Gilbert (ORCID 0000-0003-3496-2710); Luke W. Miratrix (ORCID 0000-0002-0078-1906); Mridul Joshi; Benjamin W. Domingue (ORCID 0000-0002-3894-9049) |
| Source: |
Journal of Educational and Behavioral Statistics. 2025 50(1):72-101. |
| Availability: |
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com |
| Peer Reviewed: |
Y |
| Page Count: |
30 |
| Publication Date: |
2025 |
| Document Type: |
Journal Articles; Reports - Research |
| Education Level: |
Elementary Education; Early Childhood Education; Grade 2; Primary Education |
| Descriptors: |
Causal Models; Item Response Theory; Statistical Inference; Psychometrics; Educational Assessment; Randomized Controlled Trials; Reading Instruction; Intervention; Elementary School Students; Grade 2; Reading Tests; Pretesting; Correlation; Effect Size; Test Items; Item Analysis; Difficulty Level; Test Construction |
| DOI: |
10.3102/10769986241240085 |
| ISSN: |
1076-9986; 1935-1054 |
| Abstract: |
Analyzing heterogeneous treatment effects (HTEs) plays a crucial role in understanding the impacts of educational interventions. A standard practice for HTE analysis is to examine interactions between treatment status and preintervention participant characteristics, such as pretest scores, to identify how different groups respond to treatment. This study demonstrates that the identical patterns of HTE on test score outcomes can emerge either from variation in treatment effects due to a preintervention participant characteristic or from correlations between treatment effects and item easiness parameters. We demonstrate analytically and through simulation that these two scenarios cannot be distinguished if analysis is based on summary scores alone. We then describe a novel approach that identifies the relevant data-generating process by leveraging item-level data. We apply our approach to a randomized trial of a reading intervention in second grade and show that any apparent HTE by pretest ability is driven by the correlation between treatment effect size and item easiness. Our results highlight the potential of employing measurement principles in causal analysis, beyond their common use in test construction. |
| Abstractor: |
As Provided |
| Entry Date: |
2025 |
| Accession Number: |
EJ1457138 |
| Database: |
ERIC |