| Title: |
The Reliability of Classroom Observations and Student Surveys in Non-Research Settings: Evidence from Argentina. EdWorkingPaper No. 25-1322 |
| Language: |
English |
| Authors: |
Alejandro J. Ganimian; Andrew D. Ho; Alejandra Campos Quintero; Annenberg Institute for School Reform at Brown University |
| Source: |
Annenberg Institute for School Reform at Brown University. 2025. |
| Availability: |
Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/ |
| Peer Reviewed: |
N |
| Page Count: |
44 |
| Publication Date: |
2025 |
| Sponsoring Agency: |
Inter-American Development Bank (IDB) |
| Document Type: |
Reports - Research |
| Education Level: |
Higher Education; Postsecondary Education |
| Descriptors: |
Foreign Countries; Clinical Supervision (of Teachers); Student Evaluation of Teacher Performance; Teacher Effectiveness; Evaluation Methods; Student Surveys; Classroom Observation Techniques; Observation; Response Rates (Questionnaires); Research Problems; Feedback (Response); Preservice Teachers |
| Geographic Terms: |
Argentina (Buenos Aires) |
| Abstract: |
There is a growing consensus on the need to measure teaching effectiveness using multiple instruments. Yet, guidance on how to achieve reliable ratings derives largely from formal research in high-income countries. We study the reliability of classroom observations and student surveys conducted by practitioners in a middle-income country. Both instruments can achieve relatively high reliability (0.6-0.8 on a 0-1 scale) when averaged across raters and occasions, but the reliability of observations varies widely (from 0.4 to 0.8) based mostly on how raters are assigned to teachers. We use Generalizability Theory to estimate how reliability improves by increasing the number of times teachers are observed or the number of respondents to surveys. We recommend that practitioners design their teacher feedback systems based on analyses of their own data, instead of assuming that instruments and rubrics will generate scores with the same reliability as research settings. |
| Abstractor: |
As Provided |
| Entry Date: |
2026 |
| Accession Number: |
ED678303 |
| Database: |
ERIC |