ESTIMATING MEASUREMENT ERROR AND SCORE DEPENDABILITY IN EXAMINATIONS USING GENERALIZABILITY THEORY
ABSTRACT
This study investigated the estimation of measurement error and score dependability in examinations using Generalizability Theory. The fact that the scores obtained by the students (object of measurements) in examinations were affected by multiple sources of error (facets) and these scores, were used in taking relative and absolute decisions about the students, there was the need to estimate measurement error and dependability of the scores. This was to find the extent of the contributions of these sources of error (facets) in examination scores. Four research questions and two hypotheses were posed to guide the study. The population of the study comprised 25,530 senior secondary three (SS3) students in public secondary schools in Rivers State for 2011/2012 academic year. The sample consisted of 2,553, SS3 students selected through the proportionate stratified random sampling technique. A Mathematics Achievement Test with items drawn from past WAEC and NECO SSCE questions was used for data collection. EduG version 6.0-e based on ANOVA and Generalizability theory was used to answer the four research questions. A 95% confidence interval was computed using the S E variance components to determine whether there was a significant difference in the contributions and effects of the facets and their interactions to measurement error and score dependability in examinations. The findings of the study revealed that some hidden sources of error were at play in the study. The residual made the highest contribution to measurement error. This was followed by the student factor. Similarly, the residual and the students variance components were significantly (p < 0.05) different prin their contributions to measurement error in examination scores. Conversely, questions and invigilators were not significantly different in their contributions and effects on measurement error and score dependability in examinations (p > 0.05). Invigilators and the interaction of students x invigilators has the highest effect on score dependability in examinations. The findings also revealed that an increase of invigilators to 90, increased the generalizability coefficient (EP2) and index of dependability (Ø) which rank ordered students and classified them based on their performance, irrespective of the performance of other students. Therefore, generalizability theory provided a framework for evaluating multiple sources of variability in examination scores and for deriving implications for test development and test score interpretation. It is therefore recommended that in conducting examinations, enough invigilators should be recruited so as to minimize error and maximize score reliabilities.