Designing and Validating a CEFR-Aligned Speaking Rubric for Rural Primary ESL Learners
DOI:
https://doi.org/10.37134/ejoss.vol12.Sp.12.2026Keywords:
speaking rubric, CEFR, content validity , reliability, primary ESL learners, rural educationAbstract
This study reports the development and validation of a CEFR-aligned analytic speaking rubric designed for upper primary ESL learners in rural Malaysia. Adopting a quantitative instrument development and validation design, the study focused on establishing the rubric’s content validity and scoring reliability. Six TESL experts evaluated the relevance and clarity of the rubric descriptors, while a pilot study involving 30 upper primary pupils from two rural schools provided empirical scoring data. Content validity was examined using the Item-Level Content Validity Index (I-CVI) and Scale-Level Content Validity Index (S-CVI). The findings indicated that all rubric domains achieved acceptable I-CVI values (≥ 0.83), with an overall S-CVI of 0.93, demonstrating strong expert agreement. Scoring reliability was assessed through inter-rater reliability using Cohen’s Kappa (κ = 0.61–0.76), indicating moderate to substantial agreement, and internal consistency using Cronbach’s alpha (α = 0.84), reflecting good internal reliability. These results provide initial evidence that the rubric is conceptually sound and can be applied with reasonable scoring consistency. The study contributes a context-responsive, CEFR-aligned speaking assessment rubric that supports more systematic and transparent evaluation of speaking proficiency among rural primary ESL learners.
Downloads
References
Bachman, L. F., & Palmer, A. S. (2010). Language assessment in practice: Developing language assessments and justifying their use in the real world. Oxford University Press.
Butler, Y. G. (2018). English language education among young learners in East Asia: A review of current research. Language Teaching, 51(1), 1–34. https://doi.org/10.1017/S0261444817000232
Council of Europe. (2020). Common European Framework of Reference for Languages: Learning, teaching, assessment – Companion volume. Council of Europe Publishing. https://rm.coe.int/cefr-companion-volume-with-new-descriptors-2020
DeVellis, R. F. (2017). Scale development: Theory and applications (4th ed.). Sage Publications.
Hashim, H., & Yunus, M. M. (2020). English language learning challenges in rural schools and strategies to improve proficiency. Creative Education, 11(6), 1045–1055. https://doi.org/10.4236/ce.2020.116078
Hashim, H., & Yunus, M. M. (2020). ESL teachers’ perceptions on the use of speaking assessment in Malaysian primary schools. International Journal of Learning, Teaching and Educational Research, 19(6), 1–15. https://doi.org/10.26803/ijlter.19.6.1
Leong, L. M., & Ahmadi, S. M. (2017). An analysis of factors influencing learners’ English speaking skill. International Journal of Research in English Education, 2(1), 34–41. https://doi.org/10.18869/acadpub.ijree.2.1.34
McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276–282. https://doi.org/10.11613/BM.2012.031
Mohammed, S. A., Rahman, M. S., & Yunus, M. M. (2021). Teachers’ readiness in implementing CEFR-aligned English curriculum in rural Malaysia. Asian Journal of Education and Social Studies, 17(4), 21–34. https://doi.org/10.9734/AJESS/2021/v17i430422
Mohammed, A. A., Yunus, M. M., & Hashim, H. (2021). Challenges in assessing speaking skills among Malaysian ESL learners. Asian Journal of University Education, 17(3), 1–11.
https://doi.org/10.24191/ajue.v17i3.14575
Mustapha, S. M., & Yahaya, A. (2013). Low English proficiency among Malaysian learners: Causes and strategies. English Language Teaching, 6(12), 155–163. https://doi.org/10.5539/elt.v6n12p155
Noor, N. M., Mohamed, A. R., & Rahman, N. A. (2020). Achieving CEFR benchmarks in Malaysian ESL primary schools: Challenges and implications. Asian EFL Journal, 24(3), 75–98.
Noor, N. M., Aman, I., Mustaffa, R., & Seong, T. K. (2020). Language proficiency and learning challenges among rural ESL learners in Malaysia. Issues in Educational Research, 30(2), 533–551.
North, B. (2014). Putting the Common European Framework of Reference to good use. Language Teaching, 47(2), 228–249. https://doi.org/10.1017/S0261444812000205
Polit, D. F., & Beck, C. T. (2006). The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Research in Nursing & Health, 29(5), 489–497. https://doi.org/10.1002/nur.20147
Shang, Y. (2024). A meta-analysis of the reliability of second language proficiency assessments. Language Assessment Quarterly, 21(2), 145–169. https://doi.org/10.1080/15434303.2024.1187649
Stemler, S. E. (2004). A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability. Practical Assessment, Research & Evaluation, 9(4), 1–19. https://doi.org/10.7275/96jp-xz07
Sureeyatanapas, P. (2024). Analysing marking reliability in English speaking proficiency tests: A marking consistency approach. Language Testing in Asia, 14(1), 65–79. https://doi.org/10.1186/s40468-023-00271-z
Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Emynordiana Jamil, Nor Asniza Ishak, Afza Diyana Abdullah, Zuhailah Hashim, Putri Sathirah Saaban
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
