Psychometric Evaluation of a Digital Literacy Instrument Using the Rasch Rating Scale Model in the Context of Prospective Science Teachers
DOI:
https://doi.org/10.37134/jsml.vol14.1.8.2026Keywords:
digital literacy, Rasch model, prospective science teachersAbstract
This research aims to develop a digital literacy instrument and test it on prospective science teachers using the Rasch Rating Scale Model. The study involved 81 second-semester students from the Science Education Study Program at Semarang State University (UNNES) and 59 students from UIN Salatiga. The instrument consists of 30 items, 10 essay questions, and 20 questionnaire statements measured using a four-point scale. Data were analyzed using the WINSTEPS Version 5.7.3.0 application. Content validity results show that all essay questions are valid, while 4 out of 20 questionnaire items are inconsistent and require revision. Construct validity analysis indicates that the essay questions account for 29% of the total variance, and the questionnaire explains 46%. The internal consistency (KR-20) is acceptable for essay questions (0.65) and very high for the questionnaire (0.83). Item reliability is excellent, with 0.92 for essay questions and 0.98 for the questionnaire. Item characteristics show a standard deviation of 0.35 for essay questions and 1.01 for the questionnaire. Reliability was evaluated using KR-20 for essay-based items and Cronbach’s Alpha for the questionnaire. This ensures consistency in reporting reliability across both test and non-test components. Differential Item Functioning (DIF) analysis reveals bias in several essay questions between groups, with low probability values and high DIF contrast. In contrast, the questionnaire items function similarly across both groups. Biased items were reviewed and revised. Overall, the instrument demonstrates strong reliability and is suitable for measuring digital literacy in prospective science teachers.
Downloads
References
Abulibdeh A, Zaidan E, Abulibdeh R. (2024). Navigating the confluence of artificial intelligence and education for sustainable development in the era of industry 4.0: Challenges, opportunities, and ethical dimensions. Journal of Cleaner Production, 437, 140527. doi:10.1016/j.jclepro.2023.140527
Alias Z, Mohamad Hussin MZ, Badruldin NA, Taha H. (2021). The effectiveness of flipped classroom method on achievement and readiness in sets topic for form 4 students. Journal of Science and Mathematics Letters, 9(2), 62-74. doi:10.37134/jsml.vol9.2.7.2021
Arfadila A, Aulia ERN, Nugraha RW, Humaeroh S. (2022). Penerapan E-learning dalam inovasi pendidikan untuk meningkatkan kemampuan literasi digital siswa. Jurnal Pendidikan Sosiologi Dan Humaniora, 13(2), 392. doi:10.26418/j-psh.v13i2.54771
Avinç E, Doğan F. (2024). Digital literacy scale: Validity and reliability study with the rasch model. Education and Information Technologies, 29(17), 22895-22941. doi:10.1007/s10639-024-12662-7
Azizah A, Wahyuningsih S. (2020). Penggunaan model Rasch untuk analisis instrumen tes pada mata kuliah Matematika Aktuaria. JUPITEK: Jurnal Pendidikan Matematika, 3(1), 45-50. doi:10.30598/jupitekvol3iss1pp45-50
Boone WJ. (2016). Rasch analysis for instrument development: Why,when,and how? CBE Life Sciences Education, 15(4), 4. doi:10.1187/cbe.16-04-0148
Chassignol M, Khoroshavin A, Klimova A, Bilyatdinova A. (2018). Artificial intelligence trends in education: A narrative overview. Procedia Computer Science, 136, 16-24. doi:10.1016/j.procs.2018.08.233
Eshet Y. (2012). Thinking in the digital era: A revised model for digital literacy. Issues in Informing Science and Information Technology, 9(2), 267-276. doi:10.28945/1621
Fitriarti EA. (2019). Urgensi literasi digital dalam menangkal hoax informasi kesehatan di era digital. Metacommunication; Journal of Communication Studies, 4(2), 234-246. doi:10.20527/mc.v4i2.6929
Giovanni F, Komariah N. (2019). Hubungan antara literasi digital dengan prestasi belajar siswa SMA Negeri 6 Kota Bogor. Libraria: Jurnal Perpustakaan, 7(1), 147-162. doi:10.21043/libraria.v7i1.5827
Gómez LE, Arias B, Verdugo MÁ, Navas P. (2012). Application of the Rasch rating scale model to the assessment of quality of life of persons with intellectual disability. Journal of Intellectual and Developmental Disability, 37(2), 141-150. doi:10.3109/13668250.2012.682647
Hague C, Payton S. (2010). Digital literacy across the curriculum. Bristol: Futurelab.
Halim AD, Jamari D. (2023). Exploring online learning experiences among students in full boarding school during covid-19 pandemic. Journal of Science and Mathematics Letters, 11, 103-109. doi:10.37134/jsml.vol11.sp.11.2023
Jin KY, Reichert F, Cagasan Jr LP, de La Torre J, Law N. (2020). Measuring digital literacy across three age cohorts: Exploring test dimensionality and performance differences. Computers & Education, 157, 103968. doi:10.1016/j.compedu.2020.103968
Jong C, Hodges TE, Royal KD, Welder RM. (2015). Instruments to measure elementary preservice teachers’ conceptions: An application of the Rasch Rating Scale Model. Educational Research Quarterly, 39(1), 21-48.
Lafifa F, Rosana D. (2024). Innovations in assessing students’digital literacy skills in learning science: Effective multiple choice closed-ended tests using RASCH model. Turkish Online Journal of Distance Education, 25(3), 44-56.
Le B, Lawrie GA, Wang JTH. (2022). Student self-perception on digital literacy in STEM blended learning environments. Journal of Science Education and Technology, 31(3), 303-321. doi:10.1007/s10956-022-09956-1
Lestari W, Wigati I, Sholeh MI, Pramita D. (2022). Instrumen literasi digital guru menggunakan Model Rasch. Orbital: Jurnal Pendidikan Kimia, 6(2), 104-113. doi:10.19109/ojpk.v6i2.15019
Li J, Brar A, Roihan N. (2021). The use of digital technology to enhance language and literacy skills for Indigenous people: A systematic literature review. Computers and Education Open, 2, 100035. doi:10.1016/j.caeo.2021.100035
Maulana S, Rusilowati A, Nugroho SE, Susilaningsih E. (2023). Implementasi Rasch Model dalam pengembangan instrumen tes diagnostik. Prosiding Seminar Nasional Pascasarjana, 6(1), 748-757.
Maulida F, Sunarti T. (2022). Pengembangan instrumen tes literasi sains berbasis kearifan lokal di kabupaten lamongan. ORBITA: Jurnal Pendidikan dan Ilmu Fisika, 8(1), 52-65. doi:10.31764/orbita.v8i1.8337
Morgan A, Sibson R, Jackson D. (2022). Digital demand and digital deficit: conceptualising digital literacy and gauging proficiency among higher education students. Journal of Higher Education Policy and Management, 44(3), 258-275. doi:10.1080/1360080X.2022.2030275
Mustapha MB, Jamaludin AAB. (2021). Analysis of interest towards science subject and its relation with 21st century skills. Journal of Science and Mathematics Letters, 9(2), 103-116. doi:10.37134/jsml.vol9.2.10.2021
Naamati-Schneider L. (2024). Enhancing AI competence in health management: students’ experiences with ChatGPT as a learning Tool. BMC Medical Education, 24(1), 1–16. doi:10.1186/s12909-024-05595-9
Naila I, Ridlwan M, Haq MA. (2021). Literasi digital bagi guru dan siswa sekolah dasar: Analisis konten dalam pembelajaran. Jurnal Review Pendidikan Dasar: Jurnal Kajian Pendidikan Dan Hasil Penelitian, 7(2), 166-122. doi:10.26740/jrpd.v7n2.p166-122
Ng W. (2012). Can we teach digital natives digital literacy? Computers & Education, 59(3), 1065-1078. doi:10.1016/j.compedu.2012.04.016
Nguyen LAT, Habók A. (2024). Tools for assessing teacher digital literacy: a review. Journal of Computers in Education, 11(1), 305-346. doi:10.1007/s40692-022-00257-5
OECD. (2019). An OECD Learning Framework 2030. Springer.
Peng D, Yu Z. (2022). A literature review of digital literacy over two decades. Education Research International, 1, 2533413. doi:10.1155/2022/2533413
Quintão S, Delgado AR, Prieto G. (2013). Validity study of the beck anxiety inventory (Portuguese version) by the rasch rating scale model. Psicologia: Reflexao e Critica, 26(2), 305–310. doi:10.1590/S0102-79722013000200010
Rahayu T, Mayasari T. (2018). Profil kemampuan awal literasi digital dalam pembelajaran fisika siswa SMK Kota Madiun. In Seminar Nasional Quantum, 25(1), 2477-1511.
Reddy P, Chaudhary K, Sharma B, Hussein S. (2023). Essaying the design, development and validation processes of a new digital literacy scale. Online Information Review, 47(2), 371-397. doi:10.1108/OIR-10-2021-0532
Restianty A. (2018). Literasi digital, sebuah tantangan baru dalam literasi media. Gunahumas, 1(1), 72-87. doi:10.17509/ghm.v1i1.28380
Rusilowati A, Kurniawati L, Nugroho SE, Widiyatmoko A. (2016). Developing an instrument of scientific literacy asessment on the cycle theme. International Journal of Environmental and Science Education, 11(12), 5718-5727.
Savitri EN, Dewi NR, Amalia AV, Rahmadani OEP, Kholidah A. (2021). Real science mask with QR code to improve the digital literation. Unnes Science Education Journal, 10(1), 34-40. doi:10.15294/usej.v10i1.42537
Son M, Ha M. (2025). Development of a digital literacy measurement tool for middle and high school students in the context of scientific practice. Education and Information Technologies, 30(4), 4583-4606. doi:10.1007/s10639-024-12999-z
Suherdi D, Rezky SF, Apdillah D, Sinuraya J, Sahputra A, Syahputra D, Wahyuni D. (2021). Peran literasi digital di masa pandemik. Deli Serdang-Indonesia: Cattleya Darmaya Fortuna Publisher.
Suherman A. (2020). Promoting digital literacy skills: An action research to people of Kampung Literasi. Universal Journal of Educational Research, 8(4), 1372-1386. doi:10.13189/ujer.2020.080430
Sujana A, Rachmatin D. (2019). Literasi digital abad 21 bagi mahasiswa PGSD: apa, mengapa, dan bagaimana. In Current Research in Education: Conference Series Journal, 1(1), 3-13.
Techataweewan W, Prasertsin U. (2018). Development of digital literacy indicators for Thai undergraduate students using mixed method research. Kasetsart Journal of Social Sciences, 39(2), 215-221. doi:10.1016/j.kjss.2017.07.001
Tejada AJR, Luque MN, Rojas OML, Moreno PJP. (2011). Prejudiced attitude measurement using the rasch rating scale model. Psychological Reports, 109(2), 553-572. doi:10.2466/07.17.PR0.109.5.553-572
Yuniarto B, Yudha RP. (2021). Literasi digital sebagai penguatan pendidikan karakter menuju era society 5.0. Edueksos Jurnal Pendidikan Sosial & Ekonomi, 10(2), 176-194. doi:10.37329/cetta.v6i3.2603
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Erna Noor Savitri, Sudarmin, Murbangun Nuswowati, Ani Rusilowati, Endang Susilaningsih, Sigit Saptono, Roswanna Safkolam
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
