A Systematic Review on Instruments to Assess Critical Thinking & Problem-Solving Skills
DOI:
https://doi.org/10.37134/ejsmt.vol9.sp.5.2022Keywords:
instrument, CTPS skills, assessment, critical thinking and problem-solvingAbstract
Critical Thinking and Problem Solving (CTPS) are soft skills essential to be equipped among students according to 21st-century learning. Several instruments have been developed to measure CTPS skills among students in various subjects. This review aimed to identify the type of instrument and the context they are used. The research methodology was based on the systematic literature search in online databases, mainly in Scopus and ERIC, complemented by Google Scholar and MyCite. The keywords used during the searching process included “instrument”, “test”, “assessment”, “critical thinking”, and “problem-solving”. The screening process took into consideration the type of publication in which only the articles written in Malay or English language within the year 2017 to 2021 were considered. The result of this review revealed that the test, rubric, observation sheet, and questionnaire were the most used types of instruments. Meanwhile, the contexts of measurement were by group discussion, experimental report, quiz, and answer sheet. This review can help educators and researchers make informed decisions about choosing the appropriate CTPS instrument and context for their assessments.
Downloads
References
World Economic Forum. (2020). The future of jobs report. https://www.weforum.org/reports/the-future-of-jobs-report-2020
Reynders, G., Lantz, J., Ruder, S. M., Stanford, C. L., & Cole, R. S. (2020). Rubrics to assess critical thinking and information processing in undergraduate STEM courses. International Journal of STEM Education, 7(9), 1–15. https://doi.org/10.1186/s40594-020-00208-5
Chan, C. K. Y., & Luk, L. Y. Y. (2020). Development and validation of an instrument measuring undergraduate students ’ perceived holistic competencies. Assessment & Evaluation in Higher Education, 46(3), 467–482. https://doi.org/10.1080/02602938.2020.1784392
Dwyer, C. P., Hogan, M. J., & Stewart, I. (2014). An integrated critical thinking framework for the 21st century. Thinking Skills and Creativity, 12(June 2014), 43–52. https://doi.org/10.1016/j.tsc.2013.12.004
Kementerian Pengajian Tinggi Malaysia. (2006). Modul pembangunan kemahiran insaniah (soft skills) untuk IPT Malaysia. Universiti Putra Malaysia.
Docktor, J. L., Strand, N. E., Mestre, J. P., & Ross, B. H. (2015). Conceptual problem solving in high school physics. Physical Review Special Topics - Physics Education Research, 11(2), 1–13. https://doi.org/10.1103/PhysRevSTPER.11.020106
Rear, D. (2019). One size fits all? The limitations of standardised assessment in critical thinking. Assessment & Evaluation in Higher Education, 44(5), 664–675. https://doi.org/10.1080/02602938.2018.1526255
Wren, D., & Cashwell, A. (2018). Mission Possible: Measuring Critical Thinking and Problem Solving. Educational Leadership, 75(5), 70–75.
Mueller, J. F., Taylor, H. K., Brakke, K., Drysdale, M., Kelly, K., Levine, G. M., & Ronquillo-Adachi, J. (2020). Assessment of Scientific Inquiry and Critical Thinking: Measuring APA Goal 2 Student Learning Outcomes. Teaching of Psychology, 47(4), 274–284. https://doi.org/10.1177/0098628320945114
Mabruroh, F., & Suhandi, A. (2017). Construction of Critical Thinking Skills Test Instrument Related The Concept On Sound Wave. Journal of Physics: Conference Series, 812(012056). https://doi.org/10.1088/1742-6596/755/1/011001
Alsaleh, N. J. (2020). Teaching Critical Thinking Skills: Literature Review. TOJET: The Turkish Online Journal of Educational Technology, 19(1), 21–39. http://www.tojet.net/articles/v19i1/1913.pdf
Larsson, K. (2021). Using Essay Responses as a Basis for Teaching Critical Thinking-a Variation Theory Approach. Scandinavian Journal of Educational Research, 65(1), 21–35. https://doi.org/10.1080/00313831.2019.1650824
Istiyono, E., Dwandaru, W. S. B., Lede, Y. A., Rahayu, F., & Nadapdap, A. (2019). Developing IRT-based physics critical thinking skill test: A CAT to answer 21st century challenge. International Journal of Instruction, 12(4), 267–280. https://doi.org/10.29333/iji.2019.12417a
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Guidelines and guidance preferred reporting items for systematic reviews and meta-analyses:The PRISMA statement. PLOS Medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097
Gusenbauer, M., & Haddaway, N. R. (2020). Which academic search systems are suitable for systematic reviews or meta-analyses ? Evaluating retrieval qualities of Google Scholar , PubMed , and 26 other resources. Research Syntheses Methods, 11(2), 181–217. https://doi.org/10.1002/jrsm.1378
Kurniawan, W., Pathoni, H., Muliawati, L., Kurniawan, D. A., Romadona, D. D., Ningsi, A. P., & Dari, R. W. (2020). Relationship of science process skills and critical thinking of students in physics Subject. Universal Journal of Educational Research, 8(11), 5581–5588. https://doi.org/10.13189/ujer.2020.081162
Priyaadharshini, M., & Sundaram, B. V. (2018). Evaluation of higher-order thinking skills using learning style in an undergraduate engineering in flipped classroom. Computer Applications in Engineering Education, 26(6), 2237–2254. https://doi.org/10.1002/cae.22035
Nonis, S. A., & Hudson, G. I. (2018). Developing and assessing critical thinking skills in marketing students : The power of making explicit problem-solving processes. Journal of Education for Business, 94(3), 195–203. https://doi.org/10.1080/08832323.2018.1504737
Dolapcioglu, S., & Doganay, A. (2020). Development of critical thinking in mathematics classes via authentic learning: An action research. International Journal of Mathematical Education in Science and Technology. https://doi.org/10.1080/0020739X.2020.1819573
Re, M. R., Amenduni, F., Medio, C. De, & Valente, M. (2019). How to use assessment data collected through writing activities to identify participants’ critical thinking levels. Journal of E-Learning and Knowledge Society, 15(3), 117–132. https://doi.org/10.20368/1971-8829/1135051
Kuang, C. Y., Kuen, Y. L., & Shu, F. C. (2017). The development and validation of a mechanical critical thinking scale for high school students. EURASIA Journal of Mathematics Science and Technology Education, 13(5), 1361–1376. https://doi.org/10.12973/eurasia.2017.00675a
Uvathi Mariappan, Arsaythamby Veloo, & Kanageswari Suppiah Shanmugam. (2018). Pembinaan dan pengesahan instrumen kemahiran insaniah bagi subjek perakaunan di matrikulasi. Jurnal Kurikulum & Pengajaran Asia Pasifik, 6(4), 29–39.
Aris Hanafi Abu Jalil, Gunathevan Elumalai, & Mohd Izwan Shahri. (2019). Pembinaan rubrik kemahiran insaniah bagi matapelajaran Kokurikulum pelajar program matrikulasi Kementerian Pendidikan Malaysia. Jurnal Penyelidikan Dedikasi, 17, 126–145.
Zainab Ab Rahman, Hoon, T. S., & Sidhu, G. K. (2019). Using the peer socratic questioning (PSQ) technique to develop critical thinking skills in group discussion. Jurnal Penyelidikan Tempawan, XXXVI, 91–103.
Tiruneh, D. T., Cock, M. De, Weldeslassie, A. G., Elen, J., & Janssen, R. (2017). Measuring critical thinking in physics : Development and validation of a critical thinking test in electricity and magnetism. International Journal of Science and Mathematics Education, 15(4), 663–682. https://doi.org/10.1007/s10763-016-9723-0
Lertyosbordin, C., Maneewan, S., Yampinij, S., & Thamwipat, K. (2019). Scoring rubric of problem-solving on computing science learning. International Education Studies, 12(8), 26–32. https://doi.org/10.5539/ies.v12n8p26
Bernstein, A. G., & Isaac, C. (2018). Critical thinking criteria for evaluating online discussion. The International Journal for the Scholarship of Teaching and Learning, 12(2), 1–8.
Sadhu, S., & Laksono, E. W. (2018). Development and validation of an integrated assessment for measuring critical thinking and chemical literacy in chemical equilibrium. International Journal of Instruction, 11(3), 557–572. https://doi.org/10.12973/iji.2018.11338a
Bandyopadhyay, S., & Szostek, J. (2019). Thinking critically about critical thinking: Assessing critical thinking of business students using multiple measures. Journal of Education for Business, 94(4), 259–270. https://doi.org/10.1080/08832323.2018.1524355
Azizi Alias, & Kamisah Osman. (2018). Pentaksiran alternatif: Pembinaan dan pelaksanaan rubrik dalam pendidikan Sains. Penerbit Universiti Kebangsaan Malaysia.
Huang, B., & Jong, M. S. Y. (2020). Developing a generic rubric for evaluating students’ work in STEM education. 2020 International Symposium on Educational Technology (ISET), 210–213. https://doi.org/10.1109/ISET49818.2020.00053
Boughey, S. (2019). Developing rubric to assess 3rd - 5th grade students understanding of science concepts via screencast models. Indiana University, Bloomington.
Tadele, K., & Sitotaw, B. (2018). Perception and trends in assessment of students’ learning in Physics courses. Latin-American Journal of Physics Education, 12(1), 1307–1315. https://doi.org/10.17576/jpen-2018-43.02-09
Fukuzawa, S., & DeBraga, M. (2019). Graded response method: does question type influence the assessment of critical thinking? Journal of Curriculum and Teaching, 8(1), 1–10. https://doi.org/10.5430/jct.v8n1p1
Poce, A., Medio, C. De, Amenduni, F., & Re, M. R. (2019). Critical thinking assessment : A first approach to the automatic evaluation. 18th International Conference on Information Technology Based Higher Education and Training (ITHET), 331–338.
Harahap, D. E., Festiyed, & Djamas, D. (2019). Preliminary study on development of assessment performance instruments on physics learning to improve students’ critical thinking ability. Journal of Physics: Conference Series, 1185(012128), 1–7. https://doi.org/10.1088/1742-6596/1185/1/012128
Tangkin, W. P. (2019). Pemanfaatan rubrik sebagai instrumen penilaian alternatif. Scholaria: Jurnal Pendidikan Dan Kebudayaan, 9(1), 29–39. https://doi.org/https://doi.org/10.24246/j.js.2019.v9.i1.p29-39
Brookhart, S. M. (2018). Appropriate criteria : Key to effective rubrics. Frontiers in Education, 3(22), 1–12. https://doi.org/10.3389/feduc.2018.00022
Afifah Fadhlullah, & Nurbarirah Ahmad. (2017). Thinking outside of the box: Determining students’ level of critical thinking skills in teaching and learning. Asian Journal of University Education, 13(2), 51–70.
Rathakrishnan, M., Ahmad, R., & Choo, L. S. (2017). Online discussion : Enhancing students’ critical thinking skills. AIP Conference Proceedings, 1891(1), 020120.
Beatty, P. C., Collins, D., Kaye, L., Padilla, J. L., Willis, G. B., & Wilmot, A. (2019). Advances in questionnaire design, development, evaluation and testing. John Wiley & Sons.
Batsell Jr, W. R., Perry, J. L., Hanley, E., & Hostetter, A. B. (2016). Ecological validity of the testing effect : The use of daily quizzes in introductory psychology. Teaching of Psychology, 1(6), 18–23. https://doi.org/10.1177/0098628316677492
Ahmad, A., & Awang, M. I. (2016). Pengukuran dan penilaian pendidikan. Dewan Bahasa dan Pustaka.
Tenam-Zemach, M., & Flynn, J. E. (2015). Rubric nation: Critical inquiries on the impact of rubrics in education. Information Age Publishing Inc
