COMBINATION OF CONTENT VALIDITY INDEX AND FUZZY DELPHI FOR EXPERT VALIDATION OF A THREE-TIER MATHEMATICS LEARNING DIFFICULTIES DIAGNOSTIC ASSESSMENT
Abstract
The persistent challenges in identifying mathematics learning difficulties require diagnostic instruments that are not only valid but also capable of accurately capturing students’ conceptual understanding, reasoning, and confidence. This study aims to compare the Content Validity Index (CVI) and the Fuzzy Delphi Method (FDM) in expert validation of a three-tier diagnostic assessment for mathematics learning difficulties. Theoretically, the study integrates classical content validation theory with fuzzy logic-based expert consensus analysis to strengthen the methodological rigor of instrument development. The research involved five doctoral-level experts who evaluated 21 diagnostic items. The CVI results yielded an excellent S-CVI/Ave value of 0.97, indicating high content validity, while the FDM results confirmed 20 of 21 items met the validity thresholds (d ≤ 0.2, consensus ≥ 75%, A ≥ 0.5). A key finding shows that the CVI provides clear numerical validation, whereas the FDM captures linguistic uncertainty and confidence variations among experts, producing a more comprehensive validation outcome. The integration of both methods offers a robust framework for expert validation that enhances interpretive depth and accuracy. This study contributes to the field of mathematics education by presenting a validated, reliable, and pedagogically meaningful diagnostic instrument for identifying students’ misconceptions and reasoning patterns.
Downloads
References
M. S. Ibarra-Sáiz, G. Rodríguez-Gómez, and D. Boud, “THE QUALITY OF ASSESSMENT TASKS AS A DETERMINANT OF LEARNING,” Assess Eval High Educ, vol. 46, no. 6, 2021. doi: https://doi.org/10.1080/02602938.2020.1828268.
M. A. Adarkwah, “THE POWER OF ASSESSMENT FEEDBACK IN TEACHING AND LEARNING: A NARRATIVE REVIEW AND SYNTHESIS OF THE LITERATURE,” SN Social Sciences, vol. 1, no. 3, 2021. doi: https://doi.org/10.1007/s43545-021-00086-w.
A. I. Säfström et al., “DEVELOPING A DIAGNOSTIC FRAMEWORK FOR PRIMARY AND SECONDARY STUDENTS’ REASONING DIFFICULTIES DURING MATHEMATICAL PROBLEM SOLVING,” Educational Studies in Mathematics, vol. 115, no. 2, 2024. doi: https://doi.org/10.1007/s10649-023-10278-1.
S. Ndiung and M. Jediut, “PENGEMBANGAN INSTRUMEN TES HASIL BELAJAR MATEMATIKA PESERTA DIDIK SEKOLAH DASAR BERORIENTASI PADA BERPIKIR TINGKAT TINGGI,” Premiere Educandum : Jurnal Pendidikan Dasar dan Pembelajaran, vol. 10, no. 1, 2020. doi: https://doi.org/10.25273/pe.v10i1.6274.
M. J. Gierl, O. Bulut, Q. Guo, and X. Zhang, “DEVELOPING, ANALYZING, AND USING DISTRACTORS FOR MULTIPLE-CHOICE TESTS IN EDUCATION: A COMPREHENSIVE REVIEW,” Rev Educ Res, vol. 87, no. 6, 2017. doi: https://doi.org/10.3102/0034654317726529.
M. M. Wooten, A. M. Cool, E. E. Prather, and K. D. Tanner, “COMPARISON OF PERFORMANCE ON MULTIPLE-CHOICE QUESTIONS AND OPEN-ENDED QUESTIONS IN AN INTRODUCTORY ASTRONOMY LABORATORY,” Physical Review Special Topics - Physics Education Research, vol. 10, no. 2, 2014. doi: https://doi.org/10.1103/PhysRevSTPER.10.020103.
Kemendikbudristek, “PISA 2022 DAN PEMULIHAN PEMBELAJARAN DI INDONESIA,” Laporan Pisa Kemendikbudristek, 2023.
OECD, “PISA 2022 RESULTS FACTSHEETS INDONESIA,” OECD (Organisation for Economic Co-operation and Development) Publication, 2023.
T. Fan, J. Song, and Z. Guan, “INTEGRATING DIAGNOSTIC ASSESSMENT INTO CURRICULUM: A THEORETICAL FRAMEWORK AND TEACHING PRACTICES,” Language Testing in Asia, vol. 11, no. 1, 2021. doi: https://doi.org/10.1186/s40468-020-00117-y.
K. BSKAP, “TAHAPAN IMPLEMENTASI KURIKULUM MERDEKA DI SATUAN PENDIDIKAN.,” Kemendibudristek, 2022.
B. R. Bryant et al., “THE EFFECTS OF A TIER 3 INTERVENTION ON THE MATHEMATICS PERFORMANCE OF SECOND GRADE STUDENTS WITH SEVERE MATHEMATICS DIFFICULTIES,” J Learn Disabil, vol. 49, no. 2, 2016. doi: https://doi.org/10.1177/0022219414538516.
P. Djiwandono and D. Ginting, “EVALUATING RESEARCH REPORTS ON THE QUALITIES OF TESTS OF ENGLISH LANGUAGE SKILLS IN INDONESIAN SCHOOLS: A SYSTEMATIC REVIEW,” Language Education & Assessment, vol. 8, no. 1, p. 2237, Jun. 2025. doi: https://doi.org/10.29140/lea.v8n1.2237.
A. C. Butler, “MULTIPLE-CHOICE TESTING IN EDUCATION: ARE THE BEST PRACTICES FOR ASSESSMENT ALSO GOOD FOR LEARNING?,” 2018. doi: https://doi.org/10.1016/j.jarmac.2018.07.002.
A. Zuhriyah, A. Suprijono, and A. Kasdi, “NEW MULTIPLE CHOICE QUESTIONS IN CRITICAL THINKING ASSESSMENT: AS A WAY TO DISTINGUISH BETWEEN LOGICAL ANSWERS AND GUESS ANSWERS,” International Journal of Scientific and Research Publications (IJSRP), vol. 9, no. 10, 2019. doi: https://doi.org/10.29322/ijsrp.9.10.2019.p9422.
R. M. Abu-Ghazalah, D. N. Dubins, and G. M. K. Poon, “DISSECTING KNOWLEDGE, GUESSING, AND BLUNDER IN MULTIPLE CHOICE ASSESSMENTS,” Applied Measurement in Education, vol. 36, no. 1, 2023. doi: https://doi.org/10.1080/08957347.2023.2172017.
Y. S. Pramesti, H. Mahmudi, and I. Setyowidodo, “USING THREE-TIER TEST TO DIAGNOSE STUDENTS’ LEVEL OF UNDERSTANDING,” in Journal of Physics: Conference Series, 2021. doi: https://doi.org/10.1088/1742-6596/1806/1/012013.
A. S. Pramasdyahsari, D. Viviana, S. Sugiyanti, B. P. Pant, and I. Ratnayake, “CONSTRUCTION OF THREE-TIER DIAGNOSTIC TEST FOR IDENTIFYING STUDENT MISCONCEPTIONS IN EXPONENTIAL PROPERTIES,” Jurnal Riset Pendidikan Matematika, vol. 11, no. 1, pp. 64–76, 2024. doi: https://doi.org/10.21831/jrpm.v11i1.73561.
S. Prayitno, A. Arjudin, and H. Hapipi, “ANALYZING GEOMETRY MISCONCEPTION OF PROSPECTIVE TEACHERS USING THREE-TIER DIAGNOSTIC TESt,” in Proceedings of the 1st Annual Conference on Education and Social Sciences (ACCESS 2019), Altantis Press, 2020. doi: https://doi.org/10.2991/assehr.k.200827.031.
C. M. H. Laeli, Gunarhadi, and Muzzazinah, “THE 3 TIERS MULTIPLE-CHOICE DIAGNOSTIC TEST FOR PRIMARY STUDENTS’ SCIENCE MISCONCEPTION,” Pegem Egitim ve Ogretim Dergisi, vol. 13, no. 2, 2023. doi: https://doi.org/10.47750/pegegog.13.02.13.
D. S. Anggara and C. Abdillah, “CONTENT VALIDITY ANALYSIS OF LITERACY ASSESSMENT INSTRUMENTS,” Cakrawala Pendidikan, vol. 42, no. 2, 2023. doi: https://doi.org/10.21831/cp.v42i2.55900.
A. Roebianto, S. I. Savitri, I. Aulia, A. Suciyana, and L. Mubarokah, “CONTENT VALIDITY: DEFINITION AND PROCEDURE OF CONTENT VALIDATION IN PSYCHOLOGICAL RESEARCH,” TPM Test Psychom Methodol Appl Psychol, vol. 30, no. 1, 2023. doi: https://doi.org/10.4473/TPM30.1.1.
N. H. Jamin, S. Surat, and W. M. R. Wan Mohammad, “TEACHERS’ KNOWLEDGE IN HOTS (TKHOTS) INSTRUMENT: CONTENT VALIDITY INDEX,” International Journal of Academic Research in Progressive Education and Development, vol. 11, no. 1, 2022. doi: https://doi.org/10.6007/ijarped/v11-i1/12179.
K. Beck, “ENSURING CONTENT VALIDITY OF PSYCHOLOGICAL AND EDUCATIONAL TESTS – THE ROLE OF EXPERTS,” Frontline Learn Res, vol. 8, no. 6, 2020. doi: https://doi.org/10.14786/flr.v8i6.517.
A. F. R. L. de Hierro, M. Sánchez, D. Puente-Fernández, R. Montoya-Juárez, and C. Roldán, “A FUZZY DELPHI CONSENSUS METHODOLOGY BASED ON A FUZZY RANKING,” Mathematics, vol. 9, no. 18, 2021. doi: https://doi.org/10.3390/math9182323.
Z. Zeng, X. Li, and S. W. Tho, “THE APPLICATION OF FUZZY DELPHI METHOD FOR THE DEVELOPMENT OF STEM TEACHING MODEL,” International Journal of Evaluation and Research in Education, vol. 14, no. 3, pp. 2061–2069, Jun. 2025. doi: https://doi.org/10.11591/ijere.v14i3.29780.
A. F. R. L. de Hierro, M. Sánchez, D. Puente-Fernández, R. Montoya-Juárez, and C. Roldán, “A FUZZY DELPHI CONSENSUS METHODOLOGY BASED ON A FUZZY RANKING,” Mathematics, vol. 9, no. 18, Sep. 2021. doi: https://doi.org/10.3390/math9182323.
F. B. Abdul Rahman, Z. Mustafa, and A. F. Kharuddin, “EMPLOYING FUZZY DELPHI TECHNIQUE TO VALIDATE MULTIPLE INTELLIGENCE BASED INSTRUCTIONAL TEACHING MODULE FOR PRESCHOOL CHILDREN,” Southeast Asia Early Childhood Journal, vol. 10, no. 1, 2021. doi: https://doi.org/10.37134/saecj.vol10.1.6.2021.
M. A. Hasim, J. Jabar, A. Sufian, N. F. Ibrahim, and F. A. Khalid, “EMPLOYING FUZZY DELPHI TECHNIQUES TO VALIDATE THE COMPONENTS AND CONTENTS OF E-LEARNING ANTECEDENTS AND USAGE BEHAVIOR TOWARDS E-LEARNING PERFORMANCE,” European Journal of Educational Research, vol. 12, no. 1, 2023. doi: https://doi.org/10.12973/eu-jer.12.1.467.
F. N. Pazilah, H. Hashim, and M. M. Yunus, “REFINING THE TPACK FRAMEWORK: A FUZZY DELPHI APPROACH TO 21ST-CENTURY COMPETENCY AND SELF-EFFICACY CONSTRUCTS,” Cogent Education, vol. 11, no. 1, 2024. doi: https://doi.org/10.1080/2331186X.2024.2428886.
M. F. Baharuddin, M. N. Masrek, and S. M. Shuhidan, “CONTENT VALIDITY OF ASSESSMENT INSTRUMENT FOR INNOVATIVE WORK BEHAVIOUR OF MALAYSIAN SCHOOL TEACHERS,” International Journal of Scientific and Technology Research, vol. 9, no. 4, 2020.
E. Karim, J. Ahmad, and K. Osman, “FUZZY DELPHI METHOD FOR CONTENT VALIDATION OF INTEGRATED SCIENCE PROCESS SKILLS INSTRUMENT,” International Journal of Academic Research in Business and Social Sciences, vol. 7, no. 6, 2017. doi: https://doi.org/10.6007/ijarbss/v7-i6/3037.
N. M. Mastam and R. Zaharudin, “THE FUZZY DELPHI METHOD: VALIDATION OF THE PROFESSIONAL COMPETENCIES ELEMENTS IN CAREER READINESS INSTRUMENT FOR STUDENTS WITH DISABILITIES,” International Journal of Academic Research in Progressive Education and Development, vol. 13, no. 4, Nov. 2024. doi: https://doi.org/10.6007/ijarped/v13-i4/23817.
N. M. Mastam, R. Zaharudin, and M. N. Md Noor, “THE FUZZY DELPHI METHOD: VALIDATION OF THE PERSONAL QUALITIES AND ABILITIES ELEMENTS IN CAREER READINESS INSTRUMENT FOR STUDENTS WITH DISABILITIES,” International Journal of Academic Research in Progressive Education and Development, vol. 13, no. 3, Aug. 2024. doi: https://doi.org/10.6007/ijarped/v13-i3/22402.
A. H. bin Abd Rahman, N. binti Mat Norwani, and R. binti Yahaya, “CONTENT VALIDITY OF ACCOUNTING LEARNING SELF EFFICACY INSTRUMENT AMONG MATRICULATION ACCOUNTING STUDENTS,” International Journal of Academic Research in Progressive Education and Development, vol. 12, no. 3, 2023. doi: https://doi.org/10.6007/ijarped/v12-i3/18741.
D. F. Polit, C. T. Beck, and S. V. Owen, “FOCUS ON RESEARCH METHODS: IS THE CVI AN ACCEPTABLE INDICATOR OF CONTENT VALIDITY? APPRAISAL AND RECOMMENDATIONS,” Res Nurs Health, vol. 30, no. 4, 2007. doi: https://doi.org/10.1002/nur.20199.
M. A. Jailani and C. K. Loy, “THE APPLICATION OF FUZZY DELPHI METHOD IN CONTENT VALIDITY ANALYSIS,” in 20th International Conference on Cognition and Exploratory Learning in Digital Age, CELDA 2023, 2023. doi: https://doi.org/10.33965/celda2023_202306l011.
D. F. Polit and C. T. Beck, “THE CONTENT VALIDITY INDEX: ARE YOU SURE YOU KNOW WHAT’S BEING REPORTED? CRITIQUE AND RECOMMENDATIONS,” Res Nurs Health, vol. 29, no. 5, 2006. doi: https://doi.org/10.1002/nur.20147.
I. W. Prananto, Y. Rakhmawati, and T. Pamungkas, “CONTENT VALIDITY RATIO (CVR), CONTENT VALIDITY INDEX (CVI) AND CONFIRMATORY FACTOR ANALYSIS (CFA) IN MATHEMATICS LEARNING INDEPENDENCE INSTRUMENTS,” Kontinu: Jurnal Penelitian Didaktik Matematika, vol. 6, no. 2, 2022. doi: https://doi.org/10.30659/kontinu.6.2.116-132.
R. Afrizon, L. E. Mohtar, and M. S. M. Azmi, “DEVELOPMENT OF THE FUZZY DELPHI INSTRUMENT FOR IDENTIFYING ELEMENTS OF THE PHYARECI-STEM LEARNING MODULE,” EDUCATUM Journal of Science, Mathematics and Technology, vol. 12, no. 12, pp. 33–44, Apr. 2025. doi: https://doi.org/10.37134/ejsmt.vol12.sp.4.2025.
M. S. B. Yusoff, “ABC OF CONTENT VALIDATION AND CONTENT VALIDITY INDEX CALCULATION,” Education in Medicine Journal, vol. 11, no. 2, 2019, https://doi.org/10.21315/eimj2019.11.2.6.
R. M. Rodríguez, Á. Labella, and L. Martínez, “AN OVERVIEW ON FUZZY MODELLING OF COMPLEX LINGUISTIC PREFERENCES IN DECISION MAKING,” International Journal of Computational Intelligence Systems, vol. 9, 2016, https://doi.org/10.1080/18756891.2016.1180821.
Copyright (c) 2026 Khaerun Nisa, Iva Sarifah, Riyadi Riyadi
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this Journal agree to the following terms:
- Author retain copyright and grant the journal right of first publication with the work simultaneously licensed under a creative commons attribution license that allow others to share the work within an acknowledgement of the work’s authorship and initial publication of this journal.
- Authors are able to enter into separate, additional contractual arrangement for the non-exclusive distribution of the journal’s published version of the work (e.g. acknowledgement of its initial publication in this journal).
- Authors are permitted and encouraged to post their work online (e.g. in institutional repositories or on their websites) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published works.
1.gif)
