ITEM ANALYSIS OF HIGH SCHOOL SPECIALIZATION MATHEMATICS EXAM QUESTIONS WITH ITEM RESPONSE THEORY APPROACH
Abstract
Analysis of item characteristics on test instruments is carried out to determine high-quality items. This study aims to describe the parameters of specialized high school mathematics test items using the IRT approach. It is an exploratory descriptive study employing a quantitative approach. The research subjects were 36 students of grade XI high school who took the specialization mathematics subject. Response data with dichotomous scoring were analyzed using the IRT approach with the R program to obtain information about item parameters and student ability. The results of the model fit test showed that most of the specialization mathematics exam items fit the Rasch model. The results showed that all items met the criteria of good quality because they had good difficulty parameters. Relatively, the test items were suitable for students with abilities between -2.6 and 2.8 logits. This estimation is also supported by the TIF with a maximum value of 3.049 at 0.08 logit ability and SEM of 0.541. Test items that have been proven to be of high quality can be used as examples in both teaching and diagnostic assessments. Further research could consider the discrimination parameter when analyzing the characteristics of the questions.
Downloads
References
S. Ndiung and M. Jediut, “Pengembangan instrumen tes hasil belajar matematika peserta didik sekolah dasar berorientasi pada berpikir tingkat tinggi,” Premiere Educandum : Jurnal Pendidikan Dasar dan Pembelajaran, vol. 10, no. 1, p. 94, Jun. 2020, doi: 10.25273/pe.v10i1.6274.
L. W. T. Schuwirth and C. P. M. Van der Vleuten, “Programmatic assessment: From assessment of learning to assessment for learning,” Med Teach, vol. 33, no. 6, pp. 478–485, Jun. 2011, doi: 10.3109/0142159X.2011.565828.
A. N. Castleberry, E. F. Schneider, M. H. Carle, and C. D. Stowe, “Development of a summative examination with subject matter expert validation,” Am J Pharm Educ, vol. 80, no. 2, pp. 1–9, Mar. 2016, doi: 10.5688/ajpe80229.
S. Heeneman, A. Oudkerk Pool, L. W. T. Schuwirth, C. P. M. van der Vleuten, and E. W. Driessen, “The impact of programmatic assessment on student learning: Theory versus practice,” Med Educ, vol. 49, no. 5, pp. 487–498, May 2015, doi: 10.1111/medu.12645.
L. Ivanjek et al., “Development of a two-tier instrument on simple electric circuits,” Phys Rev Phys Educ Res, vol. 17, no. 2, p. 020123, Sep. 2021, doi: 10.1103/PhysRevPhysEducRes.17.020123.
G. Svensäter and M. Rohlin, “Assessment model blending formative and summative assessments using the SOLO taxonomy,” European Journal of Dental Education, vol. 27, no. 1, pp. 149–157, Feb. 2023, doi: 10.1111/eje.12787.
D. Desilva, I. Sakti, and R. Medriati, “Pengembangan instrumen penilaian hasil belajar fisika berorientasi hots (higher order thinking skills) pada materi elastisitas dan hukum hooke,” Jurnal Kumparan Fisika, vol. 3, no. 1, pp. 41–50, Apr. 2020, doi: 10.33369/jkf.3.1.41-50.
T. M. Haladyna and M. C. Rodriguez, Developing and validating test items. Routledge, 2013. doi: 10.4324/9780203850381.
A. M. Andrés and J. D. L. Castillo, “Multiple choice tests: Power, length and optimal number of choices per item,” British Journal of Mathematical and Statistical Psychology, vol. 43, no. 1, pp. 57–71, May 1990, doi: 10.1111/j.2044-8317.1990.tb00926.x.
O. N. Bakytbekovich et al., “Distractor analysis in multiple-choice items using the rasch model,” International Journal of Language Testing, vol. 13, pp. 69–78, 2023, doi: 10.22034/IJLT.2023.387942.1236.
D. Briggs, A. Alonzo, C. Schwab, and M. Wilson, “Diagnostic assessment with ordered multiple-choice items,” Educational Assessment, vol. 11, no. 1, pp. 33–63, Feb. 2006, doi: 10.1207/s15326977ea1101_2.
S. Lions, C. Monsalve, P. Dartnell, M. P. Blanco, G. Ortega, and J. Lemarié, “Does the response options placement provide clues to the correct answers in multiple-choice tests? A systematic review,” Applied Measurement in Education, vol. 35, no. 2, pp. 133–152, Apr. 2022, doi: 10.1080/08957347.2022.2067539.
M. J. Gierl, O. Bulut, Q. Guo, and X. Zhang, “Developing, analyzing, and using distractors for multiple-choice tests in education: A comprehensive review,” Rev Educ Res, vol. 87, no. 6, pp. 1082–1116, Dec. 2017, doi: 10.3102/0034654317726529.
M. M. Wooten, A. M. Cool, E. E. Prather, and K. D. Tanner, “Comparison of performance on multiple-choice questions and open-ended questions in an introductory astronomy laboratory,” Physical Review Special Topics - Physics Education Research, vol. 10, no. 2, p. 020103, Jul. 2014, doi: 10.1103/PhysRevSTPER.10.020103.
D. Saepuzaman, H. Haryanto, , Edi Istiyono, H. Retnawati, and Y. Yustiandi, “Analysis of items parameters on work and energy subtest using item response theory,” Jurnal Pendidikan MIPA, vol. 22, no. 1, pp. 1–9, 2021, doi: 10.23960/jpmipa/v22i1.pp1-9.
R. Pratiwi, S. Reflianti, S. Antini, and A. Walid, “Analysis of item difficulty index for midterm examinations in junior high schools 5 Bengkulu City,” Asian Journal of Science Education, vol. 3, no. 1, pp. 12–18, Apr. 2021, doi: 10.24815/ajse.v3i1.18895.
H. Guo, R. Lu, M. S. Johnson, and D. F. McCaffrey, “Alternative methods for item parameter estimation: From CTT to IRT,” ETS Research Report Series, vol. 2022, no. 1, pp. 1–16, Dec. 2022, doi: 10.1002/ets2.12355.
R. Bahar, E. Istiyono, W. Widihastuti, S. Munadi, Z. Nuryana, and S. Fajaruddin, “Analisis karakteristik soal ujian sekolah hasil musyawarah guru matematika di Tasikmalaya,” AKSIOMA: Jurnal Program Studi Pendidikan Matematika, vol. 10, no. 4, pp. 2660–2674, Dec. 2021, doi: 10.24127/ajpm.v10i4.4359.
R. K. Hambleton, Hariharan. Swaminathan, and H. Jane. Rogers, Fundamentals of item response theory. Sage Publications, 1991.
D. Saepuzaman, E. Istiyono, H. Haryanto, H. Retnawati, and Y. Yustiandi, “Analisis karakteristik butir soal fisika dengan pendekatan IRT penskoran dikotomus dan politomus,” Radiasi : Jurnal Berkala Pendidikan Fisika, vol. 14, no. 2, pp. 62–75, Sep. 2021, doi: 10.37729/radiasi.v14i2.1200.
M. A. Khalaf and E. M. N. Omara, “Rasch analysis and differential item functioning of English language anxiety scale (ELAS) across sex in Egyptian context,” BMC Psychol, vol. 10, no. 1, p. 242, Oct. 2022, doi: 10.1186/s40359-022-00955-w.
Y. Liu and A. Maydeu-Olivares, “Local dependence diagnostics in IRT modeling of binary data,” Educ Psychol Meas, vol. 73, no. 2, pp. 254–274, Apr. 2013, doi: 10.1177/0013164412453841.
G. Fergadiotis et al., “Item response theory modeling of the verb naming test,” Journal of Speech, Language, and Hearing Research, vol. 66, no. 5, pp. 1718–1739, May 2023, doi: 10.1044/2023_JSLHR-22-00458.
M. Brucato, A. Frick, S. Pichelmann, A. Nazareth, and N. S. Newcombe, “Measuring spatial perspective taking: Analysis of four measures using item response theory,” Top Cogn Sci, vol. 15, no. 1, pp. 46–74, Jan. 2022, doi: 10.1111/tops.12597.
J. Kean, E. F. Bisson, D. S. Brodke, J. Biber, and P. H. Gross, “An introduction to item response theory and rasch analysis: Application using the eating assessment tool (EAT-10),” Brain Impairment, vol. 19, no. 1, pp. 91–102, Mar. 2018, doi: 10.1017/BrImp.2017.31.
H. Retnawati, Teori respons butir dan penerapannya. Nuha Medika, 2014.
M. Malaspina and B. Arias, “A Rasch modeling approach for measuring young children’s informal mathematics in Peru,” Eurasia Journal of Mathematics, Science and Technology Education, vol. 18, no. 9, pp. 1–13, Aug. 2022, doi: 10.29333/ejmste/12303.
R. Ramadhani, S. Saragih, and E. E. Napitupulu, “Exploration of students’ statistical reasoning ability in the context of ethnomathematics: A study of the Rasch model,” Mathematics Teaching Research Journal, vol. 14, no. 1, pp. 138–168, 2022.
A. Z. Khairani and H. Shamsuddin, “Application of rasch measurement model in developing calibrated item pool for the topic of rational numbers,” Eurasia Journal of Mathematics, Science and Technology Education, vol. 17, no. 12, pp. 1–11, Dec. 2021, doi: 10.29333/ejmste/11426.
T. T. Semiun and F. D. Luruk, “The quality of an english summative test of a public junior high school, Kupang-NTT,” English Language Teaching Educational Journal, vol. 3, no. 2, p. 133, Sep. 2020, doi: 10.12928/eltej.v3i2.2311.
H. Ahmad and S. E. Mokshein, “Is 3pl item response theory an appropriate model for dichotomous item analysis of the anatomy & physiology final examination?,” Malaysian Science & Mathematics Education Journal, vol. 6, no. 1, pp. 13–23, 2016, [Online]. Available: https://www.researchgate.net/publication/322200082
W. Guo and Y.-J. Choi, “Assessing dimensionality of IRT models using traditional and revised parallel analyses,” Educ Psychol Meas, vol. 83, no. 3, pp. 609–629, Jun. 2023, doi: 10.1177/00131644221111838.
J. Hattie, “Methodology review: Assessing unidimensionality of tests and itenls,” Appl Psychol Meas, vol. 9, no. 2, pp. 139–164, Jun. 1985, doi: 10.1177/014662168500900204.
M. Gökcan and D. Çobanoğlu Aktan, “Validation of the vocabulary size test,” Egit Psikol Olcme Deger Derg, vol. 13, no. 4, pp. 305–327, Dec. 2022, doi: 10.21031/epod.1144808.
E. Moradi, Z. Ghabanchi, and R. Pishghadam, “Reading comprehension test fairness across gender and mode of learning: Insights from IRT-based differential item functioning analysis,” Language Testing in Asia, vol. 12, no. 1, p. 39, Sep. 2022, doi: 10.1186/s40468-022-00192-3.
H. Ahmad, N. Mamat, M. Che Mustafa, and S. Iryani Mohd Yusoff, “Validating the teaching, learning, and assessment quality of Malaysian ECCE instrument,” International Journal of Evaluation and Research in Education (IJERE), vol. 10, no. 1, p. 135, Mar. 2021, doi: 10.11591/ijere.v10i1.20857.
W. Astuti and Adiwijaya, “Support vector machine and principal component analysis for microarray data classification,” J Phys Conf Ser, vol. 971, p. 012003, Mar. 2018, doi: 10.1088/1742-6596/971/1/012003.
S. T. Nihan, “Karl Pearsons chi-square tests,” Educational Research and Reviews, vol. 15, no. 9, pp. 575–580, Sep. 2020, doi: 10.5897/ERR2019.3817.
W. J. van der Linden, Ed., Handbook of item response theory. Chapman and Hall/CRC, 2016. doi: 10.1201/9781315374512.
A. Darmana, A. Sutiani, and Jasmidi, “Development of the thermochemistry- HOTS-tawheed multiple choice instrument,” J Phys Conf Ser, vol. 1462, no. 1, p. 012057, Mar. 2020, doi: 10.1088/1742-6596/1462/1/012057.
W.-H. Chen and D. Thissen, “Local dependence indexes for item pairs using item response theory,” Journal of Educational and Behavioral Statistics, vol. 22, no. 3, pp. 265–289, Sep. 1997, doi: 10.3102/10769986022003265.
D. O. Tobih, M. A. Ayanwale, O. A. Ajayi, and M. V. Bolaji, “The use of measurement frameworks to explore the qualities of test items,” International Journal of Evaluation and Research in Education (IJERE), vol. 12, no. 2, p. 914, Jun. 2023, doi: 10.11591/ijere.v12i2.23747.
C. D. Desjardins and O. Bulut, Handbook of educational measurement and psychometrics using R. Boca Raton, Florida : CRC Press, [2018]: Chapman and Hall/CRC, 2018. doi: 10.1201/b20498.
S. Soeharto and B. Csapó, “Evaluating item difficulty patterns for assessing student misconceptions in science across physics, chemistry, and biology concepts,” Heliyon, vol. 7, no. 11, p. e08352, Nov. 2021, doi: 10.1016/j.heliyon.2021.e08352.
J. Jumini and H. Retnawati, “Estimating item parameters and student abilities: An IRT 2PL analysis of mathematics examination,” AL-ISHLAH: Jurnal Pendidikan, vol. 14, no. 1, pp. 385–398, Mar. 2022, doi: 10.35445/alishlah.v14i1.926.
H. Chin, C. M. Chew, W. Yew, and M. Musa, “Validating the cognitive diagnostic assessment and assessing students’ mastery of ‘parallel and perpendicular lines’ using the rasch model,” Participatory Educational Research, vol. 9, no. 6, pp. 436–452, Nov. 2022, doi: 10.17275/per.22.147.9.6.
Herwin and S. C. Dahalan, “Person fit statistics to identify irrational response patterns for multiple-choice tests in learning evaluation,” Pegem Journal of Education and Instruction, vol. 12, no. 4, pp. 39–46, Jan. 2022, doi: 10.47750/pegegog.12.04.05.
Copyright (c) 2024 Lovieanta Arriza, Heri Retnawati, Rizki Tika Ayuni
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this Journal agree to the following terms:
- Author retain copyright and grant the journal right of first publication with the work simultaneously licensed under a creative commons attribution license that allow others to share the work within an acknowledgement of the work’s authorship and initial publication of this journal.
- Authors are able to enter into separate, additional contractual arrangement for the non-exclusive distribution of the journal’s published version of the work (e.g. acknowledgement of its initial publication in this journal).
- Authors are permitted and encouraged to post their work online (e.g. in institutional repositories or on their websites) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published works.
