Article Sidebar

Download
PDF
Statistic
Read Counter : 816 Download : 352

Downloads

Download data is not yet available.

Main Article Content

Abstract

Science, technology, engineering, and mathematics (STEM) problem-solving is necessary to be infused into the classroom. Nevertheless, the criticism of underrepresented mathematics in STEM problem-solving assessment is an issue. In this study, we develop and investigate the psychometric evidence of an integrated STEM-based mathematical problem-solving test. The product of the test was a mathematical essay test that contains three scientific scenarios related to the environment in every middle school grade. The mathematical contents were integrated into engineering-based design using the technology. Three experts filled an assessment sheet to assess content validity, which was analyzed using a content validity index (CVI) and intraclass correlation coefficient (ICC). The result of content validity revealed that overall items were valid and reliable. The construct validity was examined using the Rasch analysis from the data of Grades 7–9 students in Indonesia (n = 286). The construct of all scenarios and prompting items indicated fit with various difficulty levels and acceptable discrimination value. Nevertheless, four prompting items were reported as misfit based on unweighted mean square value. The recommendation for improvement is emphasized in the language clarity aspect. The inter-rater reliability was also declared good. A further study is suggested to provide a computer-based test.

Keywords

Development Mathematical Problem-Solving Psychometric Evidence STEM Test

Article Details

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite
Amalina, I. K., & Vidákovich, T. (2022). An integrated STEM-based mathematical problem-solving test: Developing and reporting psychometric evidence. Journal on Mathematics Education, 13(4), 587–604. https://doi.org/10.22342/jme.v13i4.pp587-604
Download Citation
Endnote/Zotero/Mendeley (RIS)
BibTeX

References

  1. Amalina, I. K., & Vidákovich, T. (2022a). A mathematical problem-solving framework-based Integrated STEM: Theory and practice. International Journal of Trends in Mathematics Education Research, 5(1), 1–11. https://doi.org/10.33122/ijtmer.v5i1.105
  2. Amalina, I. K., & Vidákovich, T. (2022b). Assessment in STEM Problem-Solving: A Systematic Review. The International Journal of Assessment and Evaluation, 29(2), 64–80. https://doi.org/https://doi.org/10.18848/2327-7920/CGP/v29i02/63-80
  3. Andrich, D. (2018). Advances in social measurement: A Rasch measurement theory. In F. Guillemin, A. Leplège, A. Briançon, E. Spitz, & J. Coste (Eds.), Perceived health and adaptation in chronic disease (pp. 66–91). https://psycnet.apa.org/doi/10.1201/9781315155074-7
  4. Annaggar, A., & Tiemann, R. (2020). A video game as an assessment tool of problem-solving competence. Gesellschaft Fur Didactik Der Chemie Und Physik (GDCP), April, 377–380.
  5. Bond, T. G., & Fox, C. M. (2015). Applying the Rasch Model : Fundamental Measurement in the Human Sciences Third Edition. Routledge.
  6. Bostic, J. D., & Sondergeld, T. A. (2015). Measuring Sixth-Grade Students’ Problem Solving: Validating an Instrument Addressing the Mathematics Common Core. School Science and Mathematics, 115(6), 281–291. https://doi.org/10.1111/ssm.12130
  7. Docktor, J. L., Dornfeld, J., Frodermann, E., Heller, K., Hsu, L., Jackson, K. A., Mason, A., Ryan, Q. X., & Yang, J. (2016). Assessing student written problem solutions: A problem-solving rubric with application to introductory physics. Physical Review Physics Education Research, 12(1), 1–18. https://doi.org/10.1103/PhysRevPhysEducRes.12.010130
  8. Du Toit, S., & Du Toit, G. (2013). Learner metacognition and mathematics achievement during problem-solving in a mathematics classroom. The Journal for Transdisciplinary Research in Southern Africa, 9(3), 505–518. https://doi.org/10.4102/td.v9i3.194
  9. Gao, X., Li, P., Shen, J., & Sun, H. (2020). Reviewing assessment of student learning in interdisciplinary STEM education. International Journal of STEM Education, 7(1). https://doi.org/10.1186/s40594-020-00225-4
  10. Hyrkäs, K., Appelqvist-Schmidlechner, K., & Oksa, L. (2003). Validating an instrument for clinical supervision using an expert panel. International Journal of Nursing Studies, 40(6), 619–625. https://doi.org/10.1016/S0020-7489(03)00036-1
  11. İncebacak, B. B., & Ersoy, E. (2016). Problem Solving Skills of Secondary School Students. China-USA Business Review, 15(6), 275–285. https://doi.org/10.17265/1537-1514/2016.06.002
  12. Jolly, A. (2016). STEM by design: Strategies and activities for grades 4-8. Routledge. https://doi.org/10.4324/9781315679976
  13. Jones, I., Swan, M., & Pollitt, A. (2015). Assessing Mathematical Problem Solving Using Comparative Judgement. International Journal of Science and Mathematics Education, 13(1), 151–177. https://doi.org/10.1007/s10763-013-9497-6
  14. Junpeng, P., Marwiang, M., Chiajunthuk, S., Suwannatrai, P., Chanayota, K., Pongboriboon, K., Tang, K. N., & Wilson, M. (2020). Validation of a digital tool for diagnosing mathematical proficiency. International Journal of Evaluation and Research in Education, 9(3), 665–674. https://doi.org/10.11591/ijere.v9i3.20503
  15. Karatas, I., & Baki, A. (2013). The Effect of Learning Environments Based on Problem Solving on Students ’ Achievements of Problem Solving. International Electronic Journal of Elementary Education, 5(3), 249–267. https://www.iejee.com/index.php/IEJEE/article/view/25
  16. Ke, F., & M. Clark, K. (2020). Game-Based Multimodal Representations and Mathematical Problem Solving. International Journal of Science and Mathematics Education, 18(1), 103–122. https://doi.org/10.1007/s10763-018-9938-3
  17. Kelley, T. R., & Knowles, J. G. (2016). A conceptual framework for integrated STEM education. International Journal of STEM Education, 3(1). https://doi.org/10.1186/s40594-016-0046-z
  18. Kesorn, N., Junpeng, P., Marwiang, M., Pongboriboon, K., Tang, K. N., Bathia, S., & Wilson, M. (2020). Development of an assessment tool for mathematical reading, analytical thinking and mathematical writing. International Journal of Evaluation and Research in Education, 9(4), 955–962. https://doi.org/10.11591/ijere.v9i4.20505
  19. Khairani, A. Z., & Sahari Nordin, M. (2011). The Development and Construct Validation of the Mathematics Proficiency Test for 14-Year-Old Students. Asia Pacific Journal of Educators and Education, 26(1), 33–50. http://web.usm.my/apjee/webtest/APJEE_26.1.2011/26.1.2011_33-50.pdf
  20. Koo, T. K., & Li, M. Y. (2016). A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
  21. Lasa, A., Abaurrea, J., & Iribas, H. (2020). MATHEMATICAL CONTENT on STEM ACTIVITIES. Journal on Mathematics Education, 11(3), 333–346. https://doi.org/10.22342/JME.11.3.11327.333-346
  22. López-Pina, J. A., Meseguer-Henarejos, A. B., Gascón-Cánovas, J. J., Navarro-Villalba, D. J., Sinclair, V. G., & Wallston, K. A. (2016). Measurement properties of the brief resilient coping scale in patients with systemic lupus erythematosus using rasch analysis. Health and Quality of Life Outcomes, 14(1), 1–8. https://doi.org/10.1186/s12955-016-0534-3
  23. Maass, K., Geiger, V., Ariza, M. R., & Goos, M. (2019). The Role of Mathematics in interdisciplinary STEM education. ZDM - Mathematics Education, 51(6), 869–884. https://doi.org/10.1007/s11858-019-01100-5
  24. Priemer, B., Eilerts, K., Filler, A., Pinkwart, N., Rösken-Winter, B., Tiemann, R., & Zu Belzen, A. U. (2020). A framework to foster problem-solving in STEM and computing education. Research in Science and Technological Education, 38(1), 105–130. https://doi.org/10.1080/02635143.2019.1600490
  25. Salazar-Torres, J., Rincón Leal, O., & Vergel Ortega, M. (2021). The rubric as an assessment tool for solving problem situations in the physics and mathematics teaching context. Journal of Physics: Conference Series, 1981(1). https://doi.org/10.1088/1742-6596/1981/1/012018
  26. Scherer, R., & Tiemann, R. (2012). Factors of problem-solving competency in a virtual chemistry environment: The role of metacognitive knowledge about strategies. Computers and Education, 59(4), 1199–1214. https://doi.org/10.1016/j.compedu.2012.05.020
  27. Shanta, S. (2019). Developing and assessing authentic problem-solving skills in high school pre-engineering students. ASEE Annual Conference and Exposition, Conference Proceedings. https://doi.org/10.18260/1-2--32637
  28. Shute, V. J., Wang, L., Greiff, S., Zhao, W., & Moore, G. (2016). Measuring problem solving skills via stealth assessment in an engaging video game. Computers in Human Behavior, 63, 106–117. https://doi.org/10.1016/j.chb.2016.05.047
  29. Suratno, Wahono, B., Chang, C. Y., Retnowati, A., & Yushardi, Y. (2020). Exploring a Direct Relationship between Students’ Problem-Solving Abilities and Academic Achievement: A STEM Education at a Coffee Plantation Area. Journal of Turkish Science Education, 17(2), 211–224. https://doi.org/10.36681/tused.2020.22
  30. Tan Sisman, G., & Aksu, M. (2016). A Study on Sixth Grade Students’ Misconceptions and Errors in Spatial Measurement: Length, Area, and Volume. International Journal of Science and Mathematics Education, 14(7), 1293–1319. https://doi.org/10.1007/s10763-015-9642-5
  31. Tasir, Z., Mohamad, M. N. M., Ismail, Z., & Mustafa, N. (2018). Development and Validation of Problem Solving Task Based-Integrated STEM. Proceedings - 2018 6th International Conference on Learning and Teaching in Computing and Engineering, LaTiCE 2018, 17–23. https://doi.org/10.1109/LaTICE.2018.00011
  32. Velasquez, Q. F., & Bueno, D. C. (2019). Metacognitive Skills in Problem Solving of Senior High School STEM Strand Students Metacognitive Skills in Problem-Solving among Senior High School STEM Strand Students. Institutional Multidisciplinary Research and Development, 2(June), 124–129. https://doi.org/10.13140/RG.2.2.18820.99207
  33. Wells, J. G. (2016). PIRPOSAL model of integrative STEM education: Conceptual and pedagogical framework for classroom implementation. Technology and Engineering Teacher, 75(6), 12–19. http://america.aljazeera.com/articles/2013/9/16/many-us-bridges-arestructurallyunsoundsaysnewreport.html
  34. Widodo, S. A., Ibrahim, I., Hidayat, W., Maarif, S., & Sulistyowati, F. (2021). Development of Mathematical Problem Solving Tests on Geometry for Junior High School Students. Jurnal Elemen, 7(1), 221–231. https://doi.org/10.29408/jel.v7i1.2973