Article Sidebar

Download
PDF
Statistic
Read Counter : 207 Download : 111

Downloads

Download data is not yet available.

Main Article Content

Abstract

This paper aims to describe the development process of the Observation Protocol for Teaching Activities in Mathematics (POAEM) and to report the findings from the qualitative and statistical analyses used to provide evidence of validity and reliability of the information collected with the first version of the POAEM. As part of this development process, 20 teachers from Mexican primary schools were videotaped twice while teaching mathematics. The study assessed the reliability of the POAEM rubrics. Results showed that the dimensional structure of the instrument can be grouped in one factor. A generalizability study provided information on the different sources of error in the measurement, showing that the dimensions accounted for 78% of the variance. This study provides an exemplar of the design and validation of an instrument that can help other researchers develop their own instruments and data collection to generate evidence of validity and reliability in different sociocultural contexts.

Keywords

Educational Assessment Higher Order Thinking Skills Observation Protocol Primary School Mathematics Education Teaching Mathematics

Article Details

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite
Rodriguez-Martinez, L. Y., Hernandez-Martinez, P., & Perez-Martinez, M. G. (2023). Development of a protocol to measure mathematics higher-order thinking skills in Mexican primary schools. Journal on Mathematics Education, 14(4), 781–796. https://doi.org/10.22342/jme.v14i4.pp781-796
Download Citation
Endnote/Zotero/Mendeley (RIS)
BibTeX

References

  1. Aubuson, P., Burke, P., Schuck, S., & Kearney, M. (2014). Teachers choosing rich tasks: The moderating impact of technology on student learning, enjoyment, and preparation. Educational Researcher, 43(5), 219-229. https://doi.org/10.3102/0013189X14537115
  2. Ávila, A., Altamirano, A. C., Galindo, A. A. G., Ramos, M. T. G., López-Bonilla, G., & Ramírez, J. L. (2013). Una década de investigación educativa en conocimientos disciplinares en México. Colección, 9 (786074), 510881.
  3. Basto, M., & Pereira, J. M. (2012). An SPSS R-menu for ordinal factor analysis. Journal of Statistical Software, 46(4), 1-29. http://doi.org/10.18637/jss.v046.i04
  4. Bostic, J., Lesseig, K., Sherman, M., & Boston, M. (2021). Classroom observation and mathematics education research. Journal of Mathematics Teacher Education, 24(1), 5-31.
  5. Boston, M. (2012). Assessing instructional quality in mathematics. The Elementary School Journal, 113, 76-104. https://doi.org/10.1086/666387
  6. Boston, M. D., & Smith, M. S. (2009). Transforming secondary mathematics teaching: increasing the cognitive demands of instructional tasks used in teachers' classrooms. Journal of Research in Mathematics Education, 40(2), 119-156.
  7. Boston, M., & Wolf, M. (2006). Assessing academic rigor in mathematics instruction: The development of instructional quality assessment toolkit. CSE Technical Report 672. CRESST. http://www.cse.ucla.edu/products/reports/r672.pdf
  8. Burkhardt, H. (2006). Modelling in Mathematics classrooms: Reflections on past developments and the future. ZDM Mathematics Education, 38(2), 178-195 https://doi.org/10.1007/bf02655888
  9. Cevikbas, M., & Kaiser, G. (2020). Flipped classroom as a reform-oriented approach to teaching mathematics. ZDM Mathematics Education, 52(7), 1291-1305. https://doi.org/10.1007/s11858-020-01191-5
  10. Courtney, M. G. R., & Gordon, M. (2013). Determining the number of factors to retain in EFA: Using the SPSS R-Menu v2. 0 to make more judicious estimations. Practical Assessment, Research & Evaluation, 18(8), 1-14. https://doi.org/10.7275/9cf5-2m72
  11. Donovan, M., & Bransford, J. (2005). How Students Learn: History, Mathematics and Science in the Classroom. National Research Council, Committee on How People Learn: A targeted report for teachers. National Academies Press.
  12. Gleason, J, Livers, S., & Zelkowski, J. (2017) Mathematics Classroom Observation Protocol for Practices (MCOP2): A validation study. Investigations in Mathematics Learning, 9(3), 111-129. https://doi.org/10.1080/19477503.2017.1308697
  13. Gulikers, J. T., Bastiaens, T. J., & Kirschner, P. A. (2004). A five-dimensional framework for authentic assessment. Educational technology research and development, 52(3), 67-86. https://doi.org/10.1007/BF02504676
  14. Hiebert, J., Carpenter, T., Fennema, E., Fuson, K., Wearne, D. Murray, H. Oliver, A., & Humen, P. (1997). Making sense: Teaching and learning mathematics with understanding. Hienemann.
  15. Hill, H., Charalambous, Charalambos Y., & Kraft, M. (2012). When rater reliability is not enough: Teacher observation systems for generalizability study. Educational Researcher, 41(2), 56-64. https://doi.org/10.3102/0013189X12437203
  16. Hill, H., & Shih, J. (2009). Examining the quality of statistical mathematics education research. Journal of Research in Mathematics Education, 40(3), 241-250.
  17. Instituto Nacional para la Evaluación de la Educación [INEE]. (2018). Planea resultados nacionales 2018 [Planea national results]. https://www.inee.edu.mx/images/stories/2018/planea/PLANEA06_Rueda_de_prensa_27nov2018
  18. Jensen, B., Perez Martinez, M. G., & Aguilar Escobar, A. (2016). Framing and assessing classroom opportunity to learn: The case of Mexico. Assessment in Education: Principles, Policy & Practice, 23(1), 149-172.
  19. Judson, E. (2013). Development of an instrument to assess and deliberate on the integration of mathematics into student-centered science learning. School Science and Mathematics, 113(2), 56–68. https://doi.org/10.1111/ssm.12004
  20. Llinares, S. (2008). Aprendizaje del estudiante para profesor de matemáticas y el papel de los nuevos instrumentos de comunicación. III Encuentro de Programas de Formación Inicial de Profesores de Matemáticas. Universidad Pedagógica Nacional.
  21. Lloret, S., Ferretes, A., Hernández A., & Tomás, I. (2014). Exploratory Item Factor Analysis: a practical guide revised and updated. Annals of Psychology 30(3), 1151-1169. https://doi.org/10.6018/analesps.30.3.199361
  22. López & Mota, Á. (2003). Saberes científicos, humanísticos y tecnológicos: procesos de enseñanza y aprendizaje [Scientific, humanistic and technological knowledge: teaching and learning processes]. Consejo Mexicano de Investigación Educativa.
  23. Martínez Rizo, F. (2012). Procedimientos para el estudio de las prácticas docentes: Revisión de la literatura [An empirical study of the impact of formative assessment: A literature review]. Revista Electrónica de Investigación y Evaluación Educativa, 18(1), 1–22.
  24. Martínez Rizo, F., & Chávez, Y. (2015). La enseñanza de las matemáticas y Ciencias Naturales en Educación Básica en México. Revisión de literatura [Teaching Mathematics and Natural Sciences in Basic Education in Mexico. Literature review]. Universidad Autónoma de Aguascalientes.
  25. McTighe, J. & Wiggins, G. (2012). Understanding by design framework. http://www.ascd.org/ASCD/pdf/siteASCD/publications/UbD_WhitePaper0312.pdf
  26. MET Project (2010). Validation Engine for Observational Protocols. Bill y Melinda Gates Foundation.
  27. National Council of Teachers of Mathematics [NCTM] (2014). Principles to actions. Ensuring mathematical success for all. NCTM.
  28. Newmann, F., Marks, H., & Gamoran, A. (1996). Authentic pedagogy and student performance. American Journal of Education, 104(4), 280-312. https://doi.org/10.1086/444136
  29. Newmann, F.M., Lopez, G., & Bryk, A.S. (1998) The quality of intellectual work in Chicago schools: A baseline report. Consortium on Chicago School Research.
  30. Newmann, F., & Wehlage, G. (1993). Five standards of authentic instruction. Educational Leadership, 50(7), 8-12. https://www.ascd.org/el/articles/five-standards-of-authentic-instruction
  31. Organization for Economic Co-operation and Development [OECD]. (2015). Programa para la evaluación internacional de alumnos (PISA). Resultados 2015. https://www.oecd.org/pisa/PISA-2015-Mexico-ESP.pdf
  32. Organization for Economic Co-operation and Development [OECD]. (2019). Programme for International Student Assessment (PISA). Results from PISA 2018. https://www.oecd.org/pisa/publications/PISA2018_CN_MEX.pdf
  33. Osman, K. (2013). Scientific Inventive Thinking Skills in Children. In E.G. Carayannis (Ed.), Encyclopedia of Creativity, Invention, Innovation and Entrepreneurship. Springer. https://doi.org/10.1007/978-1-4614-3858-8_389
  34. Oura, G. K. (2001). Authentic task-based materials: Bringing the real world into the classroom. Sophia Junior College Faculty Bulletin, 21, 65-84.
  35. Oviedo, H. C., & Campo-Arias, A. (2005). Aproximación al uso del coeficiente alfa de Cronbach. Revista colombiana de psiquiatría, 34(4), 572-580.
  36. Picaroni, B., & Loureiro, G. (2010). Qué matemáticas se enseña en aulas de sexto año de primaria en escuelas de Latinoamérica [What mathematics is taught in sixth grade classrooms in Latin American schools]. Páginas de Educación, 3(1), 29-60. https://doi.org/10.22235/pe.v3i1.657
  37. Rodríguez-Martínez, L.Y. (2018). Diseño y validación de un protocolo de observación para evaluar las actividades de enseñanza en quinto grado de primaria en la asignatura de Matemáticas. [Tesis de Doctorado, Universidad Autónoma de Aguascalientes]. Repositorio bibliográfico Universidad Autónoma de Aguascalientes. http://hdl.handle.net/11317/1505
  38. Schlesinger, L., & Jentsch, A. (2016). Theoretical and methodological challenges in measuring instructional quality in mathematics education using classroom observations. ZDM Mathematics Education, 48(1-2), 29-40. https://doi.org/10.1007/s11858-016-0765-0
  39. Secretaría de Educación Pública. (2011). Plan de Estudios. Educación Básica. SEP.
  40. Secretaría de Educación Pública. (2012). Desafíos matemáticos [Mathematical Challenges]. Libro para el maestro. Quinto grado de primaria. SEP.
  41. Shavelson, R. J., & Webb, N. M. (1991). A primer on generalizability theory. Sage Publications.
  42. Stein, M., Grover, B., & Henningsen, M. (1996). Building student capacity for mathematical thinking and reasoning: An analysis of mathematical task used in reform classroom. American Educational Research Journal, 33, 455-488. https://doi.org/10.3102/00028312033002455
  43. Stiggins, R. (2007). Assessment for learning: An essential foundation of productive instruction. In R. Douglas (Ed.), Ahead of the curve: The power of assessment to transform teaching and learning (pp. 59-76). Solution Tree Press.
  44. Stone, R. H., Boon, R. T., Fore III, C., Bender, W. N., & Spencer, V. G. (2008). Use of text maps to improve the reading comprehension skills among students in high school with emotional and behavioral disorders. Behavioral Disorders, 33(2), 87-98. https://doi.org/10.1177/019874290803300203
  45. Stylianides, A. J., & Stylianides, G. J. (2013). Seeking research-grounded solutions to problems of practice: Classroom-based interventions in mathematics education. ZDM, 45, 333-341. https://doi.org/10.1007/s11858-013-0501-y
  46. Swan, M. (2015). Designing tasks and lesson that develop conceptual understanding, strategic competence and critical awareness. Center for Research in Mathematics Education: University of Nottingham.
  47. Turner, R. C., Keiffer, E. A., & Salamo, G. J. (2018). Observing inquiry-based learning environments using the Scholastic Inquiry Observation Instrument. International Journal of Science and Mathematics Education, 16(1), 1455-1478. https://doi.org/10.1007/s10763-017-9843-1
  48. UNESCO (2021). Los aprendizajes fundamentales en America Latina y el Caribe. Estudio Regional Comparativo y Explicativo (ERCE 2019). Resumen ejecutivo. UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000380257
  49. Walkowiak, T. A., Berry, R. Q., Meyer, J. P., Rimm-Kaufman, S. E., & Ottmar, E. R. (2013). Introducing an observational measure of standards-based mathematics teaching practices: Evidence of validity and score reliability. Educational Studies in Mathematics, 85(1), 109-128. https://doi.org/10.1007/s10649-013-9499-x
  50. Wiggins, G. (1998). Educative assessment: Designing assessments to inform and improve student performance. Jossey-Bass Publishers. https://doi.org/10.22235/pe.v1i1.718
  51. Zolkower, B., & Bressan, A. (2012). Educación Matemática Realista [Realistic Math Education]. In M. Pochulu & M. Rodríguez (Eds.), Educación matemática. Aportes a la formación docente desde distintos enfoques teóricos. (pp. 175-200). Universitaria de Villa María y Universidad Nacional de Gral. Sarmiento.