Submitted Abstract
Mathematics skills are key to modern, knowledge-based societies. The age of digitalization renders mathematics education even more crucial since it builds the starting point for all STEM-related fields that drive innovation and technological progress. This paramount importance is mirrored in the strong focus of the educational field on monitoring students’ mathematics skills, training those skills, researching their underlying cognitive processes, and later on, making them a gate-keeper for advanced studies. But assessing, training, or studying students’ mathematical competencies creates a constant demand of high quality math problems, so-called items with known psychometric characteristics. At the same time, from a pedagogical and research perspective, it’s key to not only know if a student fails at a certain math problem but to learn why, which could directly inform interventions. Those approaches could even lead to formative, individually tailored assessment programs that fuse ideas of testing and training to the benefit of each student. The current project draws on latest psychometric innovations and adopts principles of automatic item generation to respond to both needs: (a) creating empirically proven item templates that algorithmically produce large quantities of items with known characteristics in an instant, and (b) cognitively analyzing those templates to enable fine-grained feedback on students’ mathematical subskills. Research on both topics is promising, however lacking experience with highly heterogeneous student samples that show great diversity in terms of cultural and language background. Consequently, the proposed project centers around 3 research questions: 1) The development of fair and psychometric valid item templates for mathematical competencies taught in elementary school, 2) Investigation of students’ acceptance of automatically created items compared to traditionally created tasks, and 3) the potential of diagnostic classification models to show otherwise hidden strengths in students with specific backgrounds (e.g. language minorities, immigration background) and provide guidance to optimized curricula. In addition to the scientific dissemination of the project’s insights, a central outcome will be 80 empirically validated and verifiably fair mathematics item templates that could be used in various educational settings. In sum, the project will have a lasting impact on innovating test and item development in mathematics.