This study investigated the feasibility and the applicability of using the third generation computer-based assessment of complex problem solving (CPS) based on the MicroDYN approach at the Jordanian higher education level. The tests were administered online via the eDia assessment platform for all data collections processes.
We also investigated the role of strategic exploration and different problem-solving and test-taking behaviours in CPS success, using logfile data to visualize and quantify students’ problem-solving behaviour on ten CPS problems with varying difficulty levels and characteristics. Additionally, in the present study, we go beyond the borders of most studies that focus on students’ problem-solving behaviour pattern analyses in European cultures and education systems and examine Arabic students’ CPS behaviour. Results show that students in the Arabic school system interpret CPS problems the same way. That is, we confirmed the two-dimensional model of CPS, indicating the processes of knowledge acquisition and knowledge application as separate dimensions during the problem-solving process.
Analyzing log data, we have identified large differences in students’ test-taking behavior in terms of the effectiveness of their exploration strategy, time-on-task, and number of trials at an international level (Jordan and Hungary). We identified four latent classes in both samples based on the students’ exploration strategy behavior. The tested process indicators proved to be non-invariant over the different latent profiles; that is, there are big differences in the role of the number of manipulations executed, time-on-task, and type of strategy used in actual problem-solving achievement between students that fall within different thinking profiles.
Based on the results from the studies, we can conclude that online computer-based assessment is valid and reliable in the Jordanian higher education context. This study contributes to understanding how students from different educational contexts behave while solving complex problems.