Predicting Prior Experience with Programming Process Data in Upper-Secondary Introductory ProgrammingCER
Students entering introductory programming courses often have varying levels of prior experience. In this study, we investigate how students’ prior programming experience is exposed by their programming behavior. Using fine-grained programming process data (log data) from 131 upper-secondary students, we extract features related to students’ programming behaviors and train a random forest classifier to predict self-reported prior experience levels. Results show that prior experience can be predicted from programming behavior with up to 85% accuracy and an F1 score of 0.89, particularly in the early stages of the course. However, predictive performance does not improve over time, suggesting that behavioral differences associated with prior experience diminish as students progress. Feature importance analysis indicates that temporal characteristics of programming activity, particularly the timing and consistency of solving errors, are more important to the prediction than simple event counts. These findings support previous research that claims that the effects of prior experience diminish over time and contribute by showing when and alongside which programming concepts, these effects begin to fade. As such, this work provides insights into how prior experience manifests in programming behavior and highlights the potential of learning analytics for supporting curriculum design in introductory programming.