These labels were automatically added by AI and may be inaccurate. For details, see About Literature Database.
Abstract
This full paper in the research track evaluates the usage of data logged from
cybersecurity exercises in order to predict students who are potentially at
risk of performing poorly. Hands-on exercises are essential for learning since
they enable students to practice their skills. In cybersecurity, hands-on
exercises are often complex and require knowledge of many topics. Therefore,
students may miss solutions due to gaps in their knowledge and become
frustrated, which impedes their learning. Targeted aid by the instructor helps,
but since the instructor's time is limited, efficient ways to detect struggling
students are needed. This paper develops automated tools to predict when a
student is having difficulty. We formed a dataset with the actions of 313
students from two countries and two learning environments: KYPO CRP and
EDURange. These data are used in machine learning algorithms to predict the
success of students in exercises deployed in these environments. After
extracting features from the data, we trained and cross-validated eight
classifiers for predicting the exercise outcome and evaluated their predictive
power. The contribution of this paper is comparing two approaches to feature
engineering, modeling, and classification performance on data from two learning
environments. Using the features from either learning environment, we were able
to detect and distinguish between successful and struggling students. A
decision tree classifier achieved the highest balanced accuracy and sensitivity
with data from both learning environments. The results show that activity data
from cybersecurity exercises are suitable for predicting student success. In a
potential application, such models can aid instructors in detecting struggling
students and providing targeted help. We publish data and code for building
these models so that others can adopt or adapt them.