Pereira, Filipe Dwan and Oliveira, Elaine Harada Teixeira de and Oliveira, David Braga Fernandes de and Carvalho, Leandro Silva Galvão de and Cristea, Alexandra I. (2021) 'Interpretable AI to Understand Early Effective and Ineffective Programming Behaviours from CS1 Learners.', Anais Estendidos do I Simpósio Brasileiro de Educação em Computação (EDUCOMP Estendido 2021) Online, 26-30 April 2021.
Building predictive models to estimate the learner performance in the beginning of CS1 courses is essential in education to allow early interventions. However, the educational literature notes the lack of studies on early learner behaviours that can be effective or ineffective, that is, programming behaviours that potentially lead to success or failure, respectively. Hence, beyond the prediction, it is crucial to explain what leads the predictive model to make the decisions (e.g., why a given student s is classified as `passed'), which would allow a better understanding of which early programming behaviours are to be encouraged and triggered. In this work in progress, we use a state-of-the-art unified approach to interpret black-box model predictions, which uses SHapley Additive exPlanations (SHAP) method. SHAP method can be used to explain linearly a complex model (e.g. DL or XGboost) in instance level. In our context of CS1 performance prediction, this method gets the predictive model and the features values for a given student as input and the possibility of explanation of which feature values are increasing or decreasing the learner chances of passing as output. That is, using SHAP we can identify early effective and ineffective behaviours in student-level granularity. More than that, using this local explanation as building blocks, we can also extract global data insight and give a summarisation of the model. A video explaining this work can be found at the following link (in Brazilian Portuguese): https://youtu.be/pd6Ma6uInHo
|Item Type:||Conference item (Paper)|
|Full text:||(VoR) Version of Record|
Available under License - Creative Commons Attribution Non-commercial 4.0.
Download PDF (491Kb)
|Publisher Web site:||https://doi.org/10.5753/educomp_estendido.2021.14853|
|Publisher statement:||The author(s) or third-parties are allowed to reproduce or distribute, in part or in whole, the material extracted from this work, in textual form, adapted or remixed, as well as the creation or production based on its content, for non-commercial purposes, since the proper credit is provided to the original creation, under CC BY-NC 4.0 License. EduComp’21, Abril 27–30, 2021, Jataí, Goiás, Brasil (On-line)|
|Date accepted:||No date available|
|Date deposited:||05 November 2021|
|Date of first online publication:||26 April 2021|
|Date first made open access:||05 November 2021|
Save or Share this output
|Look up in GoogleScholar|