Skip to main content

Research Repository

Advanced Search

Interpretable AI to Understand Early Effective and Ineffective Programming Behaviours from CS1 Learners

Pereira, Filipe Dwan; Oliveira, Elaine Harada Teixeira de; Oliveira, David Braga Fernandes de; Carvalho, Leandro Silva Galvão de; Cristea, Alexandra I.

Interpretable AI to Understand Early Effective and Ineffective Programming Behaviours from CS1 Learners Thumbnail


Authors

Filipe Dwan Pereira

Elaine Harada Teixeira de Oliveira

David Braga Fernandes de Oliveira

Leandro Silva Galvão de Carvalho



Abstract

Building predictive models to estimate the learner performance in the beginning of CS1 courses is essential in education to allow early interventions. However, the educational literature notes the lack of studies on early learner behaviours that can be effective or ineffective, that is, programming behaviours that potentially lead to success or failure, respectively. Hence, beyond the prediction, it is crucial to explain what leads the predictive model to make the decisions (e.g., why a given student s is classified as `passed'), which would allow a better understanding of which early programming behaviours are to be encouraged and triggered. In this work in progress, we use a state-of-the-art unified approach to interpret black-box model predictions, which uses SHapley Additive exPlanations (SHAP) method. SHAP method can be used to explain linearly a complex model (e.g. DL or XGboost) in instance level. In our context of CS1 performance prediction, this method gets the predictive model and the features values for a given student as input and the possibility of explanation of which feature values are increasing or decreasing the learner chances of passing as output. That is, using SHAP we can identify early effective and ineffective behaviours in student-level granularity. More than that, using this local explanation as building blocks, we can also extract global data insight and give a summarisation of the model. A video explaining this work can be found at the following link (in Brazilian Portuguese): https://youtu.be/pd6Ma6uInHo

Citation

Pereira, F. D., Oliveira, E. H. T. D., Oliveira, D. B. F. D., Carvalho, L. S. G. D., & Cristea, A. I. (2021). Interpretable AI to Understand Early Effective and Ineffective Programming Behaviours from CS1 Learners. . https://doi.org/10.5753/educomp_estendido.2021.14853

Conference Name Anais Estendidos do I Simpósio Brasileiro de Educação em Computação (EDUCOMP Estendido 2021)
Conference Location Online
Start Date Apr 26, 2021
End Date Apr 30, 2021
Online Publication Date Apr 26, 2021
Publication Date 2021
Deposit Date Nov 5, 2021
Publicly Available Date Mar 29, 2024
Pages 16-17
DOI https://doi.org/10.5753/educomp_estendido.2021.14853

Files

Published Conference Proceeding (502 Kb)
PDF

Publisher Licence URL
http://creativecommons.org/licenses/by-nc/4.0/

Copyright Statement
The author(s) or third-parties are allowed to reproduce or distribute, in part or in whole,
the material extracted from this work, in textual form, adapted or remixed, as well as
the creation or production based on its content, for non-commercial purposes, since
the proper credit is provided to the original creation, under CC BY-NC 4.0 License.
EduComp’21, Abril 27–30, 2021, Jataí, Goiás, Brasil (On-line)





You might also like



Downloadable Citations