Cookies

We use cookies to ensure that we give you the best experience on our website. By continuing to browse this repository, you give consent for essential cookies to be used. You can read more about our Privacy and Cookie Policy.


Durham Research Online
You are in:

Interpretable AI to Understand Early Effective and Ineffective Programming Behaviours from CS1 Learners

Pereira, Filipe Dwan and Oliveira, Elaine Harada Teixeira de and Oliveira, David Braga Fernandes de and Carvalho, Leandro Silva Galvão de and Cristea, Alexandra I. (2021) 'Interpretable AI to Understand Early Effective and Ineffective Programming Behaviours from CS1 Learners.', Anais Estendidos do I Simpósio Brasileiro de Educação em Computação (EDUCOMP Estendido 2021) Online, 26-30 April 2021.

Abstract

Building predictive models to estimate the learner performance in the beginning of CS1 courses is essential in education to allow early interventions. However, the educational literature notes the lack of studies on early learner behaviours that can be effective or ineffective, that is, programming behaviours that potentially lead to success or failure, respectively. Hence, beyond the prediction, it is crucial to explain what leads the predictive model to make the decisions (e.g., why a given student s is classified as `passed'), which would allow a better understanding of which early programming behaviours are to be encouraged and triggered. In this work in progress, we use a state-of-the-art unified approach to interpret black-box model predictions, which uses SHapley Additive exPlanations (SHAP) method. SHAP method can be used to explain linearly a complex model (e.g. DL or XGboost) in instance level. In our context of CS1 performance prediction, this method gets the predictive model and the features values for a given student as input and the possibility of explanation of which feature values are increasing or decreasing the learner chances of passing as output. That is, using SHAP we can identify early effective and ineffective behaviours in student-level granularity. More than that, using this local explanation as building blocks, we can also extract global data insight and give a summarisation of the model. A video explaining this work can be found at the following link (in Brazilian Portuguese): https://youtu.be/pd6Ma6uInHo

Item Type:Conference item (Paper)
Full text:(VoR) Version of Record
Available under License - Creative Commons Attribution Non-commercial 4.0.
Download PDF
(491Kb)
Status:Peer-reviewed
Publisher Web site:https://doi.org/10.5753/educomp_estendido.2021.14853
Publisher statement:The author(s) or third-parties are allowed to reproduce or distribute, in part or in whole, the material extracted from this work, in textual form, adapted or remixed, as well as the creation or production based on its content, for non-commercial purposes, since the proper credit is provided to the original creation, under CC BY-NC 4.0 License. EduComp’21, Abril 27–30, 2021, Jataí, Goiás, Brasil (On-line)
Date accepted:No date available
Date deposited:05 November 2021
Date of first online publication:26 April 2021
Date first made open access:05 November 2021

Save or Share this output

Export:
Export
Look up in GoogleScholar