We use cookies to ensure that we give you the best experience on our website. By continuing to browse this repository, you give consent for essential cookies to be used. You can read more about our Privacy and Cookie Policy.

Durham Research Online
You are in:

Does BERT pay attention to cyberbullying?

Elsafoury, Fatma and Katsigiannis, Stamos and Wilson, Steven and Ramzan, Naeem (2021) 'Does BERT pay attention to cyberbullying?', 44th International ACM SIGIR Conference on Research and Development in Information Retrieval Online, 11-15 Jul 2021.


Social media have brought threats like cyberbullying, which can lead to stress, anxiety, depression and in some severe cases, suicide attempts. Detecting cyberbullying can help to warn/ block bullies and provide support to victims. However, very few studies have used self-attention-based language models like BERT for cyberbullying detection and they typically only report BERT’s performance without examining in depth the reasons for its performance. In this work, we examine the use of BERT for cyberbullying detection on various datasets and attempt to explain its performance by analysing its attention weights and gradient-based feature importance scores for textual and linguistic features. Our results show that attention weights do not correlate with feature importance scores and thus do not explain the model’s performance. Additionally, they suggest that BERT relies on syntactical biases in the datasets to assign feature importance scores to class-related words rather than cyberbullying-related linguistic features.

Item Type:Conference item (Paper)
Full text:(AM) Accepted Manuscript
Download PDF
Publisher Web site:
Publisher statement:© Authors | ACM, 2021. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record,
Date accepted:15 April 2021
Date deposited:21 May 2021
Date of first online publication:11 July 2021
Date first made open access:16 July 2021

Save or Share this output

Look up in GoogleScholar