Improving chinese hate speech detection with bert-fasttext fusion and BERT-BiLSTM fusion

Hate speech detection is an essential technique in the online environment, especially on social media platforms. This technique helps to create a safer space and reduce the risk of real-world harm. In Chinese, this task is particularly challenging because of unique linguistic structures and the frequent use of indirect expressions, sarcasm, homophones, character variants, and abbreviations. This study investigates how to improve Chinese hate speech detection by combining BERT with FastText and BERT with BiLSTM. There are six model variants that are configured: frozen BERT and fine-tuned BERT, each further extended with either FastText sentence embeddings or a clause-level BiLSTM. Experiments are conducted on a self-annotated Chinese social media dataset and the public COLDataset corpus, including a cross-dataset setting where models are trained on the self-annotated data and evaluated on COLDataset. The results show that fine-tuned BERT is the main factor of performance gain, and that combining FastText or BiLSTM improves over the corresponding BERT baselines. Among all models, fine-tuned BERT combined with FastText achieves the best in-domain performance, reaching 92.58% accuracy on the self-annotated dataset, while also having strong ROC–AUC in the cross-dataset evaluation. Overall, these findings indicate that simple feature-level fusion of BERT with lexical or clause-level information is an effective and computationally practical way to improve Chinese hate speech detection.