Bilingual Code-Switching Using XLM-RoBERTa

This study discusses the linguistic complexities of code-switching Tagalog-English (Taglish), for natural language processing (NLP) applications with text classification using the XLM-RoBERTa Large model. Code-switching presents challenges due to its informal syntax, frequent language alternation, and linguistic ambiguities. Using a custom SentencePiece tokenizer and a preprocessed dataset of ambiguity-tagged Taglish reviews, the model was evaluated on the aforementioned key performance metrics: precision, F1-score, and ROC-AUC. Testing results showed that XLM-RoBERTa Large was the best-performing model in terms of testing precision at 97.62%, F1score at 95.77%, and ROC-AUC at 97.50%. Such measurements indicated that the model possessed a significant ability to face linguistic difficulties such as syntactic ambiguities, mixed-language constructs, and informal expressions. A comparative analysis was undertaken by testing XLM-RoBERTa Large against its Base variant and the multilingual BERT model (mBERT) in handling bilingual text classification tasks. The key innovations of the study were advanced preprocessing techniques including language tagging, ambiguity tagging and customized tokenization. These enhanced the model's ability to handle informal and code-switched text. The methodologies provided the model with the ability to classify constructs with mixedlanguage elements and informal expressions effectively. Thus, the study concludes that XLM-RoBERTa Large is a very powerful tool in addressing linguistic ambiguities in under-resourced and multilingual settings. Future research involves larger and even more diverse data sets, state-of-the art optimization strategies, and the further exploration of integration of contextual embeddings to further make the model stronger and scalable. This research serves as a valuable contribution to NLP tools effectiveness in bilingual text and code switching, opening room for further strides in multilingual language processing Philippine and global scales.