DEPENDENCY TREE PARSING USING TRANSFORMER-BASED LANGUAGE REPRESENTATIONS

Dependency parsing in Myanmar face many challenges due to agglutinative morphology, flexible word order, and lack of explicit word boundaries. This paper addresses these challenges by investigating the performance of Myanmar dependency parsing based three pretrained-transformer models: XLM-RoBERTa-Large, XLM-RoBERTa-base-Longformer-4096, and language-specific model, MyanBERTa. The proposed methodology utilizes a parameter-efficient model with adapter layers and a biaffine parsing mechanism by optimizing multi-task learning objective across six linguistic prediction tasks. The experimental results demonstrated that XLM-RoBERTa-Large achieves the highest Labeled Attachment Score (LAS) and Unlabeled Attachment Score (UAS) on both development set and test set. Despite XLM-RoBERTa-base-Longformer-4096 model can extended its input capacity and MyanBERTa is pretrained on a language capacity, their performance drop compared to XLM-RoBERTa-Large. These findings suggest that the scale and architectural strength of models are more critical for high-performance dependency parsing than models with an extended input capacity or those trained on a language-specific corpus.