Javascript must be enabled to continue!
Fine-Tuning Pre-Trained language models for Vietnamese: A comparative study of full fine-tuning and Lora
View through CrossRef
Pre-trained language models have brought significant improvements to Vietnamese natural language processing tasks. However, full fine-tuning of these large models remains resource-intensive and poses challenges for settings with limited computational capacity. This paper presents a comparative study between full fine-tuning and Low-Rank Adaptation – a parameter-efficient fine-tuning method, focusing on the trade-off between model performance and resource usage. Experiments are conducted on two core NLP tasks – sentiment analysis and named entity recognition – using benchmark Vietnamese datasets and pre-trained models such as PhoBERT, BARTpho and ViT5. The results show that LoRA achieves comparable accuracy to full fine-tuning while significantly reducing training cost, especially in transformer-based architectures. These findings suggest that LoRA is a viable and efficient alternative to full fine-tuning for fine-tuning Vietnamese PLMs in low-resource environments. Our work provides practical insights and experimental benchmarks to support informed decision-making in selecting fine-tuning strategies for Vietnamese NLP applications.
Ho Chi Minh City University of Foreign Languages and Information Technology
Title: Fine-Tuning Pre-Trained language models for Vietnamese: A comparative study of full fine-tuning and Lora
Description:
Pre-trained language models have brought significant improvements to Vietnamese natural language processing tasks.
However, full fine-tuning of these large models remains resource-intensive and poses challenges for settings with limited computational capacity.
This paper presents a comparative study between full fine-tuning and Low-Rank Adaptation – a parameter-efficient fine-tuning method, focusing on the trade-off between model performance and resource usage.
Experiments are conducted on two core NLP tasks – sentiment analysis and named entity recognition – using benchmark Vietnamese datasets and pre-trained models such as PhoBERT, BARTpho and ViT5.
The results show that LoRA achieves comparable accuracy to full fine-tuning while significantly reducing training cost, especially in transformer-based architectures.
These findings suggest that LoRA is a viable and efficient alternative to full fine-tuning for fine-tuning Vietnamese PLMs in low-resource environments.
Our work provides practical insights and experimental benchmarks to support informed decision-making in selecting fine-tuning strategies for Vietnamese NLP applications.
Related Results
Hubungan Perilaku Pola Makan dengan Kejadian Anak Obesitas
Hubungan Perilaku Pola Makan dengan Kejadian Anak Obesitas
<p><em><span style="font-size: 11.0pt; font-family: 'Times New Roman',serif; mso-fareast-font-family: 'Times New Roman'; mso-ansi-language: EN-US; mso-fareast-langua...
Vietnamese Language Teaching Program for Foreigners via Vietnamese Traditional Music
Vietnamese Language Teaching Program for Foreigners via Vietnamese Traditional Music
Traditional music has an important function in the process of second language teaching. In the case of Vietnamese, traditional Vietnamese music and language share an intrinsic rela...
Application of the low-rank adaptation method on the example of fine-tuning a latent diffusion model
Application of the low-rank adaptation method on the example of fine-tuning a latent diffusion model
This article explores the Low-Rank Adaptation (LoRA) method, a fast fine-tuning technique for large-parameter neural networks, and its potential application in various fields, with...
Primerjalna književnost na prelomu tisočletja
Primerjalna književnost na prelomu tisočletja
In a comprehensive and at times critical manner, this volume seeks to shed light on the development of events in Western (i.e., European and North American) comparative literature ...
Učinak poučavanja razrednomu jeziku u izobrazbi nastavnika njemačkoga
Učinak poučavanja razrednomu jeziku u izobrazbi nastavnika njemačkoga
The actual use of classroom language is principally limited to the classroom environment. As far as foreign language learning is concerned, the classroom often turns out to be the ...
An Overview of Late-onset Rheumatoid Arthritis
An Overview of Late-onset Rheumatoid Arthritis
It is well known that rheumatoid arthritis (RA) has an increased incidence in young and middle-aged adults. When this disease begins in adults between the ages of 60 and 65, it is ...
The English and Vietnamese of Vietnamese immigrants in the US
The English and Vietnamese of Vietnamese immigrants in the US
This study focuses on the Vietnamese community in Texas, aiming to explore the characteristics of English and Vietnamese languages among Vietnamese Americans in the United States w...
Vietnamese–Australian children’s language proficiency and use
Vietnamese–Australian children’s language proficiency and use
Aim:
To explore Vietnamese–Australian children’s proficiency and use of Vietnamese and English and identify associated factors that are related to demographics,...

