Search engine for discovering works of Art, research articles, and books related to Art and Culture
Javascript must be enabled to continue!
The Emergence of Chunking Structures with Hierarchical RNN
View through CrossRef
Abstract
In Natural Language Processing (NLP), predicting linguistic structures, such as parsing and chunking, has mostly relied on manual annotations of syntactic structures. This article introduces an unsupervised approach to chunking, a syntactic task that involves grouping words in a non-hierarchical manner. We present a Hierarchical Recurrent Neural Network (HRNN) designed to model word-to-chunk and chunk-to-sentence compositions. Our approach involves a two-stage training process: pretraining with an unsupervised parser and finetuning on downstream NLP tasks. Experiments on multiple datasets reveal a notable improvement of unsupervised chunking performance in both pretraining and finetuning stages. Interestingly, we observe that the emergence of the chunking structure is transient during the neural model’s downstream-task training. This study contributes to the advancement of unsupervised syntactic structure discovery and opens avenues for further research in linguistic theory.1
Title: The Emergence of Chunking Structures with Hierarchical
RNN
Description:
Abstract
In Natural Language Processing (NLP), predicting linguistic structures, such as parsing and chunking, has mostly relied on manual annotations of syntactic structures.
This article introduces an unsupervised approach to chunking, a syntactic task that involves grouping words in a non-hierarchical manner.
We present a Hierarchical Recurrent Neural Network (HRNN) designed to model word-to-chunk and chunk-to-sentence compositions.
Our approach involves a two-stage training process: pretraining with an unsupervised parser and finetuning on downstream NLP tasks.
Experiments on multiple datasets reveal a notable improvement of unsupervised chunking performance in both pretraining and finetuning stages.
Interestingly, we observe that the emergence of the chunking structure is transient during the neural model’s downstream-task training.
This study contributes to the advancement of unsupervised syntactic structure discovery and opens avenues for further research in linguistic theory.
1.
Related Results
Energy-efficient architectures for recurrent neural networks
Energy-efficient architectures for recurrent neural networks
Deep Learning algorithms have been remarkably successful in applications such as Automatic Speech Recognition and Machine Translation. Thus, these kinds of applications are ubiquit...
Enhancing RAG Performance Through Chunking and Text Splitting Techniques
Enhancing RAG Performance Through Chunking and Text Splitting Techniques
In the world of Generative Artificial Intelligence (GenAI) and Large Language Models (LLM), Retrieval-Augmented Generation (RAG) has transformed the way we interact with data. Usin...
Advanced Chunking Techniques: a Novel Approach for Semantic Splitters
Advanced Chunking Techniques: a Novel Approach for Semantic Splitters
Chunking, the process of splitting large amounts of text into processable parts, is an essential but often overlooked step for multiple Information Retrieval and Vector Databases t...
Chunking in the Second Language: Implications for Language Learning and Teaching
Chunking in the Second Language: Implications for Language Learning and Teaching
Among the various challenges that adult and other late language learners face on their journey to achieving nativelike proficiency, chunking has been identified as one of the most ...
Klasifikasi Berita Bahasa Indonesia dengan Recurrent Neural Network (RNN)
Klasifikasi Berita Bahasa Indonesia dengan Recurrent Neural Network (RNN)
Penelitian ini bertujuan untuk mengembangkan model klasifikasi dokumen berita berbahasa Indonesia menggunakan Recurrent Neural Network (RNN). Dengan kemajuan teknologi informasi da...
Hierarchical Zeolites from Production Sand Waste as Catalysts for CO2 to Carbon Nanotubes CNTs: Exploration and Production Sustainability
Hierarchical Zeolites from Production Sand Waste as Catalysts for CO2 to Carbon Nanotubes CNTs: Exploration and Production Sustainability
Abstract
This project targets to convert sand waste from oil & gas production, which is typically disposed as landfill, to be the higher-value products, called "...
Quantifying corn emergence using UAV imagery and machine learning
Quantifying corn emergence using UAV imagery and machine learning
Corn (Zea mays L.) is one of the important crops in the United States for animal feed, ethanol production, and human consumption. To maximize the final corn yield, one of the criti...
Power and Empowerment in Strict Hierarchies
Power and Empowerment in Strict Hierarchies
Hierarchically structured organisations endow its members with distinct institutional roles. In the context of decision-making, these roles depend on the decision rights individual...