Search engine for discovering works of Art, research articles, and books related to Art and Culture
Javascript must be enabled to continue!
The Emergence of Chunking Structures with Hierarchical RNN
View through CrossRef
Abstract
In Natural Language Processing (NLP), predicting linguistic structures, such as parsing and chunking, has mostly relied on manual annotations of syntactic structures. This article introduces an unsupervised approach to chunking, a syntactic task that involves grouping words in a non-hierarchical manner. We present a Hierarchical Recurrent Neural Network (HRNN) designed to model word-to-chunk and chunk-to-sentence compositions. Our approach involves a two-stage training process: pretraining with an unsupervised parser and finetuning on downstream NLP tasks. Experiments on multiple datasets reveal a notable improvement of unsupervised chunking performance in both pretraining and finetuning stages. Interestingly, we observe that the emergence of the chunking structure is transient during the neural model’s downstream-task training. This study contributes to the advancement of unsupervised syntactic structure discovery and opens avenues for further research in linguistic theory.1
Title: The Emergence of Chunking Structures with Hierarchical
RNN
Description:
Abstract
In Natural Language Processing (NLP), predicting linguistic structures, such as parsing and chunking, has mostly relied on manual annotations of syntactic structures.
This article introduces an unsupervised approach to chunking, a syntactic task that involves grouping words in a non-hierarchical manner.
We present a Hierarchical Recurrent Neural Network (HRNN) designed to model word-to-chunk and chunk-to-sentence compositions.
Our approach involves a two-stage training process: pretraining with an unsupervised parser and finetuning on downstream NLP tasks.
Experiments on multiple datasets reveal a notable improvement of unsupervised chunking performance in both pretraining and finetuning stages.
Interestingly, we observe that the emergence of the chunking structure is transient during the neural model’s downstream-task training.
This study contributes to the advancement of unsupervised syntactic structure discovery and opens avenues for further research in linguistic theory.
1.
Related Results
Energy-efficient architectures for recurrent neural networks
Energy-efficient architectures for recurrent neural networks
Deep Learning algorithms have been remarkably successful in applications such as Automatic Speech Recognition and Machine Translation. Thus, these kinds of applications are ubiquit...
Advanced Chunking Techniques: a Novel Approach for Semantic Splitters
Advanced Chunking Techniques: a Novel Approach for Semantic Splitters
Chunking, the process of splitting large amounts of text into processable parts, is an essential but often overlooked step for multiple Information Retrieval and Vector Databases t...
Chunking in the Second Language: Implications for Language Learning and Teaching
Chunking in the Second Language: Implications for Language Learning and Teaching
Among the various challenges that adult and other late language learners face on their journey to achieving nativelike proficiency, chunking has been identified as one of the most ...
Hierarchical Zeolites from Production Sand Waste as Catalysts for CO2 to Carbon Nanotubes CNTs: Exploration and Production Sustainability
Hierarchical Zeolites from Production Sand Waste as Catalysts for CO2 to Carbon Nanotubes CNTs: Exploration and Production Sustainability
Abstract
This project targets to convert sand waste from oil & gas production, which is typically disposed as landfill, to be the higher-value products, called "...
Quantifying corn emergence using UAV imagery and machine learning
Quantifying corn emergence using UAV imagery and machine learning
Corn (Zea mays L.) is one of the important crops in the United States for animal feed, ethanol production, and human consumption. To maximize the final corn yield, one of the criti...
Preventive Mechanisms Against Cyberbullying in Social Media Environments
Preventive Mechanisms Against Cyberbullying in Social Media Environments
Cyberbullying has become more common on social media sites. Since people of all ages use social media frequently, it's really important to make these platforms safer from cyberbull...
Crude Oil Cost Forecasting using Variants of Recurrent Neural Network
Crude Oil Cost Forecasting using Variants of Recurrent Neural Network
Crude oil cost plays very important role in the country’s economic growth. It is having close impact on economical stability of nation. Because of these reasons it is very importa...
Discriminating Unfamiliar Long Rhythmic Cycles: Influence of Memory and Chunking
Discriminating Unfamiliar Long Rhythmic Cycles: Influence of Memory and Chunking
Research on rhythm perception has largely focused on short rhythmic
cycles typical in Western music, leaving a gap in our understanding of
the perception and memory for longer and ...