Deep learning for text summarization using NLP for automated news digest

Abstract Text Summarization, a vital aspect of natural language processing, aims to condense text while retaining its essential meaning. This process is achieved through extractive and abstractive methods. Deep Learning faces challenges in this domain, including semantic understanding, preservation of meaning, efficient handling of long documents, and ensuring coherence and grammatical correctness. Despite these challenges, deep learning offers advantages such as time saving, facilitating information retrieval, scalability, and content personalization. Also, deep learning faces the risk of losing important details, subjectivity in information selection, difficulty in handling complex texts, and variability in summary quality. Addressing these challenges remains an ongoing focus of research and development in the field of NLP. This paper proposes text summarization approach utilizing deep learning models, namely T5-base, T5-large, BART CNN, and PEGASUS. The methodology involves initial data cleaning and preprocessing of the dataset, followed by exploratory data analysis (EDA) to gain insights into the data. Subsequently, the Rouge and BLUE scores of each model are calculated to assess their summarization performance. After training the models, the Rouge and BLUE scores are re-evaluated to measure their effectiveness in generating summaries. The primary objective is to compare the performance of these models based on their Rouge scores, aiming to identify the model that provides the highest Rouge score, indicative of better summary quality. This study contributes to the advancement of text summarization techniques and provide insights into the effectiveness of various deep learning models in this domain.