Topic Extraction from Millions of Tweets Based on Community Detection in Bipartite Networks

Social media offers a wealth of insight into how significant topics such as the Great East Japan Earthquake, the Arab Spring, and the Boston Bombing affect individuals. The scale of available data, however, can be intimidating: during the Great East Japan Earthquake, over 8 million tweets were sent each day from Japan alone. Conventional word vector-based topic-detection techniques for social media that use Latent Semantic Analysis, Latent Dirichlet Allocation, or graph community detection often cannot extract appropriate topics from such a large volume of data with accuracy due to their space and time complexity. To alleviate this problem, we propose an effective topic extraction from millions of tweets based on community detection in bipartite networks. Our method is based on the bipartite community detection technique developed by Okamoto, one of the authors of this paper. The paper demonstrates our method effectiveness on social media analysis and identifies topics from millions of tweets after the Great East Japan Earthquake. To show our method's effectiveness, we compute the coherence measure that can evaluate the semantic accuracy and the running time, and compare the method with LDA that is the major topic model.