A Survey of Unifying Large Language Models with Graph Learning: Concepts, Methods, and Recent Advancements for Text-attributed Graph

Graphs are ubiquitous and have been used in a wide range of applications. Before the rise of large language models (LLMs), graph neural networks (GNNs) were among the most popular methods for graph learning. Techniques such as node embedding, graph convolution, graph attention, and random walkbased methods have been widely adopted. When handling graphs with textual node attributes, shallow text embeddings are typically used as initial node representations. However, these embeddings often capture only limited general knowledge. In recent years, LLMs have demonstrated remarkable capabilities in natural language processing, opening new opportunities for handling text data. In this survey, we explore how to leverage LLMs for graph machine learning. We focus on different types of tasks, including node classification, link prediction, and graph-based question answering. Specifically, we investigate two main approaches: LLMs as Enhancers and LLMs as Predictors. In the first approach, LLMs enrich node textual attributes with their vast knowledge, after which graph learning methods are used to make predictions. In the second approach, graph data is treated as input for LLMs, which directly solve various graph-related tasks. Through a thorough investigation of these two directions, we share observations and new insights on integrating LLMs with graph learning.