Rotation invariance and equivariance in 3D deep learning: a survey

AbstractDeep neural networks (DNNs) in 3D scenes show a strong capability of extracting high-level semantic features and significantly promote research in the 3D field. 3D shapes and scenes often exhibit complicated transformation symmetries, where rotation is a challenging and necessary subject. To this end, many rotation invariant and equivariant methods have been proposed. In this survey, we systematically organize and comprehensively overview all methods. First, we rewrite the previous definition of rotation invariance and equivariance by classifying them into weak and strong categories. Second, we provide a unified theoretical framework to analyze these methods, especially weak rotation invariant and equivariant ones that are seldom analyzed theoretically. We then divide existing methods into two main categories, i.e., rotation invariant ones and rotation equivariant ones, which are further subclassified in terms of manipulating input ways and basic equivariant block structures, respectively. In each subcategory, their common essence is highlighted, a couple of representative methods are analyzed, and insightful comments on their pros and cons are given. Furthermore, we deliver a general overview of relevant applications and datasets for two popular tasks of 3D semantic understanding and molecule-related. Finally, we provide several open problems and future research directions based on challenges and difficulties in ongoing research.