Generalized Statistical Pooling: Learning to Model Local Distributions for Adaptive Downsampling

Pooling operations play an essential role in convolutional neural networks by providing spatial subsampling, an enlarged receptive field, and local invariance. However, existing CNNs are based on heuristic pooling operations such as max pooling or average pooling, which use a single statistic to describe the local neighborhood. However, this is a limited representation of the underlying activation distribution. In this work, we introduce a new framework for adaptive multi-statistical pooling, in which the pooling operation is formulated as a local statistical modeling problem. Unlike existing pooling operations, the proposed method extracts several statistical descriptors from the local neighborhood, such as the mean, standard deviation, median, median absolute deviation, and maximum. These statistical measures describe different aspects of the activation distribution, such as central tendency, dispersion, robustness, etc. A selection mechanism is introduced, which learns the combination of the extracted statistical measures. This pooling operation generalizes conventional pooling operations, where maximum pooling and average pooling are special cases of the proposed method. Furthermore, the pooling module is integrated into a hierarchical encoder-decoder model with multi-scale feature extraction and coordinated attention mechanisms to improve the representation of spatial features. By explicitly modeling the statistical structure of feature maps, the proposed approach improves the expressiveness and robustness of the spatial subsampling process, while remaining fully differentiable and trainable from end to end. Evaluations of the proposed approach through experiments confirmed the effectiveness and flexibility of the proposed statistical pooling approach as a more informative alternative to conventional pooling approaches.