Curvature-Enhanced Leaky AdamW: Adaptive Gradient Optimization Based on Dynamic Loss Surface Correction for Deep Neural Networks

Adaptive gradient optimization algorithms represented by AdamW have been widely used in deep learning tasks such as image classification. However, traditional AdamW suffers from fixed update rules, slow convergence speed, severe gradient oscillation in late training stages, insufficient parameter update perception, and poor stability in flat or sharp loss regions, resulting in limited final test accuracy of the model. To solve these problems, this paper proposes a novel adaptive optimization algorithm named Curvature-Enhanced Leaky AdamW (CEL-AdamW). Based on the gradient momentum difference and real-time parameter update variation, the algorithm constructs a dynamic training curvature estimation mechanism, realizing lightweight and fine-grained perception of loss surface geometric features. It designs pixel-level abnormal judgment rules for parameter stagnation and curvature overflow, and replaces invalid curvature features with momentum difference to ensure update effectiveness. A targeted curvature enhancement strategy is innovatively introduced to quantify the geometric characteristics of real-time loss surface, which adaptively corrects the step size scaling factor. Combined with Leaky nonlinear mapping and three-stage training scheduling strategy (dormancy, warm-up, full activation), the proposed algorithm significantly accelerates model convergence speed, suppresses training oscillation, and improves convergence stability and final test accuracy. Comprehensive experiments on CIFAR-10 with ResNet18 show that compared with baseline optimizers including Adam, AdamW, and AdaBelief, the proposed CEL-AdamW achieves faster loss convergence, lower training loss and higher classification accuracy. Notably, CEL-AdamW reaches the optimal test accuracy of 89.34% at approximately the 50th epoch, outperforming Adam (88.17%), AdamW (88.57%), and AdaBelief (88.86%) in both convergence speed and final precision, with the peak accuracy achieved 8 epochs earlier than Adam and AdaBelief. Strict mathematical convergence proof verifies the rationality and superiority of the proposed algorithm. The proposed dynamic curvature enhancement optimization mechanism provides an effective solution for stable, fast and high-precision training of deep neural networks.