Foundation Model‐Driven Grasping of Unknown Objects via Center of Gravity Estimation

ABSTRACT This study presents a grasping method for objects with uneven mass distribution by leveraging diffusion models to localize the center of gravity (CoG) on unknown objects. In robotic grasping, CoG deviation often leads to postural instability, where existing keypoint‐based or affordance‐driven methods exhibit limitations. We constructed a dataset of 790 images featuring unevenly distributed objects with keypoint annotations for CoG localization. A vision‐driven framework based on foundation models was developed to achieve CoG‐aware grasping. Experimental evaluations across real‐world scenarios demonstrate that our method achieves a 49% higher success rate compared to conventional keypoint‐based approaches and an 11% improvement over state‐of‐the‐art affordance‐driven methods. The system exhibits strong generalization with a 76% CoG localization accuracy on unseen objects. This provides an innovative solution for precise and stable grasping tasks, with its scientific validity further validated in complex and dynamic scenarios.