3D semantic enhanced diffusion policy for intelligent shipborne robot

Objectives To address the tension between limited onboard computation/deployment conditions and the need for high-level perception and decision-making in shipborne robots, we aim to enhance real-time environmental understanding and autonomous decision-making under resource constraints. MethodsWe propose SGDP, a lightweight 3D semantic-augmented framework centered on a diffusion policy. The framework first performs semantic modeling of manipulated objects via 3D Gaussian Splatting, then maintains semantic consistency in dynamic scenes through a semantic flow updated by real-time pose estimation. We further design a multimodal diffusion policy that fuses semantic cues, geometry, and joint states, improving semantic perception while remaining suitable for lightweight shipborne deployment.ResultsOn a shipborne robotic testbed and across three complex tasks—placing a knife, grasping a marker pen, and pouring water from a bottle—SGDP achieved an average success rate of 81.67% on tasks with known objects and 78.33% on tasks with unseen objects, significantly outperforming DP3 and GenDP.Conclusions The results demonstrate an efficient, integrated perception-decision solution that coordinates environmental understanding and autonomous decision-making under limited resources, providing a practical pathway to reconcile lightweight deployment with high-level autonomy for single-unit unmanned systems such as shipborne robots.