Multi-AUV cooperative expulsion method based on multi-agent reinforcement learning under probabilistic sensing

Objectives A multi-AUV cooperative expulsion decision-making framework is proposed for cooperative area protection under probabilistic sensing and highly maneuvering targets. Methods First, a probabilistic forward looking sonar detection model is established by considering range dependent attenuation and azimuthal field-of-view constraints. An intent-driven motion mode transition mechanism is then introduced, and an intent-aware interacting multiple model iterated extended Kalman filter (I3EKF) is developed to achieve accurate and continuous target state estimation. Second, the future feasible paths and reachable points of the target are predicted based on the motion characteristics of underactuated targets. By analyzing their spatial interactions with the Apollonius circles, a quantitative cooperative repulsion utility evaluation model is developed. Finally, based on this model, a cooperative expulsion reward mechanism is designed, and MATD3 is employed to optimize multi-AUV cooperative positioning and trajectory guidance strategies, enabling effective expulsion of highly maneuvering targets despite a speed disadvantage. Results Simulation results show that, compared with IMM-IEKF, I3EKF reduces the position and velocity estimation errors by 39.2% and 42.7%, respectively, while improving position estimation smoothness by 67.5%. Compared with the formation control method, the proposed decision-making method improves the repulsion success rate by 30%, while reducing the mean episode step and mean movement distance by 26.4% and 30.1%, respectively. Conclusions The proposed framework enables stable target state estimation and efficient cooperative expulsion with a high success rate under probabilistic sensing and highly maneuvering targets, providing a new solution for underwater cooperative area protection.