DRL-empowered secure and green beamforming optimization for maritime heterogeneous ISAC

Objective The integrated sensing and communications (ISAC) system for offshore communication faces challenges such as frequent node mobility, strong time-varying interference in the channel, and cross-network eavesdropping threats. Traditional optimization methods are computationally complex and difficult to respond in real time. Method To address this issue, an intelligent beamforming optimization framework based on deep reinforcement learning (DRL) is proposed. The problem of maximizing security and energy efficiency is modeled as a Markov decision process, and a composite reward function is designed to guide the strategy optimization. Rate-splitting multiple access (RSMA) is introduced to finely manage cross-network interference, and a novel approach is adopted to utilize the sensing signal as an inherent green interference against eavesdroppers, thereby enhancing physical layer security without additional power consumption. The proximal policy optimization (PPO) algorithm is employed, combined with a hybrid training mechanism of supervised pre-training and online fine-tuning, to achieve rapid convergence and dynamic adaptability. Result Based on typical offshore parameters with a carrier frequency of 18 GHz, an emission power of 35 dBm, and 3 users, the simulation results show that the security energy efficiency (SEE) of the proposed scheme is more than 18% higher than that of the traditional RSMA alternating optimization scheme, and the convergence speed is more than 30% faster. Moreover, it demonstrates stronger robustness in scenarios of channel estimation errors and topology mutations. Conclusion The proposed DRL intelligent beamforming method can achieve a synergistic improvement in security energy efficiency, real-time response, and robustness in complex marine environments, providing a new idea for intelligent resource management in ISAC.