Reinforcement Learning for Cognitive Phased Array Radar Surveillance

Abstract

The proliferation of phased array radar (PAR) has significantly increased the flexibility of radar systems, making it possible to use a single radar to perform a variety of operational modes such as surveillance and tracking that each traditionally required a dedicated system.

To fully take advantage of these capabilities, algorithms must be developed to efficiently distribute the radar's finite time, energy, and processing budget between competing tasks. Although many resource management methods exist for tracking applications, it is common to use a fixed strategy (e.g., a raster scan) for the surveillance task. The resulting allocation of resources is often sub-optimal since fixed approaches do not leverage prior knowledge and thus spend a disproportionate amount of time searching regions that are unlikely to contain new targets.

This thesis presents a novel approach to more effectively utilize the radar timeline in surveillance and track initiation tasks. A variant of particle swarm optimization (PSO) is derived to estimate the density of untracked targets in the search volume, which is then used to inform the parameter selection process for each radar dwell. The resulting method, known as Surveillance PSO (SPSO), is computationally efficient and suitable for real-time implementation on a general-purpose CPU or GPU. SPSO is also highly general, making few assumptions about the properties of the target or the underlying radar system. Finally, the output of the algorithm is a constant-length tensor that can be incorporated into systems that utilize deep learning and reinforcement learning. Two cognitive agents are developed to demonstrate the utility of the SPSO algorithm. The first is a deterministic agent that directly uses the output of the SPSO algorithm to make decisions on where to steer the radar beam at each dwell. The second is reinforcement learning (RL) agent that uses a slight modification of SPSO to simultaneously steer and spoil the transmitted beam based on the current environment. The performance of each agent is evaluated in several simulated surveillance environments, where both are shown to outperform the standard raster scan approach.