Cross Layer Routing in Cognitive Radio Network Using Deep Reinforcement Learning

Abstract

Development of 5G technology and Internet of Things (IoT) devices has resulted in higher bandwidth requirements leading to increased scarcity of wireless spectrum. Cognitive Radio Networks (CRNs) provide an efficient solution to this problem. In CRNs, multiple secondary users share the spectrum band that is allocated to a primary network. This spectrum sharing of the primary spectrum band is achieved in this work by using an underlay scheme. In this scheme, the Signal to Interference plus Noise Ratio (SINR) caused to the primary due to communication between secondary users is kept below a threshold level. In this work, the CRNs perform cross-layer optimization by learning the parameters from the physical and the network layer so as to improve the end-to-end quality of experience for video traffic. The developed system meets the design goal by using a Deep Q-Network (DQN) to choose the next hop for transmitting based on the delay seen at each router, while maintaining SINR below the threshold set by primary channel. A fully connected feed-forward Multilayer Perceptron (MLP) is used by secondary users to approximate the action value function. The action value comprises of SINR to the primary user (at the physical layer) and next hop to the routers for each packet (at the network layer). The reward to this neural network is Mean Opinion Score (MOS) for video traffic which depends on the packet loss rate and the bitrate used for transmission. As compared to the implementation of DQN learning at the physical layer only, this system provides 30\% increase in the video quality for routers with small queue lengths and also achieves a balanced load on a network with routers with unequal service rates