Abstract
Improper Gaussian signaling has the ability over proper (conventional) Gaussian signaling to improve the achievable rate of systems that suffer from interference. In this paper, we study the impact of using improper Gaussian signaling on the performance limits of the underlay cognitive radio system by analyzing the achievable outage probability of both the primary user (PU) and secondary user (SU). We derive the exact outage probability expression of the SU and construct upper and lower bounds of the PU outage probability which results in formulating an approximate expression of the PU outage probability. This allows us to design the SU signal by adjusting its transmitted power and the circularity coefficient to minimize the SU outage probability while maintaining a certain PU quality-of-service. Finally, we evaluate the derived expressions for both the SU and the PU and the corresponding adaptive algorithms by numerical results.