Abstract
In this paper, we study optimal precoding matrix for spectrum sharing multiple-input multiple-output (MIMO) cognitive multiple-relay systems. Our aim is to maximize the system capacity subject to the total relay transmit power constraint and the interference power constraint at the primary user. For general case, all the nodes have multiple antennas. The precoder design problem is a non-convex problem. Currently there is no global optimal solution for it. We propose a suboptimal algorithm in which we first transform the original precoder design problem to a unitary matrix constrained problem. And then, we develop a modified Riemannian steepest descent algorithm (MRSDA) to solve it. For the special case that the secondary user has a single antenna and other nodes have multi-antennas, we propose an optimal solution based on rank-one decomposition theorem to transform the original problem into a semidefinite programming. Simulation results demonstrate the effectiveness of the proposed precoding matrix design algorithms.