Abstract
This paper aims at maximizing the spectral efficiency of the recently proposed hybrid FSK/QAM scheme by the authors. The optimization problem incorporates the notion of discrete rate and respects prescribed constraints on average power, probability of error, and peak power. Further, we highlight the impact on performance if we use a discretized version of optimum continuous rate versus optimum discrete rate. Numerical examples reflect that the former approach is inferior to the latter when sole QAM is considered whereas both approaches are almost identical when dealing with hybrid FSK/QAM.