Abstract
We consider a time-slotted primary system where both the primary channel and primary activity are modeled as two independent two-state Markov chains. The primary transmitter can be idle or busy, whereas the channel can be in erasure or not. Moreover, the sensing channel between the primary transmitter and secondary transmitter is modeled as a two-state Markov chain to represent two levels of sensing reliability. At the beginning of each time slot, the secondary transmitter may remain idle, transmit directly, or probe the channel and access the channel only if it is sensed to be free. At the end of the slot the secondary transmitter overhears the ARQ feedback from the primary receiver, which is used to update its belief regarding primary activity, primary channel state, and sensing channel state. We devise optimal transmission strategies for the cognitive radio so as to maximize secondary throughput with the primary throughput guaranteed to be above a certain specified value. The optimization accounts for the instantaneous reward and also the expected future reward given secondary action.