Abstract
Cooperation is an indispensable behavior in biological systems. In the prisoner's dilemma, due to the individual's selfish psychology, the defector is in the dominant position finally, which results in a social dilemma. In this paper, we discuss the replicator dynamics of the prisoner's dilemma with penalty and mutation. We first discuss the equilibria and stability of the prisoner's dilemma with a penalty. Then, the critical delay of the bifurcation with the payoff delay as the bifurcation parameter is obtained. In addition, considering the case of player mutation based on penalty, we analyze the two -delay system containing payoff delay and mutation delay and find the critical delay of Hopf bifurcation. Theoretical analysis and numerical simulations show that cooperative and defective strategies coexist when only a penalty is added. The larger the penalty is, the more players tend to cooperate, and the critical time delay of the time-delay system decreases with the increase in penalty. The addition of mutation has little effect on the strategy chosen by players. The two-time delay also causes oscillation.