Complex dynamics of a predator-prey model with hunting cooperation-induced fear and diffusion

Abstract Hunting cooperation is typical behavior among some large predators that increases the predator's ability to capture prey, and hunting cooperation can induce fear, which reduces the birth rate of prey. Based on this fact, in this paper we propose two predator-prey models with hunting cooperation-induced fear. In the first model, we consider the case without the spatial diffusion of populations: we study the bifurcation of the equilibrium points and their stability. Our study finds that (1) the model admits at most two positive equilibria, and when they exist, the model exhibits bistability structures: either between predator-free equilibrium and periodic solution, or between predator-free equilibrium and unique positive equilibrium. (2) When the model has a unique positive equilibrium, it undergoes the Hopf bifurcation. In addition, we analyze the stability and direction of Hopf bifurcated periodic solution, which shows that low-level fear and high-intensity hunting cooperation can cause Hopf bifurcation. (3) If the two positive equilibria coincide, saddle-node bifurcation and Bogdanov-Takens bifurcation of codimension two may occur. Moreover, if the predator population does not hunt cooperatively, the saddle-node bifurcation and Bogdanov-Takens bifurcation disappear. Our research findings generalize the existing results. For the model with spatial diffusion, we first pay attention to whether the model has Turing instability, which favors the formation of biodiversity. We find that when the ratio of the diffusion rate of prey to the diffusion rate of predator exceeds a certain critical value, Turing instability occurs. In addition, we present the existence of Hopf bifurcation and discuss the stability, direction, and period of spatially homogeneous and inhomogeneous periodic solutions. That shows that diffusion accelerates the occurrence of Hopf bifurcation, which destabilizes the population in advance and turns the number of the population into a periodic oscillating. At last, the theoretical analysis is illustrated by numerical simulations. The results show that populations can exhibit predator extinction, stable coexistence, or oscillatory coexistence under the influence of hunting cooperation, fear, or diffusion.Mathematics Subject Classification 92B05 · 92D25 · 35K57 · 37G15