From sheaves on P 2 to a generalization of the Rademacher expansion

Moduli spaces of stable coherent sheaves on a surface are of much interest for both mathematics and physics. Yoshioka computed generating functions of Poincar\'e polynomials of such moduli spaces if the surface is ${\Bbb P}^2$ and the rank of the sheaves is 2. Motivated by physical arguments, this paper investigates the modular properties of these generating functions. It is shown that these functions can be written in terms of the Lerch sum and theta function. Based on this, we prove a conjecture by Vafa and Witten, which expresses the generating functions of Euler numbers as a mixed mock modular form. Moreover, we derive an exact formula of Rademacher-type for the Fourier coefficients of this function. This formula requires a generalization of the classical Circle Method. This is the first example of an exact formula for the Fourier coefficients of mixed mock modular forms, which is of independent mathematical interest.