Incentive Issues in Developing Factual LLMs

The advent of large language models (LLMs) has drastically lowered the cost of producing misinformation, leading to an explosion in its volume and making fact-fiction distinctions increasingly difficult. Paradoxically, the same LLMs can also be powerful tools for fact-checking and factual content generation, provided they are trained on high-quality factual data. Such factual data are primarily produced by traditional news organizations and content providers, for whom fact collection is a costly process, and they may not have enough incentive to share these facts with LLMs. We study the incentives for fact sharing between LLMs and content providers using a game-theoretic model. We show that content providers with low monetizability may naturally partner with LLMs without any compensation. Interestingly, such partnerships reduce the LLM's factual quality. This is because partnership reduces the competition between the LLM and the content provider, which leads to less effort from the content provider in fact collection. The factual quality of LLMs improves only when they partner with providers with highly monetizable content. However, such content providers require financial compensation for partnering with LLMs. We analyze several compensation schemes and find, counterintuitively, that compensation can further reduce the LLM's factual quality compared to no compensation, when the content provider's monetizability is low or moderate. This is because the compensation further reduces competition between the LLM and the content provider, thereby reducing the content provider's incentive to exert effort in fact collection. Our results highlight nuances in the partnership between LLMs and content providers and show that such partnerships do not always yield a more factual LLM.