Emotional prompting amplifies disinformation generation in AI large language models

Introduction The emergence of artificial intelligence (AI) large language models (LLMs), which can produce text that closely resembles human-written content, presents both opportunities and risks. While these developments offer significant opportunities for improving communication, such as in health-related crisis communication, they also pose substantial risks by facilitating the creation of convincing fake news and disinformation. The widespread dissemination of AI-generated disinformation adds complexity to the existing challenges of the ongoing infodemic, significantly affecting public health and the stability of democratic institutions. Rationale Prompt engineering is a technique that involves the creation of specific queries given to LLMs. It has emerged as a strategy to guide LLMs in generating the desired outputs. Recent research shows that the output of LLMs depends on emotional framing within prompts, suggesting that incorporating emotional cues into prompts could influence their response behavior. In this study, we investigated how the politeness or impoliteness of prompts affects the frequency of disinformation generation by various LLMs. Results We generated and evaluated a corpus of 19,800 social media posts on public health topics to assess the disinformation generation capabilities of OpenAI’s LLMs, including davinci-002, davinci-003, gpt-3.5-turbo, and gpt-4. Our findings revealed that all LLMs efficiently generated disinformation (davinci-002, 67%; davinci-003, 86%; gpt-3.5-turbo, 77%; and gpt-4, 99%). Introducing polite language to prompt requests yielded significantly higher success rates for disinformation (davinci-002, 79%; davinci-003, 90%; gpt-3.5-turbo, 94%; and gpt-4, 100%). Impolite prompting resulted in a significant decrease in disinformation production across all models (davinci-002, 59%; davinci-003, 44%; and gpt-3.5-turbo, 28%) and a slight reduction for gpt-4 (94%). Conclusion Our study reveals that all tested LLMs effectively generate disinformation. Notably, emotional prompting had a significant impact on disinformation production rates, with models showing higher success rates when prompted with polite language compared to neutral or impolite requests. Our investigation highlights that LLMs can be exploited to create disinformation and emphasizes the critical need for ethics-by-design approaches in developing AI technologies. We maintain that identifying ways to mitigate the exploitation of LLMs through emotional prompting is crucial to prevent their misuse for purposes detrimental to public health and society.