OpenAI has raised alarms over the misuse of its popular AI tool, ChatGPT, in attempts to influence US elections. In a report released on Wednesday, the company revealed that cybercriminals have been exploiting the capabilities of ChatGPT to generate fake content aimed at manipulating voters. This development underscores growing concerns about AI’s role in the spread of misinformation and the impact it could have on democratic processes.
Cybercriminals have discovered how to use AI models like ChatGPT to produce highly persuasive, coherent text on a large scale, which can then be deployed in disinformation campaigns. OpenAI’s report highlights cases where its technology was used to create fake news articles, fraudulent campaign materials, and deceptive social media posts. These AI-generated messages often mimic legitimate news sources, making it difficult for the average citizen to distinguish between real and fabricated information.
One of the most troubling aspects of this misuse is the ability to target specific demographics. By using data mining techniques, bad actors can analyse voter preferences and tailor messages that resonate with these groups. This level of personalisation enhances the effectiveness of disinformation campaigns, allowing them to exploit political divisions and amplify societal discord.
OpenAI disclosed that it had thwarted more than 20 attempts to misuse ChatGPT for influence operations in 2024. In August, the company blocked accounts that were generating election-related articles, and in July, accounts from Rwanda were banned for producing social media content aimed at influencing that country’s elections. Despite these efforts, the speed at which AI can generate content remains a challenge. Misinformation can spread rapidly, overwhelming traditional fact-checking mechanisms, which struggle to keep pace with the flood of false information.
The report also emphasised the potential danger of ChatGPT being used in automated social media campaigns. AI-generated content can be deployed at scale to influence public perception and voter sentiment in real-time, especially during critical moments in election cycles. So far, however, OpenAI noted that attempts to influence elections through ChatGPT-generated content have not gained significant traction. None of the efforts have gone viral or reached a substantial audience, but the company views this as a major threat to electoral integrity.
In addition to OpenAI’s concerns, the US Department of Homeland Security has raised alarms about foreign interference in elections. According to the department, countries such as Russia, Iran, and China are reportedly using AI-driven disinformation tactics to sway public opinion and create division, posing a significant threat to the upcoming November elections in the US.
