New details have emerged on how cybercriminals, and in particular state-sponsored groups, are exploiting AI to make their campaigns more efficient.
In a report released this week, OpenAI says it has shut down more than 40 coordinated networks using ChatGPT for hacking, scams, and propaganda — including state-linked campaigns from Russia, China, and North Korea.
OpenAI says that the threat actors aren’t being given new capabilities by ChatGPT but instead are using the platform to make existing workflows more efficient.
The company says its models refused direct requests to produce malware or exploits.
“In this report, we describe our disruption of cyber activity that reflected patterns of well-known conventional adversary tradecraft,” OpenAI said.
“We actively invest in detecting and disrupting malicious cyber threat activity across our platform. Relevant information derived from our disruptions is used by our safety teams to improve our threat modeling and detections, model policies, and model behavior.”
The report details several examples of how specific groups are attempting to use ChatGPT for nefarious purposes.
In one instance, a Russian-language developer attempted to use ChatGPT to try and develop and refine malware, including a remote‑access trojan and credential stealers.
In another campaign, Chinese language threat actors used ChatGPT accounts to develop phishing and malware tools, which Proofpoint had previously reported were used to target Taiwan’s semi-conductor industry.
OpenAI said it also banned accounts apparently linked to Chinese government entities that used ChatGPT for surveillance-related proposals and profiling tasks.
Scam operations were another major issue, with scammers attempting to use OpenAI models to “deceive and defraud” people. This ranges from lone actors to large scale criminal gangs.
Scam campaigns normally begin with cold outreach, either on email, text message chains, or using paid social media ads to generate excitement, before “the sting,” where cybercriminals attempt to solicit or steal money.
In all cases, OpenAI banned the accounts associated with malicious activity. Since 2024, the company has banned and reported over 40 groups that violated its usage policies.
Why This Matters
One of the most interesting takeaways is how difficult it can be to spot genuine malicious activity with AI tools.
“A meaningful share of threat activity continues to fall into a gray zone — prompts and
generations that could [be harmful], depending on their context,” OpenAI said.
This could include activity that could be innocent, or potentially malicious, such as modifying code or creating a website.
This represents a major challenge to the AI giants, like OpenAI, who must then take a decision on what activity to block, without creating workflow issues for legitimate users.
“To detect and disrupt threats effectively without disrupting the work of everyday users, we employ a nuanced and informed approach that focuses on patterns of threat actor behavior rather than isolated model interactions,” OpenAI explained.
The report also highlights cross-model workflows, where threat actors combine OpenAI tools with others like Anthropic’s Claude or DeepSeek, underscoring the need for industry-wide transparency and coordination.
The big picture
Malicious use of AI tools has been a major discussion point in the cybersecurity industry for several months.
In August, Anthropic released a report outlining how threat actors were misusing their AI tools for similar purports, including North Korean remote workers, ransomware-as-a-service gangs, and one hacker who breached 17 companies.
Just this week, TheHackerNews has reported on a new attack targeting Perplexity’s agentic AI browser Comet, which embeds malicious prompts to try and steal sensitive data from connected services like email and calendar.
The report paints a complex picture: malicious use of AI is real and growing, but it has not led to new innovations in cybercrime.
Threat actors use large-language models to streamline social engineering and development tasks, not to invent new exploits.
For defenders, OpenAI’s findings underline that the AI-powered threat landscape is evolutionary, not revolutionary — yet.
But organizations must be aware of how cybercriminals are using AI tools and have their own strategies in place for using AI to shore up defensive strategies.
“As the threatscape evolves, we expect to see further adversarial adaptations and Innovations,” OpenAI said.
Read more:
