OpenAI disrupts accounts using ChatGPT to build social‑media surveillance tools
OpenAI says it disrupted a now‑banned account originating in China that used ChatGPT to design promotional materials and project plans for a social‑media listening tool reportedly intended for a government client. The tool was described as a “probe” that could crawl platforms such as X, Facebook, Instagram, Reddit, TikTok and YouTube for political, ethnic or religious content as defined by the operator. OpenAI said it could not independently verify whether a government entity actually used the tool.
High‑risk Uyghur tracking proposal
OpenAI also banned an account that used ChatGPT to develop a proposal for a “High‑Risk Uyghur‑Related Inflow Warning Model,” which would have aided tracking movements of “Uyghur‑related” individuals. The Uyghur population in China has long faced accusations of human rights abuses, and the proposal raised serious ethical and safety concerns.
Broader misuse and threat reporting
Since February 2024 OpenAI has published threat reports highlighting state‑affiliated actors using large language models for malicious purposes, including debugging malware, crafting phishing campaigns and fueling influence operations. The company has flagged similar misuse in Russia, Korea and elsewhere, and identified networks in Cambodia, Myanmar and Nigeria using ChatGPT to help run scams.
- OpenAI estimates ChatGPT is used to detect scams roughly three times more often than to create them.
- Earlier this summer, the company disrupted operations in Iran, Russia and China that used ChatGPT to generate posts and engagement for influence campaigns.
Sources
This summary is based on reporting from Engadget and OpenAI’s public threat reporting. For more detail, see the full article on Engadget and OpenAI’s blog:
Engadget: OpenAI disrupts more accounts using ChatGPT to create social‑media surveillance tools
If you want this post adjusted (tone, length, or to add/remove links), tell me which changes you want.