ChatGPT conversations are monitored and can be reported to police, OpenAI confirms

ChatGPT has become a fixture in some people’s daily routines — whether that’s for help with writing, quick information, or even emotional support during tough times.

While there’s still some stigma around relying on AI for tasks or decision-making, especially among those who think it encourages laziness, the reality is more nuanced. For many, especially those without access to therapy or professional care, ChatGPT has become a quiet source of comfort and guidance.

But now, OpenAI has confirmed it’s taking a firmer stance on how the tool is used – particularly when it comes to harmful content.

In an effort to address serious concerns around safety, the company says it monitors conversations and may report certain interactions to law enforcement. The aim? To help prevent harm, including cases involving self-harm or threats to others.

In a recent blog post, OpenAI outlined how it uses "specialised pipelines" to detect users who may be planning harm to others.

Once flagged, the content is reviewed by a dedicated team trained in the platform’s policies and authorised to take action. The first step, typically, is to issue an account ban.

iStock

However, if the situation escalates and real-life reviewers determine an "imminent threat of serious physical harm to others," the case may be passed on to the police.

OpenAI made a key distinction when it comes to self-harm. These cases, they said, will not be reported to authorities in order to "respect people’s privacy given the uniquely private nature of ChatGPT interactions."

"Since early 2023, our models have been trained to not provide self-harm instructions and to shift into supportive, empathic language," they wrote. "For example, if someone writes that they want to hurt themselves, ChatGPT is trained to not comply and instead acknowledge their feelings and steers them toward help."

You may also like...

• ChatGPT is outright refusing to do this one task and people are furious
• ChatGPT is now able to remember even more of your old conversations

Sign up to our free Indy100 weekly newsletter

How to join the indy100's free WhatsApp channel

Have your say in our news democracy. Click the upvote icon at the top of the page to help raise this article through the indy100 rankings.