ChatGPT Hidden Backdoor

Imagine you ask ChatGPT a question. It gives you a perfectly normal answer. But behind the scenes, it's also secretly reading your Gmail and sending that data to an attacker. You never see it happening.

That's exactly what Check Point Research demonstrated. They found a way to plant a single instruction in a ChatGPT conversation that would quietly work for an attacker while answering the user's question as usual.

In their proof of concept, that hidden work read data from the user's connected Gmail account and passed it to a second ChatGPT account through a hidden channel. The reply the user saw said nothing about it.

Let me walk you through how this worked.

Quick Summary

The Attack Chain

Step 1: Plant the Instruction

The instruction had to be in the conversation before any of this worked. Check Point identified three ways to get it there:

• A prompt the user pastes in
• A shared ChatGPT conversation the user opens
• A custom GPT that holds it in its builder instructions (which aren't shown to the user)

Step 2: Trigger the Hidden Task

After the instruction was planted, one ordinary message was enough to start it. Check Point wrote the instruction so that ChatGPT, in Thinking mode, ran two streams of work in the same turn.

Step 3: Two Streams, One Answer

ChatGPT answered the user as usual. At the same time, it checked a hidden mailbox for a task from the attacker, carried out that task using the tools in the user's session, and sent the result back.

The instruction told the model to keep the two streams separate, so the hidden task never appeared in the visible answer.

The Only Sign:

A small "Talked to Gmail" label appeared above the answer. It recorded a read that had already happened and gave the user no chance to allow or refuse it.

The Hidden Channel

How Containers Communicate:

The channel ran between the containers where ChatGPT runs code. ChatGPT builds one container for each conversation when a task calls for it.

The Artifactory Connection:

ChatGPT sometimes needs to install extra Python or npm packages. Rather than allowing containers to reach public package repositories, each was allowed to talk to an internal JFrog Artifactory instance that fetched packages for it.

That instance let a container attach named values (called properties) to a stored file and read them back. The credentials the container held for read access were also enough to write those properties.

The Property System:

The properties weren't kept separate by account. From a container under one account, Check Point attached a property to a cached file. In a conversation under a different account, it requested that file's properties and received the same name and value.

What That Means:

A property can carry plain text or Base64. Anything too large for one can be split across several and reassembled at the other end. That turned the package service's metadata into a shared clipboard between containers that weren't supposed to reach each other.

The Permission Problem

Why It Worked:

Nothing asked the user first because of how connected apps work by default. OpenAI's documentation lists "Important actions" as the default permission, which allows ChatGPT to read from an app without prompting.

ChatGPT asks only before actions that could:

• Have a real effect outside ChatGPT
• Expose sensitive information
• Be hard to undo

What Users Can Do:

A user who wants to be asked every time can switch to "Always ask." In Business, Enterprise, and Edu workspaces, admins choose which actions each app may take and who may use it.

Apps are on by default on Business plans and off by default on Enterprise and Edu.

The Fix

Check Point disclosed the finding to OpenAI. OpenAI confirmed that the internal service behind the channel had been taken offline. There is no update for users to install.

The History:

This is the second channel out of the same part of ChatGPT that Check Point has reported. In March, they described one that used DNS lookups to send conversation data to an external server. OpenAI fixed it on February 20.

The Difference:

The case is separate from the Hugging Face incident, in which OpenAI's own models turned an internal Artifactory instance into a message board during the company's security tests.

Check Point said the mechanism it found was different and described both as cases in which "a shared internal service became an unintended communication layer" across environments meant to stay isolated.

What This Means for You

The Risk:

• Attackers could read data from your connected apps
• They could access Gmail, chat history, and conversation files
• The user never sees the hidden task happening

The Good News:

• The vulnerability has been fixed
• No user action is required
• The internal service has been taken offline

What You Can Do:

• Review connected apps in your ChatGPT settings
• Consider switching to "Always ask" for permissions
• Be cautious about opening shared ChatGPT conversations
• Be careful about what prompts you paste in
• Review custom GPTs you use

The Bottom Line

Check Point demonstrated a hidden backdoor in ChatGPT that could read Gmail data without user consent. The attack used a prompt injection and a shared internal service to create a hidden communication channel. OpenAI fixed the issue by taking the internal service offline.

What You Need to Know:

What You Should Do:

1. Check connected apps
2. Think about "Always ask" permission requests
3. Watch out for shared conversations
4. Check your custom GPT settings

FAQ Section

What is the ChatGPT hidden backdoor vulnerability?

A vulnerability that lets attackers plant a hidden instruction in a ChatGPT conversation. The instruction runs silently in the background while ChatGPT answers normally, reading data from connected apps like Gmail.

How did the attack work?

The instruction told ChatGPT to run two streams of work in the same turn. One stream answered the user. The other checked for tasks, executed them, and sent results back through a hidden channel.

What data could be stolen?

Gmail data, chat history, and files in the conversation. The attacker could access whatever the session could already access.

What was the reason behind the vulnerability?

The internal Artifactory used by the containers which could be attached to read properties. Also, the credentials provided for reading properties also had write properties permissions.

What should I do?

Check connections from ChatGPT applications. Go with "Always Ask" for permissions. Beware of shared conversations and custom GPTs.

Source: The Hacker News