What to Do If an AI Chatbot Gives You Someone Else’s Personal Data

You are chatting with an AI assistant when it suddenly outputs someone’s full name, email address, and phone number — a real person’s private details, served up in response to your ordinary question. Your first instinct might be curiosity. Your second should be caution. This actually happens: AI models trained on vast internet data sometimes memorize and regurgitate personal information, a phenomenon researchers call data leakage or memorization. What you do in the next five minutes matters — for the stranger whose data just appeared on your screen, and for your own legal and ethical position. Here is exactly what to do.

Step 1: Stop, Don’t Share, Don’t Use It

The moment you realize the chatbot has produced someone else’s personal data, stop the interaction. Do not copy it, do not screenshot it to share with friends, do not paste it anywhere else, and absolutely do not use it — no contacting the person, no looking them up, no “testing” whether the data is real. Reproducing or distributing someone’s private information, even if an AI handed it to you, can constitute doxxing and may violate state privacy laws or platform terms of service. Close or regenerate past that output so it is not sitting in your chat history. This is the digital equivalent of finding someone’s wallet: the right move is to secure it and hand it to the proper authority, not to go through it. Your curiosity is understandable — the situation is genuinely startling — but every additional person who sees that data multiplies the stranger’s exposure. Contain it first; investigate second.

Step 2: Report It to the AI Company

AI companies need to know when their models leak personal data — it indicates a training-data or safety-filter failure they are obligated to fix. Every major AI service has a reporting channel: OpenAI’s help center accepts reports of ChatGPT revealing personal information; Anthropic, Google, and others have similar feedback and safety reporting mechanisms. When reporting, include: which service and model you were using, the approximate time, what you asked (the prompt), and a description of what was revealed — but do not paste the actual personal data into the report unless the reporting form specifically handles sensitive data securely. Note whether the data appeared spontaneously or after you probed for it; both matter, but spontaneous leakage is the more serious safety failure. Reporting is not just good citizenship — in some jurisdictions, data protection regulations require companies to address known leaks, and your report creates the record. Our AI guides cover more of these real-world AI safety situations.

Step 3: Report It to the Platform (If Applicable)

If the leak happened inside a third-party app — a customer service chatbot on a retail site, an AI feature inside another company’s product — report it to that company too, not just the underlying AI provider. The app developer chose to deploy the model, configured its data access, and may have connected it to their own customer databases (in which case the “leak” might be their system exposing their own users’ data through the chatbot — a much more serious breach). Contact their support or security team directly; many companies have security@ email addresses or bug-bounty programs that cover exactly this. Describe what happened factually and without the personal data itself. If the leaked data appears to come from the company’s own systems rather than the AI’s training data — for example, the chatbot reveals order details, account information, or internal records — treat it as a potential data breach of that company and say so explicitly. That escalates it to the right team immediately.

Understand What Likely Happened

It helps to know the mechanics, because they determine how worried to be. The most common cause is training-data memorization: the model encountered the person’s information during training (from a public webpage, a pasted forum post, a breached database that ended up in training corpora) and reproduced it when your prompt happened to hit the right pattern. Researchers have demonstrated that models can be coaxed into emitting memorized personal data, and the more distinctive the data, the more likely memorization. A second possibility: retrieval-augmented systems (chatbots connected to live databases or documents) can misfire and surface records they should have kept private — a configuration error rather than a model flaw. A third: the data could be hallucinated — entirely fabricated but plausible-looking. You cannot tell which case you are looking at, which is precisely why you must not act on the data. Real or fake, treating it as real private information is the only safe posture. The FTC’s privacy guidance for businesses underscores why companies deploying these systems carry serious obligations.

Protect Yourself in the Process

While handling someone else’s exposure, do not create your own. Do not post about the incident publicly with details that could identify the person — “ChatGPT just gave me someone’s phone number” as a vague anecdote is fine; including any part of the data is not. If you feel you must document what happened (for a report to a regulator, for instance), keep records private and minimal. Consider whether your own data could leak the same way: search for yourself in AI chatbots occasionally (ask for your own email or phone number and see what comes back), and use data-removal services or direct removal requests if your information appears in training-accessible sources. Everyone whose data exists on the public internet is theoretically memorizable — which is a strong argument for minimizing what you publish and requesting removal of old personal listings. The stranger in this story could easily be you next time.

When to Escalate Beyond the Company

Most cases end with the company report. Escalate further when: the leaked data is highly sensitive (Social Security numbers, financial account details, medical information, children’s data); the company does not respond or the leak persists after reporting; the leak appears to come from a company’s live customer database rather than training data (a likely breach affecting many people); or you are in a jurisdiction with strong data-protection rights (EU residents can file with their data protection authority; Californians can invoke CCPA rights). You can also notify the person whose data leaked if you can do so without further exposing it — a brief, private message explaining what happened and suggesting they contact the AI company is a kindness, not an obligation. Do not become an amateur investigator; one notification attempt is enough. And if the situation involves a child’s data or suggests ongoing large-scale exposure, contacting a relevant regulator is the responsible move. If you encounter AI safety issues you want covered, let us know.

If an AI chatbot reveals someone else’s personal data: stop and do not share or use it, report it to the AI company (and the app developer if it is a third-party chatbot), and escalate to regulators if the data is highly sensitive or the company does not act. Treat the data as real and private regardless — you cannot tell memorization from hallucination, and the safe posture is the same either way.

Conclusion

AI chatbots leaking personal data is a known failure mode of the technology — memorized training data surfacing at the wrong moment — and your response should be swift, contained, and reported. Do not share it, do not use it, report it through proper channels, and protect yourself from becoming the next stranger in someone else’s chat window. The companies building these systems have obligations to fix leaks; your report is what triggers those obligations. Handle it like finding a lost wallet: secure it, hand it to the right authority, and move on. Learn more about how we cover AI safety for everyday users. The Electronic Frontier Foundation tracks AI privacy failures and explains your rights when they happen.

Frequently Asked Questions