In January a federal judge in the Southern District of New York affirmed an order requiring OpenAI to produce a sample of 20 million ChatGPT conversations to the news organizations and authors suing it over how its models were trained. The coverage treated it as a copyright development, and in the narrow sense it is one. The part we keep coming back to is smaller and more practical. A court decided that AI chat logs are records, and it decided that the privacy interests of the millions of people inside those logs were handled well enough by de-identification and a protective order instead of by keeping the conversations out of the case.
That is worth unpacking slowly, because the same reasoning applies to a 12 person company in Atlanta that has never sued anyone.
What the court actually decided
The order came from Magistrate Judge Ona T. Wang in November, and District Judge Sidney Stein affirmed it in full in January. The news plaintiffs originally asked for 120 million logs from the tens of billions OpenAI has preserved. OpenAI countered with 20 million, roughly half a percent, and then changed course and proposed running keyword searches so it would only produce conversations that touched the plaintiffs' specific works. The court rejected that narrowing. Logs that do not reproduce a copyrighted work, the judges reasoned, still bear on the fair use defense, because fair use turns in part on what the model produces across a broad range of requests.
On privacy, the court acknowledged what it called sincere interests on the part of users, then found those interests adequately protected by three things: cutting the sample down, de-identifying it, and keeping it under a protective order. The sentence that should stop any business owner is the distinction the judge drew between this case and a securities case OpenAI had leaned on, where wiretapped phone calls were at issue. ChatGPT users, the court said, voluntarily submitted their communications. The wiretap comparison failed for that reason.
Read that again, because it is the whole story. A conversation typed into a chat window is treated less like a private call and more like a document you handed over. Courts put AI logs in the same bucket as email, Slack messages and server logs.
There was an earlier order too, in May, before any of the production fights. It stopped OpenAI from deleting logs at all, including conversations users had deleted themselves, across the free, Plus, Pro and Team tiers, and it held for months. Enterprise arrangements and zero-retention deals were carved out of that hold. So when a client asks us whether deleting a chat helps, the honest answer is that the delete button changes what your account looks like and not much else. It is a product feature, and a litigation hold sits above it.
Why this reaches a company that is not being sued
Nothing about this case is specific to OpenAI, and nothing about it requires you to be a defendant. It requires you to have a lawsuit somewhere in your future, or a vendor who does, which for a business of any age is close to a certainty.
Now think about what actually goes into the chat windows at your office on a normal Tuesday. Contract language somebody wants a second read on. A customer email pasted in with the question "how would you answer this". A spreadsheet of quotes and margins with a client's name in the first column. A support ticket that includes an account number, a medical detail, or a note about a dispute. Somebody at ten at night pasting a brief for a client website into a consumer assistant to get a first pass on positioning. None of that feels like creating a record. All of it is one.
We are not making a legal argument here and we are not your lawyer. We are describing a technical fact that legal teams are now working around. The records exist, they sit with a provider who keeps them for its own reasons, and discovery reaches them.
What the retention pages actually say
Since we are talking about documents, read the documents. Take one provider as a worked example, because the language is public and specific. On Anthropic's own privacy pages, deleting a conversation removes it from your history immediately and takes it out of back-end storage within 30 days. If you have left model improvement switched on, chats and coding sessions can be held in de-identified form for up to five years in training pipelines. If automated safety systems flag a session as a usage policy violation, inputs and outputs are kept for up to two years and the classification scores for up to seven. Incognito chats stay out of your history and are not used for training at all, and the help pages still say a copy is retained for 30 days for safety. Then the sentence that matters at the bottom of the page: in all cases, the company may retain chats and coding sessions as required by law and to resolve disputes.
That last line is not a scandal. It is an honest description of the world the providers operate in, and it is the same world you operate in once you paste client material into their product. None of this means the privacy settings are pointless. They shorten the window and they keep your work out of training, which is real value. But a setting controls what a provider does by default. It does not control what a judge can order once there is a case. Those are two different questions, and most companies only ever ask the first one.
What we tell clients to do about it
Four moves, in the order we would make them.
Read the contract sentence, not the marketing page. Two questions: what is retained, and for how long, in the tier you are actually on. The OpenAI orders are instructive on this point. The customers who were not swept up were the enterprise and zero-retention ones, sitting in separate data stores with terms that said so. If your company is buying individual consumer seats on a corporate card, you are in the pool with everyone else.
Sort your work into three buckets before anything else. Fine to paste, which is public information, generic drafts and your own marketing copy. Needs review, which is internal process, non-public pricing and anything with a client name attached. Never, which is client identifiers, health and financial details, material covered by an NDA you signed with your own client, and anything touching a live dispute. Write one page with four real examples in each bucket, in the words your staff actually use. "Sensitive" means nothing to a 24 year old designer at 6pm. "No account numbers, no medical notes, no client names with a dollar figure attached" means something.
Move the work that carries the most risk onto models you run yourself. Open-weight models you can download and run on hardware you already own are the practical version of this, and it is a configuration where retention is your decision rather than a vendor's. We have written about why control of the model is a business decision and not a technical preference, and the honest caveat belongs here. Documents you hold are exactly what a court asks you for, so this does not make you invisible. What it changes is who else holds a copy, and whether your prompts are sitting in someone's 20 million record pool waiting for a judge to approve production.
Write down the rules and then actually train on them. Which tools are approved, what goes in each one, who can approve a new tool, what your vendor contracts say about retention and litigation holds, and what to do when somebody is in a hurry on a Friday. We put the shape of this into a governance guide for business leaders because the request keeps arriving in the same form. One warning from experience: a policy nobody has read is a document that will be quoted back at you in a deposition.
The prediction, with one correction
The thought this piece started from was that as more people realize how easy their AI conversations are to reach, anonymous, encrypted AI and offline AI will boom, and that it will be a reckoning for the big labs. We think that is right. We would add one correction, because it decides which products actually win.
Encryption helps with the middle of the trip. It does nothing about the copy the provider keeps at the end of it, and the copy at the end is the one that gets subpoenaed. A chat that is encrypted in transit, held on someone else's server, and produced under a protective order was never private in the sense people mean when they use the word. The property that matters is not how the conversation travels. It is who runs the machine that produced the answer, and whether that machine keeps a log at all.
That reframes the whole thing as a deployment question instead of a settings question. Which parts of your work should run on a model in your own building, on your own account, under your own retention rules. Which parts can safely go to a hosted service, and which hosted service has terms you could defend in front of your own lawyer. When that lawyer starts telling you which prompts not to send, the market moves, and the vendors that sold privacy as a toggle will be the ones explaining themselves.
Where to start on Monday
Most of the companies we talk to are not trying to hide anything. They are trying to use these tools to get real work done without creating a record they cannot explain later. That is a solvable problem, and it is boring work: decide which lanes stay local, write down what counts as sensitive, pick tools whose retention terms you can read out loud, and stop treating the delete button as a strategy. The transcripts worth keeping should be kept on purpose, in a place you control, and the ones that should never have existed should not have been typed in the first place.
If you are putting AI into a business that handles other people's information and you want a second opinion on where the line should sit, that is the conversation we have with Atlanta businesses all the time. Bring the list of what your team pasted into a chat window last week. It is usually a shorter list than people expect, and a more interesting one.