Month: November 2025

Posted on

A handful of cherries does not make a sundae

Why content licensing cannot solve AI’s training-data problem

I have just published an article as part of the ProMarket symposium for the University of Chicago, the Booth School of Business, “The False Hope of Content Licensing at Internet Scale”

Although the article is not very long, I thought I would summarize my point even more briefly here.

AI developers have been on a shopping spree. Since mid-2023, OpenAI, Google, Anthropic and Meta have collectively spent hundreds of millions of dollars striking deals with publishers. OpenAI alone has inked agreements with everyone from the Associated Press to Condé Nast, gaining access to archives from The New Yorker, Vogue, The Wall Street Journal and dozens of other publications.

To many watching from the sidelines, these deals offer tantalizing proof that AI companies can—and should—pay for the content they consume.

However, the agreements grabbing headlines represent a tiny fraction of the data needed to train cutting-edge language models. Modern AI systems require trillions of diverse tokens scraped from across the internet—a scale and diversity that traditional licensing simply cannot reach.

To see why, read the full article: The False Hope of Content Licensing at Internet Scale

Posted on

Copyright Winter is Coming (to Wikipedia?)

Judge Stein’s Order Denying OpenAI’s Motion to Dismiss in Authors Guild v. OpenAI, Inc., No. 25-md-3143 (SHS) (OTW) (S.D.N.Y. Oct. 27, 2025)

A new ruling in Authors Guild v. OpenAI has major implications for copyright law, well beyond artificial intelligence. On October 27, 2025, Judge Sidney Stein of the Southern District of New York denied OpenAI’s motion to dismiss claims that ChatGPT outputs infringed the rights of authors such as George R.R. Martin and David Baldacci. The opinion suggests that short summaries of popular works of fiction are very likely infringing (unless fair use comes to the rescue).

This is a fundamental assault on the idea, expression, distinction as applied to works of fiction. It places thousands of Wikipedia entries in the copyright crosshairs and suggests that any kind of summary or analysis of a work of fiction is presumptively infringing.

A white walker in a desolate field reading Wikipedia (an AI Image by Gemini)

Copyright and derivative works

In Penguin Random House LLC v. Colting, the Southern District of New York found that defendant’s “The Kinderguide” series, which condensed classic works of literature into children’s books, infringed the copyrights in the original works despite being marketed as educational tools for parents to introduce literature to young children.

Every year, I ask students in my copyright class why the children’s versions of classic novels in Colting were found to be infringing but a Wikipedia summary of the plots of those same books probably wouldn’t be. A recent ruling in the consolidated copyright cases against OpenAI means I might have to reconsider.

The ruling

On October 27, 2025, Judge Stein of the Southern District of New York denied OpenAI’s motion to dismiss the output-based copyright infringement claims brought by a class of authors including David Baldacci, George R.R. Martin, and others.

OpenAI had argued, reasonably enough, that the authors’ complaint failed to plausibly allege substantial similarity between any of their works and any of ChatGPT’s outputs. It is standard practice in copyright litigation to attach a copy of the plaintiff’s work and the allegedly infringing work, but the court held that “the outputs plaintiffs submitted along with their opposition to OpenAI’s motion were incorporated into the Consolidated Class Action Complaint by reference” and that it was enough that their Complaint repeatedly made “clear, definite and substantial references” to the outputs. Losing that civil procedure skirmish was probably a bad sign for OpenAI—a bit like the menacing prologue in A Game of Thrones, you sense that Copyright Winter is Coming .  

Judge Stein then went on to evaluate one of the more detailed chat-GPT generated summaries relating to A Game of Thrones, the 694 page novel by George R. R. Martin which eventually became the famous HBO series of the same name. Even though this was only a motion to dismiss, where the cards are stacked against the defendant, I was surprised by how easily the judge could conclude that:

“A more discerning observer could easily conclude that this detailed summary is substantially similar to Martin’s original work, including because the summary conveys the overall tone and feel of the original work by parroting the plot, characters, and themes of the original.”

The judge described the ChatGPT summaries as:

“most certainly attempts at abridgment or condensation of some of the central copyrightable elements of the original works such as setting, plot, and characters”

He saw them as:

“conceptually similar to—although admittedly less detailed than—the plot summaries in Twin Peaks and in Penguin Random House LLC v. Colting, where the district court found that works that summarized in detail the plot, characters, and themes of original works were substantially similar to the original works.” (emphasis added).

To say that the less than 580-word GPT summary of A Game of Thrones is “less detailed” than the 128-page Welcome to Twin Peaks Guide in the Twin Peaks case, or the various children’s books based on famous works of literature in the Colting case, is a bit of an understatement.

The Wikipedia comparison

To see why the latest OpenAI ruling is so surprising, it helps to compare the ChatGPT summary of A Game of Thrones to the equivalent Wikipedia plot summary. I read them both so you don’t have to.

The ChatGPT summary of a Game of Thrones is about 580 words long and captures the essential narrative arc of the novel. It covers all three major storylines: the political intrigue in King’s Landing culminating in Ned Stark’s execution (spoiler alert), Jon Snow’s journey with the Night’s Watch at the Wall, and Daenerys Targaryen’s transformation from fearful bride (more on this shortly) to dragon mother across the Narrow Sea. In this regard, it is very much like the 800 word Wikipedia plot summary. Each summary presents the central conflict between the Starks and Lannisters, the revelation of Cersei and Jaime’s incestuous relationship, and the key plot points that set the larger series in motion.

I could say more about their similarities, but I’m concerned that if I explored the summaries in any greater detail, the Authors Guild might think that I am also infringing George R. R. Martin’s copyright, so I’ll move on to the minor differences.

The key difference between the Wikipedia summary and the GPT summary is structural. The Wikipedia summary takes a geographic approach, dividing the narrative into three distinct sections based on location: “In the Seven Kingdoms,” “On the Wall,” and “Across the Narrow Sea.” This structure mirrors the way the novel follows different characters in different locations, to the point where you begin to wonder whether these characters will ever meet. In contrast, the GPT summary follows a more analytical structure, beginning with contextual information about the setting and the series as a whole, then proceeding through sections that follow a roughly chronological progression through the major plot points.

There are some minor differences. The Wikipedia summary provides more granular plot details and clearer causal chains between events. It explains, for instance, how Catelyn’s arrest of Tyrion leads to Tywin’s retaliatory raids on the Riverlands, which in turn necessitates Robb’s strategic alliance with House Frey to secure a crucial bridge crossing. The Wikipedia summary also includes more secondary characters and subplots, such as Tyrion’s recruitment of Bronn as his champion in trial by combat, and Jon’s protection of Samwell Tarly.

The Wikipedia summary probably assumes a greater familiarity with the fantasy genre, whereas the GPT summary might be more helpful to the uninitiated. The GPT summary explains the significance of the long summer and impending winter and explicitly sets out the novel’s major themes.

In broad strokes, however, there is very little daylight between these two summaries. They are remarkably similar in what they include and in what they leave out. Most notably, both summaries sanitize Daenerys’s storyline by omitting the sexual violence that is fundamental to her character arc. This is particularly striking because sexual violence is central to Martin’s narrative in so many places and to the narrative arc of several of the main characters.

If GPT is substantially similar, so is Wikipedia

I don’t see how the ChatGPT summary could infringe the copyright in George R. R. Martin’s novel, if the Wikipedia summary doesn’t. A chilling prospect indeed, but I don’t think that either one is infringing.

It’s absolutely true that you can infringe the copyright in a novel by merely borrowing some of the key characters, plot points and settings, and spinning out a sequel or a prequel. In copyright, we call this a derivative work. But just because sequels and children’s versions of novels are often infringing, doesn’t mean that a dry and concise analytical summary of a novel is infringing.

Why not? It’s actually the act of taking those key structural elements, the skeleton of the novel if you like, and adding new flesh to them to create a new fully realized work that makes an unauthorized sequel infringing.

What’s at stake

Judge Stein’s order doesn’t resolve the authors’ claims, not by a long shot. And he was careful to point out that he was only considering the plausibility of the infringement allegation and not any potential fair use defenses. Nonetheless, I think this is a troubling decision that sets the bar on substantial similarity far too low.

The fact that “[w]hen prompted, ChatGPT can generate accurate summaries of books authored by plaintiffs and generate outlines for potential sequels to plaintiffs’ books” falls well short of demonstrating that such outputs by themselves would be regarded by the ordinary observer as substantially similar to a fully realized novel.

Posted on

Do law schools need Harvey.AI?

Harvey.AI is following the playbook of Westlaw and Lexis by trying to establish itself as the go-to AI tool of choice for lawyers before they even become lawyers. I asked my university library to organize a Harvey demo so that we could think about joining the ranks of Stanford, UCLA, NYU, Notre Dame, WashU, Penn, UChicago, Boston University, Fordham, BYU, UGA, Villanova, Baylor, SMU, and Vanderbilt. (As reported by Above The Law)  (https://abovethelaw.com/2025/10/harvey-snags-even-more-seats-in-the-t14).

This post is primarily based on a one-hour product demonstration given to us by a Harvey representative. To have a really well informed view on the product, I would want more hands-eye experience but there is surprisingly little information about what Harvey is actually offering online beyond the company’s own press releases. So, I thought my colleagues at other universities might find this assessment interesting.

TLDR

Meh, it’s OK, but law schools probably don’t need it and are probably only jumping on the bandwagon so that they can be part of the press release.

What is Harvey?

Harvey.AI is a legal-tech and professional services AI company whose flagship product is a generative AI assistant designed specifically for legal workflows used by law firms, in-house legal teams, and other professional services organizations. On its website, Harvey characterizes itself as “Professional Class AI” for leading professional service firms, emphasizing that its technology is domain-specific. In other words, it’s an AI system fine-tuned and optimized for legal and related professional work.

Use Cases and Contraindications

The first thing to understand about Harvey is that it is categorically not a legal research tool. Harvey essentially offers its clients a way of integrating generative AI into some routine drafting and analytical tasks that are quite common in legal practice.

Here are some common use scenarios:

If you have already identified the relevant case law and have a memo template to hand, Harvey AI can help you draft a legal research memo in double-quick time.

Alternatively, Harvey can help you review the key terms of a lengthy contract or almost any other synthesis or summarization task you could imagine.

Another good use case for the Harvey AI platform would be drafting an agreement or marking up the other side’s agreement in light of your own preferred templates. Harvey’s process for drafting from scratch seems directly analogous to vibe coding in software, but with a nice Microsoft Word integration.

You can also use Harvey for analysis and ideation (i.e., brainstorming). I can imagine coming to the end of a 3-month trial, throwing all the relevant documents into Harvey, and then launching into a discussion about closing argument strategy. Or, uploading a motion for summary judgment and the other side’s response, and then trying to anticipate the kinds of questions you might get from the bench.

The Harvey’s Value Proposition

You can already do almost all of this with ChatGPT, Gemini, Claude, and the like, subject to volume limitations on how many documents you upload. So, the natural question is, what value add does Harvey AI offer?

Fine tuning and model switching

One of the advantages claimed by Harvey is that rather than using foundation models like GPT directly, you would be engaging with custom versions of those model, fine-tuned on training data relevant to law and legal analysis. I could imagine that in some fields this would be a significant advantage, but I wonder how much of an advantage it is in the legal field given that most of that fine-tuning data is going to be public domain legal texts that are already well represented in the foundation models.

Another thing Harvey sees as a benefit is that they are not tied to any one model. They currently use three different fine-tuned foundation models, GPT, Gemini, and Claude, and they allocate tasks according to comparative advantage.

Security and confidentiality

By default, prompts and documents transmitted to a company like OpenAI may be used in training, will definitely be stored on OpenAI’s servers (at least for a while), and thus might be subject to discovery through appropriate legal processes. OpenAI has a setting where users can opt out of training that specifies that their data will only be retained for 30 days. This is probably good enough for many casual uses and even some mildly sensitive uses, but it’s obviously not enough for material that is subject to attorney-client privilege.

Accordingly, one of the key differentiators offered by Harvey AI is that the documents you upload and the prompts you write will not be accessible to Harvey or any third party, and that all of the information processing takes place in a secure Microsoft Azure environment with end-to-end encryption. This is probably the absolute minimum necessary to use LLMs for legal work. A large law firm could go one step further and actually host its own model in-house rather than relying on Microsoft. That extra layer of security might be required by some especially restrictive protective orders in litigation or by some especially sensitive clients. That sounds great, but I’m pretty sure I already get all that from Microsoft Copilot (although I would have to do a deep dive into the terms and conditions, Microsoft offers my university, to be sure).

Another nice feature of Harvey is that the client administrator can set permissions for individual users and for particular teams of users. This is critical in a corporate law environment where access to sensitive documents needs to be compartmentalized. It’s also critical if Harvey is being made available to students in a law school environment because students taking courses such as foundational Legal Writing and Research classes should probably not have access to Harvey AI.

Document Review (Retrieval-Augmented Generation)

Harvey AI has a good user interface for analyzing large volumes of documents. That is essentially an implementation of retrieval-augmented generation (RAG).

What’s RAG?

In very simple terms, RAG is an alternative to just answering a question through next-token prediction, relying on bulk context and whatever knowledge and understanding is latent in a foundation model. In a RAG process, the user query is translated into a document query. The document query identifies sections of documents that seem relevant to the query. Those sections are then collated and fed back into a general model which attempts to answer the question based on the specifically retrieved chunks of text. Platforms like ChatGPT are using a process like this any time you see them searching the web and providing links back to particular documents.

Harvey does RAG pretty well

RAG sounds like a great idea in theory. But whether it works in practice depends on how good the matching method is, which can vary a lot from context to context. In any RAG process, you will never know what relevant chunks of text were overlooked, and you won’t know whether the interpretive part of the model has drawn the appropriate inferences from the chunks it has retrieved unless you go back and check the original sources. One of the things I liked about the Harvey UX is that it made it easy to inspect the original document fragments and it had a clear process for checking off that these had actually been interrogated.

Example use cases would be looking for a change of control provisions in licensing agreements, as part of merger due diligence, or in document review for litigation. The Harvey representative we spoke to candidly admitted that the system performed really well in establishing a chronology, except in relation to emails. This makes sense, because an email thread contains lots of different dates all jumbled in together, but it is clearly a major limitation.

Prompting and training

Another value-add our representative stressed was prompting. Our representatives seem to be saying not only that Harvey would be running some thoughtfully-crafted prompts in the background, essentially running interference between user instructions and the models, but also that individual clients could do this for themselves. I can see why this might be an appealing feature to some people, but I’m not entirely convinced that making the steps in an analytical process obscure from the user is a good idea.

My Assessment

Generative AI as legal technology

Before we get into the specific pros and cons of Harvey, we need to consider the appropriate uses of generative AI as a legal technology more generally.

Many key deliverables in the legal field are in the form of text. But it’s relatively rare that the value of that text is entirely contained within the document itself. When a lawyer explains something to a client, they aren’t just helping their client understand something. They are also making a set of representations about the thought, diligence, and analysis that has gone into formulating that advice. Clients don’t just want text for its own sake, they want text you stand behind.

Accordingly, the most significant uses of generative AI in the legal field will be ones that accelerate a drafting-review or document-analysis process, as opposed to merely substituting for the underlying analysis.

Responsible use of generative AI in the legal field must be accompanied by either:

  • strong validation mechanisms (such as a process for clicking through the footnotes to confirm that the document in question really says what the model represented),
  • a knowing and well-informed acceptance of certain risks, or
  • the kind of external validation that a lawyer who is already familiar with the underlying materials intrinsically provides.

The validity questions that need to be answered before deploying generative AI as a legal technology are not limited to the problem of hallucinations in the narrow sense of invented cases, citations, and quotations.

Harvey claims to do very well in dealing with hallucinations, but it’s important to situate this in the context that Harvey is not a legal research tool. The kinds of tasks that Harvey says that its product should be used for are exactly the kind of tasks where one would expect a much lower instance of hallucinations. Why? Because they are mostly summary or translation tasks where the model has specific documents or templates to draw from. Even so, I’m a bit skeptical that the rate of hallucinations is really as low as Harvey claims.

The value proposition for law firms

Depending on the cost, I can see that Harvey would be a very attractive proposition for law firms of all sizes. Most of what Harvey offers can be replicated through an enterprise agreement with one of the main AI providers. Harvey offers a turnkey solution and a good user interface. You can think of it as ChatGPT in a black turtleneck, but that’s no bad thing.

Is it worth it? That depends on the cost, and the cost of the alternatives.

The value proposition for law schools

There is no doubt that most of our students are already using generative AI. It seems appropriate that we begin training them to do so properly and responsibly at the earliest opportunity. That said, the availability of generative AI to students taking specific skills courses could easily undermine the development of those skills. Rather than simply making Harvey available to all students, it makes sense to exclude first-year students and perhaps some upper-level skills courses. But obviously, we would want students in our Advanced Legal Writing course (where we are teaching AI skills) to have access to this tool.

If we decide that we don’t want students in our clinics using generative AI, then one of the major selling points of Harvey disappears. Our students don’t need the robust confidentiality protection that Harvey offers.

If Harvey is offering commercially reasonable terms, I still think it is an attractive proposition. But its value in legal education seems to me to be really quite limited. Our students are not conducting massive document review exercises or working with in-house templates. Most of the things students would find compelling about using Harvey, they can already do with Microsoft Co-Pilot, ChatGPT, Gemini, and Claude.