Tag: AI

Posted on

The Mouse and the Model: the Disney-OpenAI Deal

The other shoe has finally dropped.

Today, December 11, 2025, OpenAI and Disney announced a partnership that essentially signals a marriage between generative AI and legacy media. Although some kind of deal was inevitable, the range and scope of this one are striking. Disney is sinking $1 billion into OpenAI for an equity stake and warrants, while simultaneously inking a three-year licensing deal.

The immediate result? OpenAI’s Sora and ChatGPT will legally ingest over 200 marquee characters from the Disney, Marvel, Pixar, and Star Wars vaults. We’ll see AI-generated Disney content on Disney+, and Disney employees will get enterprise-grade access to OpenAI’s tools. Notably, actor likenesses are off the table—a nod to the sensitivities of the recent labor strikes—but the direction of travel is clear. For more reporting, see the Verge.

Why it matters

Addressing the “Snoopy Problem“

AI companies and copyright industries are beginning to understand, and become reconciled to, the fact that neither side is going to score an absolute victory when it comes to the fair use issue for AI training. AI training that results in a model that learns from, but does not reproduce, the training data looks very likely to be upheld as fair use. Two recent cases held as much on summary judgement and this aligns with a line of precedent “nonexpressive use” cases that predate generative AI.

However, it’s becoming increasingly clear that it’s hard to train generative AI models to be really useful without some degree of memorization of the training data along the way. This is particularly problematic when it comes to copyrightable characters, because copyright protects characters more abstractly than most things. This is the well-known Snoopy problem (a term I coined in 2023).

Faced with this increasingly clear reality, it makes sense for consumer facing AI companies and entertainment Giants like Disney to think about licensing arrangements.

This deal signals a retreat from the fair use absolutism of early AI development. OpenAI and Disney have effectively priced the risk of memorization. Instead of spending the next decade in discovery arguing over pixel similarities, they are moving to a licensing regime. Disney gets paid and retains control; OpenAI gets legal certainty and the ability to serve the entertainment industry without looking over its shoulder.

Capital Crunch?

With competitors like Anthropic eyeing public listings, OpenAI’s decision to take strategic capital from a corporate giant like Disney may be telling. It suggests we are hitting a saturation point for traditional venture capital at the scale these foundation models require. It also hints that OpenAI sees more value in “smart money,” than in the volatility of the public markets. Disney isn’t just a piggy bank; it’s a hedge. By entangling itself with the world’s premier IP holder, OpenAI makes itself indispensable to the very industry that threatened to sue it out of existence. Or, I’m sure that’s the theory, whether it pans out that way remains to be seen.

The End of the Scaling Era?

Finally, this move also adds to the “Data Scarcity” thesis. The era of simply scraping the open web to make models smarter (2017–2025) might be over. The low-hanging fruit of the public internet has been picked, processed, recycled into synthetic data, and processed again, every which way you can imagine. To get better, and to stay ahead of open source rivals, companies like OpenAI are going to need access to data that no one else has. Google has YouTube; OpenAI now has the Magic Kingdom.

The Bottom Line

This is the template for the future. We are moving away from total war between AI and Content, toward a negotiated partition of the world. The tech companies provide the engine; the media giants provide the fuel. And for now, at least, both sides seem to think that’s a better outcome than leaving it up to a judge.

I wrote this blog post the morning the deal was announced, because it fits surprisingly well with a Law Review article I am writing, “The Snoopy Solution: How Fair Use and Licensing for Generative AI Can Coexist” based on a talk I gave at Yale last month.

Posted on

Copyright Winter is Coming (to Wikipedia?)

Judge Stein’s Order Denying OpenAI’s Motion to Dismiss in Authors Guild v. OpenAI, Inc., No. 25-md-3143 (SHS) (OTW) (S.D.N.Y. Oct. 27, 2025)

A new ruling in Authors Guild v. OpenAI has major implications for copyright law, well beyond artificial intelligence. On October 27, 2025, Judge Sidney Stein of the Southern District of New York denied OpenAI’s motion to dismiss claims that ChatGPT outputs infringed the rights of authors such as George R.R. Martin and David Baldacci. The opinion suggests that short summaries of popular works of fiction are very likely infringing (unless fair use comes to the rescue).

This is a fundamental assault on the idea, expression, distinction as applied to works of fiction. It places thousands of Wikipedia entries in the copyright crosshairs and suggests that any kind of summary or analysis of a work of fiction is presumptively infringing.

A white walker in a desolate field reading Wikipedia (an AI Image by Gemini)

Copyright and derivative works

In Penguin Random House LLC v. Colting, the Southern District of New York found that defendant’s “The Kinderguide” series, which condensed classic works of literature into children’s books, infringed the copyrights in the original works despite being marketed as educational tools for parents to introduce literature to young children.

Every year, I ask students in my copyright class why the children’s versions of classic novels in Colting were found to be infringing but a Wikipedia summary of the plots of those same books probably wouldn’t be. A recent ruling in the consolidated copyright cases against OpenAI means I might have to reconsider.

The ruling

On October 27, 2025, Judge Stein of the Southern District of New York denied OpenAI’s motion to dismiss the output-based copyright infringement claims brought by a class of authors including David Baldacci, George R.R. Martin, and others.

OpenAI had argued, reasonably enough, that the authors’ complaint failed to plausibly allege substantial similarity between any of their works and any of ChatGPT’s outputs. It is standard practice in copyright litigation to attach a copy of the plaintiff’s work and the allegedly infringing work, but the court held that “the outputs plaintiffs submitted along with their opposition to OpenAI’s motion were incorporated into the Consolidated Class Action Complaint by reference” and that it was enough that their Complaint repeatedly made “clear, definite and substantial references” to the outputs. Losing that civil procedure skirmish was probably a bad sign for OpenAI—a bit like the menacing prologue in A Game of Thrones, you sense that Copyright Winter is Coming .  

Judge Stein then went on to evaluate one of the more detailed chat-GPT generated summaries relating to A Game of Thrones, the 694 page novel by George R. R. Martin which eventually became the famous HBO series of the same name. Even though this was only a motion to dismiss, where the cards are stacked against the defendant, I was surprised by how easily the judge could conclude that:

“A more discerning observer could easily conclude that this detailed summary is substantially similar to Martin’s original work, including because the summary conveys the overall tone and feel of the original work by parroting the plot, characters, and themes of the original.”

The judge described the ChatGPT summaries as:

“most certainly attempts at abridgment or condensation of some of the central copyrightable elements of the original works such as setting, plot, and characters”

He saw them as:

“conceptually similar to—although admittedly less detailed than—the plot summaries in Twin Peaks and in Penguin Random House LLC v. Colting, where the district court found that works that summarized in detail the plot, characters, and themes of original works were substantially similar to the original works.” (emphasis added).

To say that the less than 580-word GPT summary of A Game of Thrones is “less detailed” than the 128-page Welcome to Twin Peaks Guide in the Twin Peaks case, or the various children’s books based on famous works of literature in the Colting case, is a bit of an understatement.

The Wikipedia comparison

To see why the latest OpenAI ruling is so surprising, it helps to compare the ChatGPT summary of A Game of Thrones to the equivalent Wikipedia plot summary. I read them both so you don’t have to.

The ChatGPT summary of a Game of Thrones is about 580 words long and captures the essential narrative arc of the novel. It covers all three major storylines: the political intrigue in King’s Landing culminating in Ned Stark’s execution (spoiler alert), Jon Snow’s journey with the Night’s Watch at the Wall, and Daenerys Targaryen’s transformation from fearful bride (more on this shortly) to dragon mother across the Narrow Sea. In this regard, it is very much like the 800 word Wikipedia plot summary. Each summary presents the central conflict between the Starks and Lannisters, the revelation of Cersei and Jaime’s incestuous relationship, and the key plot points that set the larger series in motion.

I could say more about their similarities, but I’m concerned that if I explored the summaries in any greater detail, the Authors Guild might think that I am also infringing George R. R. Martin’s copyright, so I’ll move on to the minor differences.

The key difference between the Wikipedia summary and the GPT summary is structural. The Wikipedia summary takes a geographic approach, dividing the narrative into three distinct sections based on location: “In the Seven Kingdoms,” “On the Wall,” and “Across the Narrow Sea.” This structure mirrors the way the novel follows different characters in different locations, to the point where you begin to wonder whether these characters will ever meet. In contrast, the GPT summary follows a more analytical structure, beginning with contextual information about the setting and the series as a whole, then proceeding through sections that follow a roughly chronological progression through the major plot points.

There are some minor differences. The Wikipedia summary provides more granular plot details and clearer causal chains between events. It explains, for instance, how Catelyn’s arrest of Tyrion leads to Tywin’s retaliatory raids on the Riverlands, which in turn necessitates Robb’s strategic alliance with House Frey to secure a crucial bridge crossing. The Wikipedia summary also includes more secondary characters and subplots, such as Tyrion’s recruitment of Bronn as his champion in trial by combat, and Jon’s protection of Samwell Tarly.

The Wikipedia summary probably assumes a greater familiarity with the fantasy genre, whereas the GPT summary might be more helpful to the uninitiated. The GPT summary explains the significance of the long summer and impending winter and explicitly sets out the novel’s major themes.

In broad strokes, however, there is very little daylight between these two summaries. They are remarkably similar in what they include and in what they leave out. Most notably, both summaries sanitize Daenerys’s storyline by omitting the sexual violence that is fundamental to her character arc. This is particularly striking because sexual violence is central to Martin’s narrative in so many places and to the narrative arc of several of the main characters.

If GPT is substantially similar, so is Wikipedia

I don’t see how the ChatGPT summary could infringe the copyright in George R. R. Martin’s novel, if the Wikipedia summary doesn’t. A chilling prospect indeed, but I don’t think that either one is infringing.

It’s absolutely true that you can infringe the copyright in a novel by merely borrowing some of the key characters, plot points and settings, and spinning out a sequel or a prequel. In copyright, we call this a derivative work. But just because sequels and children’s versions of novels are often infringing, doesn’t mean that a dry and concise analytical summary of a novel is infringing.

Why not? It’s actually the act of taking those key structural elements, the skeleton of the novel if you like, and adding new flesh to them to create a new fully realized work that makes an unauthorized sequel infringing.

What’s at stake

Judge Stein’s order doesn’t resolve the authors’ claims, not by a long shot. And he was careful to point out that he was only considering the plausibility of the infringement allegation and not any potential fair use defenses. Nonetheless, I think this is a troubling decision that sets the bar on substantial similarity far too low.

The fact that “[w]hen prompted, ChatGPT can generate accurate summaries of books authored by plaintiffs and generate outlines for potential sequels to plaintiffs’ books” falls well short of demonstrating that such outputs by themselves would be regarded by the ordinary observer as substantially similar to a fully realized novel.

Posted on

Do law schools need Harvey.AI?

Harvey.AI is following the playbook of Westlaw and Lexis by trying to establish itself as the go-to AI tool of choice for lawyers before they even become lawyers. I asked my university library to organize a Harvey demo so that we could think about joining the ranks of Stanford, UCLA, NYU, Notre Dame, WashU, Penn, UChicago, Boston University, Fordham, BYU, UGA, Villanova, Baylor, SMU, and Vanderbilt. (As reported by Above The Law)  (https://abovethelaw.com/2025/10/harvey-snags-even-more-seats-in-the-t14).

This post is primarily based on a one-hour product demonstration given to us by a Harvey representative. To have a really well informed view on the product, I would want more hands-eye experience but there is surprisingly little information about what Harvey is actually offering online beyond the company’s own press releases. So, I thought my colleagues at other universities might find this assessment interesting.

TLDR

Meh, it’s OK, but law schools probably don’t need it and are probably only jumping on the bandwagon so that they can be part of the press release.

What is Harvey?

Harvey.AI is a legal-tech and professional services AI company whose flagship product is a generative AI assistant designed specifically for legal workflows used by law firms, in-house legal teams, and other professional services organizations. On its website, Harvey characterizes itself as “Professional Class AI” for leading professional service firms, emphasizing that its technology is domain-specific. In other words, it’s an AI system fine-tuned and optimized for legal and related professional work.

Use Cases and Contraindications

The first thing to understand about Harvey is that it is categorically not a legal research tool. Harvey essentially offers its clients a way of integrating generative AI into some routine drafting and analytical tasks that are quite common in legal practice.

Here are some common use scenarios:

If you have already identified the relevant case law and have a memo template to hand, Harvey AI can help you draft a legal research memo in double-quick time.

Alternatively, Harvey can help you review the key terms of a lengthy contract or almost any other synthesis or summarization task you could imagine.

Another good use case for the Harvey AI platform would be drafting an agreement or marking up the other side’s agreement in light of your own preferred templates. Harvey’s process for drafting from scratch seems directly analogous to vibe coding in software, but with a nice Microsoft Word integration.

You can also use Harvey for analysis and ideation (i.e., brainstorming). I can imagine coming to the end of a 3-month trial, throwing all the relevant documents into Harvey, and then launching into a discussion about closing argument strategy. Or, uploading a motion for summary judgment and the other side’s response, and then trying to anticipate the kinds of questions you might get from the bench.

The Harvey’s Value Proposition

You can already do almost all of this with ChatGPT, Gemini, Claude, and the like, subject to volume limitations on how many documents you upload. So, the natural question is, what value add does Harvey AI offer?

Fine tuning and model switching

One of the advantages claimed by Harvey is that rather than using foundation models like GPT directly, you would be engaging with custom versions of those model, fine-tuned on training data relevant to law and legal analysis. I could imagine that in some fields this would be a significant advantage, but I wonder how much of an advantage it is in the legal field given that most of that fine-tuning data is going to be public domain legal texts that are already well represented in the foundation models.

Another thing Harvey sees as a benefit is that they are not tied to any one model. They currently use three different fine-tuned foundation models, GPT, Gemini, and Claude, and they allocate tasks according to comparative advantage.

Security and confidentiality

By default, prompts and documents transmitted to a company like OpenAI may be used in training, will definitely be stored on OpenAI’s servers (at least for a while), and thus might be subject to discovery through appropriate legal processes. OpenAI has a setting where users can opt out of training that specifies that their data will only be retained for 30 days. This is probably good enough for many casual uses and even some mildly sensitive uses, but it’s obviously not enough for material that is subject to attorney-client privilege.

Accordingly, one of the key differentiators offered by Harvey AI is that the documents you upload and the prompts you write will not be accessible to Harvey or any third party, and that all of the information processing takes place in a secure Microsoft Azure environment with end-to-end encryption. This is probably the absolute minimum necessary to use LLMs for legal work. A large law firm could go one step further and actually host its own model in-house rather than relying on Microsoft. That extra layer of security might be required by some especially restrictive protective orders in litigation or by some especially sensitive clients. That sounds great, but I’m pretty sure I already get all that from Microsoft Copilot (although I would have to do a deep dive into the terms and conditions, Microsoft offers my university, to be sure).

Another nice feature of Harvey is that the client administrator can set permissions for individual users and for particular teams of users. This is critical in a corporate law environment where access to sensitive documents needs to be compartmentalized. It’s also critical if Harvey is being made available to students in a law school environment because students taking courses such as foundational Legal Writing and Research classes should probably not have access to Harvey AI.

Document Review (Retrieval-Augmented Generation)

Harvey AI has a good user interface for analyzing large volumes of documents. That is essentially an implementation of retrieval-augmented generation (RAG).

What’s RAG?

In very simple terms, RAG is an alternative to just answering a question through next-token prediction, relying on bulk context and whatever knowledge and understanding is latent in a foundation model. In a RAG process, the user query is translated into a document query. The document query identifies sections of documents that seem relevant to the query. Those sections are then collated and fed back into a general model which attempts to answer the question based on the specifically retrieved chunks of text. Platforms like ChatGPT are using a process like this any time you see them searching the web and providing links back to particular documents.

Harvey does RAG pretty well

RAG sounds like a great idea in theory. But whether it works in practice depends on how good the matching method is, which can vary a lot from context to context. In any RAG process, you will never know what relevant chunks of text were overlooked, and you won’t know whether the interpretive part of the model has drawn the appropriate inferences from the chunks it has retrieved unless you go back and check the original sources. One of the things I liked about the Harvey UX is that it made it easy to inspect the original document fragments and it had a clear process for checking off that these had actually been interrogated.

Example use cases would be looking for a change of control provisions in licensing agreements, as part of merger due diligence, or in document review for litigation. The Harvey representative we spoke to candidly admitted that the system performed really well in establishing a chronology, except in relation to emails. This makes sense, because an email thread contains lots of different dates all jumbled in together, but it is clearly a major limitation.

Prompting and training

Another value-add our representative stressed was prompting. Our representatives seem to be saying not only that Harvey would be running some thoughtfully-crafted prompts in the background, essentially running interference between user instructions and the models, but also that individual clients could do this for themselves. I can see why this might be an appealing feature to some people, but I’m not entirely convinced that making the steps in an analytical process obscure from the user is a good idea.

My Assessment

Generative AI as legal technology

Before we get into the specific pros and cons of Harvey, we need to consider the appropriate uses of generative AI as a legal technology more generally.

Many key deliverables in the legal field are in the form of text. But it’s relatively rare that the value of that text is entirely contained within the document itself. When a lawyer explains something to a client, they aren’t just helping their client understand something. They are also making a set of representations about the thought, diligence, and analysis that has gone into formulating that advice. Clients don’t just want text for its own sake, they want text you stand behind.

Accordingly, the most significant uses of generative AI in the legal field will be ones that accelerate a drafting-review or document-analysis process, as opposed to merely substituting for the underlying analysis.

Responsible use of generative AI in the legal field must be accompanied by either:

  • strong validation mechanisms (such as a process for clicking through the footnotes to confirm that the document in question really says what the model represented),
  • a knowing and well-informed acceptance of certain risks, or
  • the kind of external validation that a lawyer who is already familiar with the underlying materials intrinsically provides.

The validity questions that need to be answered before deploying generative AI as a legal technology are not limited to the problem of hallucinations in the narrow sense of invented cases, citations, and quotations.

Harvey claims to do very well in dealing with hallucinations, but it’s important to situate this in the context that Harvey is not a legal research tool. The kinds of tasks that Harvey says that its product should be used for are exactly the kind of tasks where one would expect a much lower instance of hallucinations. Why? Because they are mostly summary or translation tasks where the model has specific documents or templates to draw from. Even so, I’m a bit skeptical that the rate of hallucinations is really as low as Harvey claims.

The value proposition for law firms

Depending on the cost, I can see that Harvey would be a very attractive proposition for law firms of all sizes. Most of what Harvey offers can be replicated through an enterprise agreement with one of the main AI providers. Harvey offers a turnkey solution and a good user interface. You can think of it as ChatGPT in a black turtleneck, but that’s no bad thing.

Is it worth it? That depends on the cost, and the cost of the alternatives.

The value proposition for law schools

There is no doubt that most of our students are already using generative AI. It seems appropriate that we begin training them to do so properly and responsibly at the earliest opportunity. That said, the availability of generative AI to students taking specific skills courses could easily undermine the development of those skills. Rather than simply making Harvey available to all students, it makes sense to exclude first-year students and perhaps some upper-level skills courses. But obviously, we would want students in our Advanced Legal Writing course (where we are teaching AI skills) to have access to this tool.

If we decide that we don’t want students in our clinics using generative AI, then one of the major selling points of Harvey disappears. Our students don’t need the robust confidentiality protection that Harvey offers.

If Harvey is offering commercially reasonable terms, I still think it is an attractive proposition. But its value in legal education seems to me to be really quite limited. Our students are not conducting massive document review exercises or working with in-house templates. Most of the things students would find compelling about using Harvey, they can already do with Microsoft Co-Pilot, ChatGPT, Gemini, and Claude.

Posted on

Legal Scholars Roundtable on Artificial Intelligence 2026 (save the date)

Emory Law is proud to host the 5th annual Legal Scholars Roundtable on Artificial Intelligence on April 9-10, 2026, at Emory University in Atlanta, Georgia. The Legal Scholars Roundtable on Artificial Intelligence is a forum for the discussion of current legal scholarship on AI, covering a range of methodologies, topics, perspectives, and legal intersections.

We will make a formal call for papers in January with submission deadline some time in February.

The AI Roundtable is conveyed by Prof. Matthew Sag (Emory Law) and Prof. Charlotte Tschider (Loyola Law Chicago)

Posted on

Competition from AI music, Country Girls Make Do

As of October 2025, Suno and Udio are two text-to-music AI platforms that let users create full songs—including lyrics, vocals, and artwork—simply by entering text prompts. Some of this music is unappealing, even to its creators (protagonists?), but music scene insiders have assured me that some of the music emanating from these platforms is good enough to provoke a wistful, “I wish I had written that.”

AI music is also becoming more popular. A recent article in The Economist (of all places) recounts the viral success of “Country Girls Make Do,” a raunchy parody country song generated by artificial intelligence under the pseudonym Beats By AI. The song apparently features on TikTok where users prank the unsuspecting by playing it under false pretenses.

This is more than a one off. Acts such as Aventhis and The Velvet Sundown, also AI-based, have attracted hundreds of thousands of monthly listeners on Spotify. These tools allow for rapid and prolific production: Beats By AI reportedly releases a new song every day. This is not simply a case of streaming fraud where AI slop steals music plays from real artists by adopting confusing names—Spotify recently removed 75 million such tracks, citing “bad actors” flooding the platform with low-quality content. Some people at least, like some AI music. The Economist reports a Luminate survey finding that, one-third of Americans accept AI-written instrumentals, nearly 30% are fine with AI lyrics, and over a quarter do not mind AI vocals.

No music stands alone, but AI music arguably even less so

The appeal of these tracks lies partly in their echos of established genres and tropes, with a dash of irony and experimentation thrown in. It’s to be seen whether this portends a consumer-driven revolution in content creation where listeners generate their own entertainment rather than relying on record labels.

What does this mean for copyright law?

Although the Copyright Office would not regard works of The Velvet Sundown or Beats By AI as copyrightable, Spotify seems happy to royalties for AI music, provided the works themselves (as opposed to the copying that fed the AI process that created the works) don’t infringe on other artists songs.

AI music may destabilize entrenched business models at the fringes, but it might also foster broader participation and new forms of cultural expression. Does AI pose the same threat to the economic and cultural standing of musicians as it does to stock photography and digital art? Or will AI-generated music remain a hybrid layer within popular culture that feeds off and refers back to mainstream music without replacing the central role of human creation? If so, perhaps at least some country girls will make do.

Posted on

Piracy, Proxies, and Performance: Rethinking Books3’s Reported Gains

A new NBER working paper by Stella Jia and Abhishek Nagaraj makes some stunning claims about the effects of pirated book corpora on large-language-model (LLM) performance. In Cloze Encounters: The Impact of Pirated Data Access on LLM Performance (May 19, 2025) (working paper)( https://www.nber.org/papers/w33598), the authors contend that access to Books3—a pirated collection of  full-text books—raises measured performance by roughly 21–23 percent in some LLMs.  

This astonishing finding is an artifact of the paper’s methodology and the very narrow definition of “performance” that it adopts, as such it should not be taken at face value. 

Cloze Encounters’ methodology and claims

Jia and Nagaraj assemble a 12,916-book evaluation set and apply a “name cloze” task: mask a named entity in a short passage and ask the model to supply it.

For instance, given a sentence like “Because you’re wrong. I don’t care what he thinks. [MASK] pulled his feet up onto the branch” from The Lightning Thief, the model should identify “Grover” as the missing name.

The main results of Cloze Encounters are estimates of “performance” showing large, statistically significant gains for GPT-class models (about 21–23 percent relative to baseline), smaller gains for Claude/Gemini/Llama-70B (about 7–9 percent), and no detectable effect for Llama-8B. The effects are stronger for less-popular book titles, consistent with fewer substitutes (Internet reviews or summaries) in other training data.

This is all well and good, but the way authors explicitly link these findings to current controversies relating to copyright policy, licensing markets, and training-data attribution is troubling.

Cloze Encounters is not measuring “performance” any way that people should care about

The first thing that raised my suspicion about this paper is that I had already seen this exact methodology used as a clever way to illustrate memorization and to show how some books are memorized more than others. See, Kent Chang et al., “Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4” (https://arxiv.org/abs/2305.00118). Cloze Encounters scales and repurposes that approach for a causal analysis of how access to pirated books in the Books3 lead to improved LLM “performance.” But it doesn’t make sense to me that what counted as a memorization probe in one paper could just be relabeled as a general “performance” metric in another.

Why is memorization so different to performance?

This is a question of construct validity. The method in Cloze Encounters tests recall of a masked name from a short passage, scored as a binary hit. This kind of lexical recall is a narrow slice of linguistic ability that is highly sensitive to direct exposure to the source text. It’s a proxy for memorization rather than the broad competencies that make LLMs interesting and useful.

The capabilities that matter in practice—long-context understanding, abstraction and synthesis, factual grounding outside narrative domains, reliable instruction following—are largely orthogonal to masked-name recall. Calling the cloze score “LLM performance” is a massive over-generalization from a task that measures a thin, exposure-sensitive facet of behavior. As an evaluation device, name-cloze is sharp for detecting whether models learned from—or memorized—a specific source; it is blunt for assessing overall performance. There is no reason to think that evidence of snippets of memorization from particular works in the books3 dataset has any necessary relationship with being a better translator, drafter, summarizer, brainstorming partner, etc.

This paper is begging to be misread and misapplied in policy and legal debates

I wouldn’t go so far as to say that success on the cloze score tells us “literally nothing” about LLM performance: “almost nothing” is a fairer estimate. To see why, think about the process of pre-training. Pre-training optimizes next-token prediction over trillions of tokens; the cloze outcome is, by construction, and basically the same as that objective. So it is not surprising that it is unusually sensitive to direct exposure to given pieces of training data. There probably is a broad correlation between next-token  accuracy  and perceived usefulness (we certainly saw this in the transition from GPT-3.5 to GPT-4), but the relationship is not lockstep, and it’s easy to imagine a model that excels  at memorization alone but generalizes poorly.

The authors nod to these limitations at various points in the manuscript but they still frame it as a measure a  of “LLM performance” in a way that is just begging to be misread and misapplied in policy and legal debates. Abstract-level claims travel further than caveats; many readers will see the former and miss the latter.

Nor does the identification strategy employed in the paper do anything to rescue the limits of the construct. The instrumental variable—publication-year share in Books3—may isolate an exogenous shock to exposure. Even granting the exclusion restriction, the estimate remains the effect of Books3 on a name-cloze score. It tells us little about summarization, reasoning, instruction following, safety behavior, or cross-domain generalization.

Bottom line

Cloze Encounters usefully documents that access to Books3 leaves a measurable imprint on exposure-sensitive recall. But its central metric does not justify broad the claims it makes about “LLM performance.” The study measures whether models can fill in masked strings drawn from particular books; it does not show that such access improves the flexible, user-tailored generation that makes these systems valuable.

Posted on

I have moved to Emory University School of Law

Posts on this website are infrequent these days. But I thought it was worth mentioning that I have moved to Atlanta to take a position on the amazing Emory Law faculty. I was hired as a Professor of Law in Artificial Intelligence, Machine Learning, and Data Science as part of Emory’s bold new AI.Humanity initiative.

You can read the Emory announcement here: https://law.emory.edu/news-and-events/releases/2022/04/sag_joins_emory_law.html