Technology can change familiar problems in legally significant ways. A technology can lower the cost of harmful conduct, increase its scale, make previously unavailable inferences possible, alter who makes a decision, obscure how a decision was reached, redistribute costs and benefits, or create capabilities that existing institutions were not designed to handle. Law responds to abstract categories of harm through practical assumptions about how those harms occur, and changes to those assumptions can matter a great deal.
The declining cost of malicious activity
Consider fraud. You might think that using an AI-generated voice to dupe someone out of their money is just a very old crime facilitated by new technology and thus requires no new laws. But if combining deepfakes with language models reduces the marginal cost of fraud to close to zero, we can expect an avalanche of fraud with devastating and corrosive effects. That might well justify new and different regulatory responses, focusing on authentication systems, payment delays, identity verification laws, new investigatory powers, etc. It might not. But simply observing that we have always had fraud doesn’t really address the issue of scale.
Surveillance through data accumulation
Following a person around a city has been possible for as long as cities have existed. But for most of our history, it has also been extravagantly expensive to devote hundreds of man-hours a day to following a single person. Digital surveillance via automated recognition systems or combining information like cell phone location data has changed that equation.
Surveillance tools can be used badly, leading to harassment, mistaken arrests, alienation of entire communities, etc. They can be used by authoritarian regimes to terrorize and control the population. But they can also be used to save lives, preserve the peace, and apprehend criminals. In the United Kingdom, facial recognition is routinely used at football games to scan attendees against a watch list of people wanted for serious offenses and subject to banning orders. If you like to watch football without the company of violent racists, this might be a good thing. But of course, one good use case doesn’t answer the hard questions about the appropriate scope of use and problems of misuse.
Information and inference
Privacy law has traditionally paid attention to the collection and disclosure of information. Machine learning makes inference increasingly important.
In the early 2010s, Target’s statisticians built a system for predicting whether customers were likely to be pregnant based on changes in ordinary purchases. As has been widely reported, this resulted in an irate father objecting to Target sending his teenage daughter the kind of ads one would send to an expectant mother. The father had no idea his daughter was pregnant, but Target’s machine learning algorithm had connected the dots in a way that felt very much like an invasion of privacy. The key thing to understand in this case study is that the retailer did not possess sensitive health information. It inferred that information from a collection of humdrum ordinary transactions.
The ability of various big data techniques to infer things like political views, health conditions, and financial distress, from seemingly benign data has posed a massive challenge to conceptions of privacy and the role of privacy law.
Decision rules, proxies, and bias
Machine learning systems learn from historical data, and historical data records historical inequalities.
The resulting problems are often described simply as bias. That description is accurate but incomplete, because the mechanism is almost always far more subtle than using protected characteristics like race, age, and gender to make important decisions.
A useful example comes from a 2019 study published in Science of a healthcare algorithm used to identify patients for high-risk care management. The researchers found that the system assigned lower risk scores to Black patients than to white patients with comparable health conditions. Black patients therefore had to be sicker to qualify for additional care. The system’s target was future healthcare spending, used as a proxy for healthcare need. Data scientists use proxies they can measure easily to stand in for things that are hard to measure all the time, but in retrospect, this was a bad choice because healthcare spending reflects access to care as well as need for care. If Black patients receive less medical care for the same underlying conditions, they also generate lower costs. A system trained to predict cost can therefore encode unequal access as if it were unequal medical need.
It’s important to understand that this was not an irredeemable problem, and indeed, the developers worked with the researchers who uncovered the issue to correct the system. The broader lesson is that apparently neutral objectives can reproduce inequalities through the choice of proxy. Questions about algorithmic discrimination therefore concern more than biased training data. They include what the system predicts, why that target was selected, what population it was trained on, how errors are distributed, and what happens when its predictions are converted into decisions.
Opacity and explanation
Deep learning systems do not readily produce explanations of their internal operation. Human decision-making is not perfectly transparent either, and a retrospective explanation offered by a person is not necessarily a reliable account of the process that produced the decision. Law nevertheless often requires reasons.
Administrative agencies explain decisions. Employers may need to justify employment practices. Creditors provide reasons for adverse actions. Courts examine whether someone acted reasonably, intentionally, knowingly, in good faith, or for a prohibited reason. AI systems can make those inquiries harder where an important decision depends on patterns distributed across a large model and no person can say why a particular output followed from a particular input.
I don’t use generative AI to grade my students’ exams because “this is the result that minimized a loss function” is not a very satisfying answer to the question, “why did I get a B plus?”
Research on explainable AI and particularly on the interpretability of large language models has made progress since the launch of ChatGPT, but it is far from producing the kind of explanation legal rules often assume is or should be available. This creates two related policy questions: whether important decisions should be made using systems that cannot adequately explain themselves, and what kind of explanation should count when the internal causal story is unavailable.
Manipulation and decisional autonomy
Personalized recommendation, targeted advertising, and curated content all attempt to influence behavior. Digital systems have also long used so-called dark patterns: pre-selected options favorable to a platform, interfaces that make the privacy-protective choice harder, manufactured urgency, or friction placed selectively in the path of an unwanted decision. It seems inevitable that generative AI will make adaptive persuasion far more effective going forward, with particularly dire consequences when applied to the very young, the very old, people suffering mental health conditions, financial distress, or isolation.
Distribution
Some of the most important AI policy questions are distributive.
If AI changes the organization of work, the gains and losses will not necessarily fall on the same people. There are reasons to expect labor-market disruption to be slower than many dramatic predictions suggest. Reorganizing businesses is costly. Regulation, professional norms, collective bargaining, customer preferences, and the physical nature of many jobs all slow substitution. Previous waves of automation generated predictions of mass unemployment that did not materialize in the predicted form. Nonetheless, there are reasons for concern. Dislocation can be concentrated within particular occupations even when aggregate employment remains high, and AI may affect the internal structure of some professions in a distinctive way.
Then there is the broken ladder problem. Many of the onramps to rewarding higher-skilled careers begin with some of the more elementary work that AI is likely to automate. This will be an issue for lawyers as much as graphic designers.
Distribution also matters outside labor markets. A data center may impose grid costs, water demands, land-use consequences, or infrastructure requirements on a particular community, while much of the economic benefit from the computation accrues elsewhere. A facial-recognition system may produce security benefits for one group while imposing the risk of misidentification on another. An AI system may make professional services cheaper overall while reducing the income of people who previously supplied them.
Concentration and infrastructure
Frontier AI development requires substantial capital, specialized hardware, infrastructure, and technical expertise, which currently limits the number of organizations capable of training the largest systems. That concentration can raise conventional competition concerns, restrict access to important inputs, and place consequential decisions about safety, access, and acceptable uses in the hands of a small number of private firms. For how long is unclear.
AI also has a growing physical footprint. Training and running large models consumes substantial electricity, while data centers can require significant water and infrastructure. Aggregate claims about those costs have been wildly exaggerated in the media and in popular books. For individuals, prompting large language models is probably one of the least environmentally impactful things they will do in a day, but that is cold comfort in those places where data centers have significant local impacts.
← Previous: Benefits, Harms, and Uncertainty · Contents · Next: How to Think About Law and AI →