LLM-Assisted Legal Propositions Identification from Party Arguments in the U.S. Supreme Court Briefs

Authors

• Heng Zheng
• Alex Zhang, Duke Law SchoolFollow

Document Type

Chapter of Book

Publication Date

2026

Keywords

U.S. Supreme Court merits briefs, human-AI collaboration, argument mining, legal propositions, large language models

Abstract

Merits briefs are central to U.S. litigation, serving as the primary means for parties to present arguments and persuade judges. Legal propositions in these merits briefs are the atomic units of arguments, whose relationships evolve throughout litigation and inform court decisions and precedent. Large language models (LLMs) have been applied to legal document review, but there is limited evidence on their ability to identify legal propositions in merits briefs. Given the labor-intensive nature of the task, we evaluate a human-AI collaborative approach to identifying legal propositions in the U.S. Supreme Court merits briefs, in which legal annotators review and revise LLM-generated propositions rather than identify them from scratch. We focus on cases involving First Amendment issues. We evaluate our approach across three LLMs by measuring recall, editing burden, and time efficiency relative to manual identification. GPT-4.1 mini achieves the highest recall, recognizing 65.76% of expert-identified legal propositions, though performance varies across merits briefs, indicating limited stability and a real need for human intervention. Nevertheless, acceptance rates for LLM-generated propositions are consistently high across annotators and models, ranging from 75.00% to 86.67%, suggesting that the approach can reduce the burden of identifying legal propositions from scratch. Our approach combines LLM-based identification of legal propositions with human oversight to ensure output quality while reducing annotation effort. Future work will focus on improving recall and assessing the generalizability of our approach.

Citation

Heng Zheng & Alex Zhang, LLM-Assisted Legal Propositions Identification from Party Arguments in the U.S. Supreme Court Briefs, in Computational Models of Argument: Proceedings of COMMA 2026 209-220 (Katie Atkinson et al. eds., 2026)

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License