Volltext-Downloads (blau) und Frontdoor-Views (grau)

The digital transformation of jurisprudence : an evaluation of ChatGPT-4’s applicability to solve cases in business law

  • In the evolving landscape of legal information systems, ChatGPT-4 and other advanced conversational agents (CAs) offer the potential to disruptively transform the law industry. This study evaluates commercially available CAs within the German legal context, thereby assessing the generalizability of previous U.S.-based findings. Employing a unique corpus of 200 distinct legal tasks, ChatGPT-4 was benchmarked against Google Bard, Google Gemini, and its predecessor, ChatGPT-3.5. Human-expert and automated assessments of 4000 CA-generated responses reveal ChatGPT-4 to be the first CA to surpass the threshold of solving realistic legal tasks and passing a German business law exam. While ChatGPT-4 outperforms ChatGPT-3.5, Google Bard, and Google Gemini in both consistency and quality, the results demonstrate a considerable degree of variability, especially in complex cases with no predefined response options. Based on these findings, legal professionals should manually verify all texts produced by CAs before use. Novices must exercise caution with CA-generated legal advice, given the expertise needed for its assessment.

Download full text files

Export metadata

Additional Services

Search Google Scholar

Statistics

frontdoor_oas
Metadaten
Author of HS ReutlingenSchweitzer, Sascha; Conrads, Markus
URN:urn:nbn:de:bsz:rt2-opus4-50805
DOI:https://doi.org/10.1007/s10506-024-09406-w
ISSN:0924-8463
Erschienen in:Artificial Intelligence and Law
Publisher:Springer
Place of publication:Berlin
Document Type:Journal article
Language:English
Publication year:2024
Page Number:26
DDC classes:340 Recht
Open access?:Ja
Licence (German):License Logo  Creative Commons - CC BY - Namensnennung 4.0 International