About
NLLP Workshop 2024 took place on 16 November 2024, co-located with the EMNLP 2024 conference.
The workshop proceedings are available here.
The recording of the workshop is available here:
Sponsors
Program
All times are in EST time zone
| 09:00 - 10:30 | Session 1 |
| 09:00 - 09:15 | Workshop opening |
| 09:15 - 09:20 | Summarizing Long Regulatory Documents with a Multi-Step Pipeline Mika Sie, Ruby Beek, Michiel Bots, Sjaak Brinkkemper, Albert Gatt |
| 09:20 - 09:25 | Towards an Automated Pointwise Evaluation Metric for Generated Long-Form Legal Summaries Shao Min Tan, Quentin Grail, Lee Quartey |
| 09:25 - 09:30 | Cross Examine: An Ensemble-based approach to leverage Large Language Models for Legal Text Analytics Saurav Chowdhury, Lipika Dey, Suyog Joshi |
| 09:30 - 09:35 | LexSumm and LexT5: Benchmarking and Modeling Legal Summarization Tasks in English Santosh T.Y.S.S, Cornelius Johannes Weiss and Matthias Grabmair |
| 09:35 - 09:40 | Algorithm for Automatic Legislative Text Consolidation Matias Etcheverry, Thibaud Real-del-sarte, Pauline Chavallard |
| 09:40 - 09:50 | Joint Q&A |
| 09:50 - 09:55 | LeGen: Complex Information Extraction from Legal Sentences using Generative Models Chaitra C R, Sankalp Kulkarni, Sai Rama Akash Varma Sagi, Shashank Pandey, Rohit Yalavarthy, Dipanjan Chakraborty, Prajna Devi Upadhyay |
| 09:55 - 10:00 | Information Extraction for Planning Court Cases Drish Mali, Rubash Mali, Claire Barale |
| 10:00 - 10:05 | Automated Anonymization of Parole Hearing Transcripts Abed El Rahman Itani, Wassiliki Siskou, Annette Hautli-Janisz |
| 10:05 - 10:10 | BLT: Can Large Language Models Handle Basic Legal Text? Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme |
| 10:10 - 10:15 | Classify First, and Then Extract: Prompt Chaining Technique for Information Extraction Alice Saebom Kwak, Clayton T. Morrison, Derek Bambauer, Mihai Surdeanu |
| 10:15 - 10:20 | HiCuLR: Hierarchical Curriculum Learning for Rhetorical Role Labeling of Legal Documents Santosh T.Y.S.S, Apolline Isaia, Shiyu Hong, Matthias Grabmair |
| 10:20 - 10:30 | Joint Q&A |
| 10:30 - 11:00 | Break |
| 11:00 - 12:30 | Session 2 |
| 11:00 - 11:05 | Rethinking Legal Judgement Prediction in a Realistic Scenario in the Era of Large Language ModelsShubham Kumar Nigam, Aniket Deroy, Subhankar Maity, Arnab Bhattacharya |
| 11:05 - 11:10 | The CLC-UKET Dataset: Benchmarking Case Outcome Prediction for the UK Employment Tribunal Huiyuan Xie, Felix Steffek, Joana Ribeiro de Faria, Christine Carter, Jonathan Rutherford |
| 11:10 - 11:15 |
Transductive Legal Judgment Prediction Combining BERT Embeddings with Delaunay-Based GNNs Hugo Attali, Nadi Tomeh |
| 11:15 - 11:20 |
Comparative Study of Explainability Methods for Legal Outcome Prediction
Ieva Raminta Staliunaite, Josef Valvoda, Ken Satoh |
| 11:20 - 11:25 |
Incorporating Precedents for Legal Judgement Prediction on European Court of Human Rights Cases
Santosh T.Y.S.S, Mohamed Hesham Elganayni, Stanisław Sójka, Matthias Grabmair |
| 11:25 - 11:30 |
The Craft of Selective Prediction: Towards Reliable Case Outcome Classification - An Empirical Study on European Court of Human Rights Cases
Santosh T.Y.S.S, Irtiza Chowdhury, Shanshan Xu, Matthias Grabmair |
| 11:30 - 11:45 | Joint Q&A |
| 11:45 - 11:50 |
Quebec Automobile Insurance Question-Answering With Retrieval-Augmented Generation
David Beauchemin, Richard Khoury, Zachary Gagnon |
| 11:50 - 11:55 |
Attributed Question Answering for Preconditions in the Dutch Law
Felicia Redelaar, Romy van Drie, Suzan Verberne, Maaike de Boer |
| 11:55 - 12:00 |
Measuring the Groundedness of Legal Question-Answering Systems
Dietrich Trautmann, Natalia Ostapuk, Quentin Grail, Adrian Alan Pol, Guglielmo Bonifazi, Shang Gao, Martin Gajek |
| 12:00 - 12:10 | Joint Q&A |
| 12:10 - 14:00 | Lunch & In-Person Poster Session (Lunch provided) |
| 14:00 - 15:30 | Session 3 |
| 14:00 - 15:00 | Keynote Talk Omri Ben-Shahar - Privacy Protection, At What Cost? |
| 15:00 - 15:15 |
Shared Task: Enhancing Legal Violation Identification with LLMs and Deep Learning Techniques: Achievements in the LegalLens 2024 Competition Ben hagag, Gil Gil Semo, Dor Bernsohn, liav harpaz, Pashootan Vaezipoor, Rohit Saha, Kyryl Truskovskyi, Gerasimos Spanakis |
| 15:15 - 15:30 |
Shared Task Winner presentation |
| 15:30 - 16:00 | Break |
| 16:00 - 17:30 | Session 4 |
| 16:00 - 16:05 | LLMs to the Rescue: Explaining DSA Statements of Reason with Platform's Terms of Services Marco Aspromonte, Andrea Filippo Ferraris, Federico Galli, Giuseppe Contissa |
| 16:05 - 16:10 | Enhancing Contract Negotiations with LLM-Based Legal Document Comparison Savinay Narendra, Kaushal Shetty, Adwait Ratnaparkhi |
| 16:10 - 16:15 | Multi-Property Multi-Label Documents Metadata Recommendation based on Encoder Embeddings Nasredine Cheniki, Vidas Daudaravicius, Abdelfettah Feliachi, Didier Hardy, Marc Wilhelm Küster |
| 16:15 - 16:20 | CLERC: A Dataset for U. S. Legal Case Retrieval and Retrieval-Augmented Analysis Generation Abe Bohan Hou, Orion Weller, Guanghui Qin, Eugene Yang, Dawn Lawrie, Nils Holzenberger, Andrew Blair-Stanek, Benjamin Van Durme |
| 16:20 - 16:25 | Empowering Air Travelers: A Chatbot for Canadian Air Passenger Rights Maksym Taranukhin, Sahithya Ravi, Gabor Lukacs, Evangelos Milios, Vered Shwartz |
| 16:25 - 16:30 | The impact of formulaic language in the Court of Justice of the European Union on the performance of lexical and dense retrieval methods Larissa Mori, Carlos Sousa de Oliveira, Yuehwern Yih, Mario Ventresca |
| 16:30 - 16:40 | Joint Q&A |
| 16:40 - 16:45 | Gaps or Hallucinations? Scrutinizing Machine-Generated Legal Analysis for Fine-grained Text Evaluations Abe Bohan Hou, William Jurayj, Nils Holzenberger, Andrew Blair-Stanek, Benjamin Van Durme |
| 16:45 - 16:50 | How Many Van Goghs Does It Take to Van Gogh? Finding the Imitation Threshold Sahil Verma, Royi Rassin, Arnav Mohanty Das, Gantavya Bhatt, Preethi Seshadri, Chirag Shah, Jeff Bilmes, Hannaneh Hajishirzi and Yanai Elazar |
| 16:50 - 16:55 | Towards Supporting Legal Argumentation with NLP: Is More Data Really All You Need? Santosh T.Y.S.S, Kevin Ashley, Katie Atkinson and Matthias Grabmair |
| 16:55 - 17:00 | Misinformation with Legal Consequences (MisLC): A New Task Towards Harnessing Societal Harm of Misinformation Chu Fei Luo, Radin Shayanfar, Rohan V Bhambhoria, Samuel Dahan, Xiaodan Zhu |
| 17:00 - 17:10 | Joint Q&A |
| 17:10 - 17:15 | LAR-ECHR: A New Legal Argument Reasoning Task and Dataset for Cases of the European Court of Human Rights Odysseas S. Chlapanis, Dimitris Galanis, Ion Androutsopoulos |
| 17:15 - 17:20 | Developing a Pragmatic Benchmark for Assessing Korean Legal Language Understanding in Large Language Models Kimyeeun, Choi Youngrok, Eunkyung Choi, JinHwan Choi, Hai Jin Park, Wonseok Hwang |
| 17:20 - 17:25 | Enhancing Legal Expertise in Large Language Models through Composite Model Integration: The Development and Evaluation of Law-Neo Zhihao Liu, Yanzhen Zhu, Mengyuan Lu |
| 17:25 - 17:30 | Joint Q&A |
| 17:30 - 17:40 | Best Presentation Award |
Committees
Organizing Committee
- Nikolaos Aletras - University of Sheffield (UK)
- Leslie Barrett - Bloomberg Law (US)
- Ilias Chalkidis - University of Copenhagen (Denmark)
- Catalina Goanta - Utrecht University (The Netherlands)
- Daniel Preotiuc-Pietro - Bloomberg (US)
- Gerasimos (Jerry) Spanakis - Maastricht University (The Netherlands)
Program Committee
- Sallam Abualhaisa - University of Luxembourg (Luxembourg)
- Tomaso Agnoloni - Institute of Legal Information Theory and Technologies (Italy)
- Ion Androutsopoulos - Athens University of Economics and Business (Greece)
- Elliott Ash - ETH Zurich (Switzerland)
- Tom Ault - Bloomberg Law (US)
- Breck Baldwin - Columbia University (US)
- Claire Barale - University of Edinburgh (UK)
- Thales Bertaglia - Utrecht University (The Netherlands)
- Andrew Blair-Stanek - Johns Hopkins University (US)
- Stephanie Brandl - University of Copenhagen (Denmark)
- Rajarathnam Chandramouli - Stevens Institute of Technology (US)
- Jiahong Chen - University of Sheffield (UK)
- Odysseas Spyridon Chlapanis - Athens University of Business and Economics (Greece)
- Ashish Chouhan - Heidelberg University (Germany)
- Marina Danilevsky - IBM Research (US)
- Tony Davis - Southern Oregon University (US)
- Stefania Degaetano-Ortlieb - Saarland University (Germany)
- Arthur Dyevre - KU Leuven (Belgium)
- Nicolas Garneau - University of Copenhagen (Denmark)
- Piyush Ghai - Relativity (US)
- Kimani Goddard - Maastricht University (The Netherlands)
- Ben Hagag - Darrow (Israel)
- Nils Holzenberger - Télécom Paris (FR)
- Karthik Kannapur - Amazon Web Services (US)
- Daniel Katz - Illinois Tech - Chicago Kent College of Law (US)
- Ken Kawamura - Revelata, Inc. (US)
- Aykut Koç - Bilkent University (Turkey)
- Manolis Koubarakis - University of Athens (Greece)
- Ruta Liepina - University of Bologna (Italy)
- Pauline McBride - University of Glasgow (UK)
- Adam Meyers - New York University (US)
- Jelena Mitrović - University of Passau (Germany) & Institute for AI R&D of Serbia (Serbia)
- Maria Navas-Loro - Universidad Politécnica de Madrid (Spain)
- Joel Niklaus - University of Bern (Switzerland)
- Milda Norkute - Thomson Reuters Labs (Switzerland)
- Kartik Palani - iManage LLC (US)
- Henrik Palmer Olsen - University of Copenhagen (Denmark)
- Paulo Quaresma - University of Evora (Portugal)
- Georg Rehm - DFKI (Germany)
- T.Y.S.S. Santosh - Technical University of Munich (Germany)
- Vageesh Saxena - Maastricht University (The Netherlands)
- Madhavan Seshadri - Bloomberg Law (US)
- Dan Simonson - BlackBoiler LLC (US)
- Jerrold Soh - Singapore Management University (Singapore)
- Alexandru Sotropa - Maastricht University (The Netherlands)
- Andrea Tagarelli - University of Calabria (Italy)
- Dimitrios Tsarapatsanis - University of York (UK)
- Gijs Van Dijck - Maastricht University (The Netherlands)
- Josef Valvoda - University of Cambridge (UK)
- Jacob van de Kerkhof - Utrecht University (The Netherlands)
- Hellen van der Kroef - Maastricht University (The Netherlands)
- Jianqiang Wang - SUNY Buffalo (US)
- Hannes Westermann - Maastricht University (The Netherlands)
- Adam Wyner - Swansea University (UK)
- Marcos Zampieri - Rochester Institute of Technology (US)
- Frederike Zufall - Max Planck Institute for Research on Collective Goods (Germany)
Speaker
Omri Ben-Shahar (University of Chicago Law School)
Title: Privacy Protection, At What Cost?
Abstract: Data privacy protection is the dominant paradigm in the regulation of the digital economy. In this keynote lecture, University of Chicago Law Professor Omri Ben-Shahar challenges the adequacy of the data privacy paradigm. He argues that it fails to capture the most worrisome harms—what he calls 'data pollution'—which are inflicted against public rather than private interests. He further demonstrates that privacy-fueled restrictions on valuable data technologies impose a large, mostly unrecognized, harms on society.
Bio: Omri Ben-Shahar earned his PhD in Economics and SJD from Harvard in 1995 and his BA and LLB from the Hebrew University in 1990. Before coming to Chicago, he was the Kirkland & Ellis Professor of Law and Economics at the University of Michigan. Prior to that, he taught at Tel-Aviv University, was a member of Israel's Antitrust Court, and clerked at the Supreme Court of Israel. He teaches contracts, sales, trademark law, insurance law, consumer law, sales law, e-commerce, food law, law and economics, and game theory and the law. He writes primarily in the fields of contract law and consumer protection. He is the co-author of Personalized Law: Different Rules for Different People (Oxford 2021, with Ariel Porat) and More Than You Wanted to Know: The Failure of Mandated Disclosure (Princeton 2014, with Carl Schneider). Professor Ben-Shahar is the Kearney Director of the Coase-Sandor Institute for Law and Economics. He is also the Co-Reporter for the American Law Institute’s Restatement of Consumer Contracts.