Automated Boilerplate: Prevalence and Quality of Contract Generators in the Context of Swiss Privacy Policies
This addresses compliance challenges for small businesses lacking legal resources, though it is incremental in applying LLMs to legal analysis.
The study tackled the problem of firms struggling to comply with digital regulations by evaluating automated contract generators for Swiss privacy policies, finding that generators, used by 18% of websites, increased compliance by up to 15 percentage points. It developed a GPT-5-based method to assess compliance across a multilingual dataset, showing compliance improvements after a 2023 law revision.
It has become increasingly challenging for firms to comply with a plethora of novel digital regulations. This is especially true for smaller businesses that often lack both the resources and know-how to draft complex legal documents. Instead of seeking costly legal advice from attorneys, firms may turn to cheaper alternative legal service providers such as automated contract generators. While these services have a long-standing presence, there is little empirical evidence on their prevalence and output quality. We address this gap in the context of a 2023 Swiss privacy law revision. To enable a systematic evaluation, we create and annotate a multilingual benchmark dataset that captures key compliance obligations under Swiss and EU privacy law. Using this dataset, we validate a novel GPT-5-based method for large-scale compliance assessment of privacy policies, allowing us to measure the impact of the revision. We observe compliance increases indicating an effect of the revision. Generators, explicitly referenced by 18% of local websites, are associated with substantially higher levels of compliance, with increases of up to 15 percentage points compared to privacy policies without generator use. These findings contribute to three debates: the potential of LLMs for cross-lingual legal analysis, the Brussels Effect of EU regulations, and, crucially, the role of automated tools in improving compliance and contractual quality.