Measuring Transparency in Consumer Contracts:The Usefulness of Readability Formulas Empirically Assessed

Article 5 of the Unfair Contract Terms Directive requires terms in consumer contracts to be ‘drafted in plain, intelligible language’. The European Court of Justice has ruled on several occasions that transparency essentially equates to ‘foreseeability’: if a term’s legal consequences can be foreseen by the average consumer, it is transparent. Foreseeability therefore requires, so we hypothesise, that the legal effects of terms are comprehensible. Terms are comprehensible if they are plain and intelligible, which means that they are readable. Courts struggle with this transparency test and some authors call for an empirical approach. We wonder whether empirically tested readability formulas could help legal professionals answer this call, albeit in a more efficient, less time-consuming way. An obvious method to assess foreseeability would consist in the application of a readability formula. In this paper we focus on readability formulas, more specifically on the Flesch readability formula. We also devise our own formula, one that is specifically geared towards legalese. We then test both formulas – the Flesch formula and our own – empirically in order to evaluate their usefulness. Since we wish to address legal effects in a broad sense (without restricting ourselves to financial consequences) we opted to assess the foreseeability of general terms and conditions in gaming contracts. These terms and conditions have recently been criticised for being detrimental to the consumer (in the French Steam-case). Our main findings are threefold. Our first conclusion is that the Flesch readability formula cannot indicate whether terms are foreseeable. Second, our revised formula also failed the test. There is no correlation between readability as assessed by either formula and the actual comprehensibility of the terms. Third, we found that the legal effects of the terms and conditions in gaming contracts we presented to our participants (96) are understood well and are thus foreseeable