In-house lawyers use legal precedent documents every day and are grateful for the time they save but do we ever stop to think ‘just how good is this document, and how exactly does it help?’ and what test would we use to assess it? Until now, chances are you would rely on lawyer’s intuition and why not, it’s what everyone’s been doing since… forever.
This article is about three propositions. First, it suggests the criteria for a ‘good’ precedent to meet the organisation’s goals while creating efficiencies inside an in-house legal team. Secondly, it makes the case for using techniques that are grounded in science, not intuition, in applying those criteria and thirdly, it describes ADAmetrics, an innovation to assess legal precedents against those criteria.
What are the criteria for a good precedent?
When a lawyer is asked to assess the quality of a contract precedent, what criteria1 would they use? When we place a sample in front of several lawyers there is always a huge variety of responses ‘this is great’, ‘that's not too bad,’ or ‘that’s not very good’ and when asked to explain their reasoning, most struggle. In Engo’s2 experience, lawyers use intuition – not science – and the benchmark for ‘good’ is pretty hazy.
Many lawyers believe the only important criteria is maximum protection, which is sometimes true but rarely the sole organisational objective. My concern3 is that most lawyers, consider ‘good’ to be based upon what they learned from the partner that trained them (and if they're lucky, some of the principles that they retained from study). Thomas Barton4 describes this as an outdated view of contracts, which results in ‘documents written by lawyers for lawyers’. I agree with the challenging notion that more contracts should not be written nor even a document but that’s a bigger idea.
Pulling back from the future into the current land of written contracts, I believe there is an optimum balance5 that delivers ultimate efficiency whilst providing the adequate, not maximum, protection. In my view, a good precedent is unwaveringly focused on helping the organisation achieve its goals. Therefore, a lawyer should look at the way their precedents help (or hinder) the organisation. If the legal team is truly helping then the engrained thinking that protection is all that counts is flawed.
Engo has developed 14 criteria6 that can be roughly summarised into five categories to assess if a precedent is ‘good’:
1. Fair and middle ground from the outset. It is tempting for a lawyer to rate highly the legal protection offered by a precedent. They may say, ‘How strong are the protections in this document? The stronger the better…’. My view is that this can be a false economy when the time and resources dedicated to negotiating a stronger document outweigh the economic benefit of having the protections in place. In many cases, the better solution is to adopt a ‘middle ground’ position7 so that contracts are negotiated less and finalised quickly.
- How would a sales team feel about this? They would love it; happy to trade ‘over-protection’ for market share any day.
- What about Procurement? Surely, they would swap over-protection for thousands of hours saved in negotiating low risk 30-page contracts.
Perhaps we can pause to let that sink in – a ‘strong’ precedent may well be costing more money than it saves...
2. Using plain English – The academic commentary indicates that the more complicated the language, the more time spent by Legal answering questions from internal clients. It is flattering for an in-house lawyer to be asked to explain a long and complicated paragraph, but how does old Legalese help otherwise? The message is clear – use Plain English and make it easy for the parties to understand their positions. ADAmetrics (discussed below) uses several academically produced assessment formulas to calculate plain English usage.
3. Readability – In a similar way, the poor structure of a document (for example, the overuse of cross-referencing and legalese, together with poor layout) inhibits the ability to understand and amend without the use of Legal8. A good example of simplification and readability is BT’s move to use customer-centred design to use visuals to create Public Disclosure Statements that could be more easily understood whilst still complying with ASIC rules.9
4. Easy to use. One of Engo’s Legal Efficiency theories follows that in-house lawyers should prioritise their time on high importance matters (factors of value and risk). Having non-lawyers complete more of the document before legal review pushes the lawyer higher up the value tree. ADAmetrics assesses whether the precedent is easy for the lawyer, and more importantly, a non-lawyer to amend and populate. For example, are the variables hidden away in the body of the document? Does the precedent avoid the horrors of formatting? Is it future-proofed and easily adopted by automation software? Consider the time (and therefore money) that a clumsy document exhausts. ADAmetrics assesses the document on the potential and likelihood of a non-lawyer populating the precedent free of defects.
5. Concise. This one is obvious. Short is sweet. Unnecessary words are unnecessary.10
The case for metrics over intuition
Having established the criteria, how should it be measured? Some might argue there are too many dynamics and subjective issues to consider and conclude that intuition from a carbon-based lifeform is still the best tool. My view, if that is true now it is unlikely to remain the case for long because other disciplines are leading the way in using metrics to replace intuition.
Let’s ask Brad Pitt (excuse the sport reference but Brad + sport = something for everyone). His character in the film, Moneyball, is based on the real-life baseball coach, Billy Beane, who applied a theory called SABERmetrics11 that used strict mathematical criteria instead of intuition to select players.
In a similar way to the lawyers using ‘life-learned intuition’, the traditional selectors in Moneyball were saying, ‘He’s had his problems off the field and he is thick around the waist’. Beane said, ‘well …his percentage is all we’re looking at now’. Despite being ridiculed initially, Beane’s approach was successful and resulted in a re-think of the way to value players. Scientific metrics is often a better technique, even in areas that are steeped in intuition, like lawyering.
Other examples closer to home:
- Paint matching in interior design. Instead of asking a painter to experiment by mixing colours, we can now use spectrophotometers - this happens at Bunnings every day.
- Medicine relies on intuition but is always open to better, more scientific approaches. Traditionally, GPs used intuition to assess the various inputs when diagnosing a patient, but science is closing in, with systems to measure inputs and apply logic. For example, MIT's Computer Science and Artificial Intelligence Lab, has a system that either makes a decision or recognises its own limitations and then turns to a carbon-based lifeform (GP) to confirm the decision. In my view it’s not a big leap to apply this concept to the process that a lawyer follows.12
ADAmetrics Explained
At Engo we set about creating our own scientific methodology for assessing legal precedents that we call ‘ADAmetrics’. The name is a nod to Ada Lovelace, believed by some to be the first to recognise that the machine had applications beyond pure calculation.
ADAmetrics uses an objective, data-driven approach to assess documents, based on principles we believe in. ADA (as the tool is known) takes a document and tests it against 14 criteria that in Engo’s view should be present in best practice contract precedents.
The technique that we have developed involves a lawyer using some elements of technology (ie word count and readability tests) together with some amount of qualitative analysis (eg the ambit of an indemnity) to compare the precedent across the benchmarked 14 data sets. Once a score is generated, amendments can then be made to bring it more in line with ‘good’.
Summary
While there has been some academic focus on the complexity of legislation there is less academic focus on the commercial benefits of practising less complex law. We do not dispute that the human brain trained over five years at university is a handy tool to retain a body of knowledge and apply familiar scenarios relatively quickly. We also acknowledge the immense variables at play. However, if the criteria and the principles are clearly enunciated, then a scientific tool can do that more accurately and much faster without the input of bias (which is a topic for another day). At a minimum, if it achieves nothing else, it at least gives us a better language to assess ‘good’.
Some lawyers might feel a sadness or a fear in all this, perhaps feeling that the humanity, spirit and the poetry of law is somehow reduced as ‘civilisation advances, poetry almost necessarily declines’.13 We politely disagree, feeling that it is instead a cause for celebration, a re-thinking. Useful precedents can free a lawyer from the mundane and help an organisation in fundamental ways. We acknowledge that if the lawyer was given the purpose of the precedent and the principles then they may be able to apply that methodology accurately, but not as fast as a tool. The best combination is not just a lawyer and not a tool but a lawyer using a clever tool.
Jason Ryan
Principal at Engo, specialising in Legal Efficiency
Join Gavin Davis, Principal, Engo, and Jason Ryan, Principal, Engo, at the In-house Legal Virtual Conference for a session on The Smart Set – Learning from the best of best outside of Law
References
Hazard J, Haapio H (2017) Wise contracts: smart contract that work for people and machines. In: Schweighofer E et al (eds) Trends and communities of legal informatics. In: Proceedings of the 20th international legal informatics symposium IRIS 2017. Österreichische Computer Gesellschaft/ books@ocg.at, Wien
T. D. Barton etc al (2019) Successful Contracts: Integrating Design and Technology. In M. Corrales et al. (eds.), Legal Tech, Smart Contracts and Blockchain, Perspectives in Law, Business and Innovation, Springer Nature Singapore Pte Ltd
Louis Kaplow (1995) A Model of Optimal Complexity of Legal Rules, 11 J. L. ECON. & ORG.
Jan M Smits (2015), Do Small Jurisdictions Have a More Complex Law? A Numerical Experiment in Constitutional and Private Law, Maastricht Working Paper No. 2015/05.
Lee N, Barton T-D (2015) Contract flexibility through trademarks: ‘Branded’ intellectual property licensing practices. Lapland Law Rev (2):247–279. http://www.ulapland.fi/InEnglish/Units/Faculty-of-Law/Research/Lapland-…). Accessed 18 June 2018
Passera S, Haapio H, Barton T-D (2013) Innovating contract practices: merging contract design with information design. In: Chittenden J (ed) Proceedings of the 2013 IACCM academic forum. International Association for Contract and Commercial Management, Ridgefield (CT). http://www.iaccm.com/resources/?id=4958. Accessed 4 Jul 2018
Jenkins, J (2011) What constitutes effective disclosure? A case study in designing accessible product information for investors. In Information Design Journal 19(3), 233–248 John Benjamins Publishing Company doi : 10.1075/idj.19.3.04jen
1For an example of criteria for developing Smart Contracts incorporating technology, see Hazard and Haapio (2017)
2A legal efficiency firm, www.engo.co
3Supported by T. D. Barton etc al (2019)
4Ibid
5Kaplow (1995) seeks to analyse a similar balance for legislative complexity which can be problematic for a system of justice.
6Although not related to contracts, Smit (2015) posed useful criteria when analysing the national constitutions of 27 member countries of the European Union.
7Finding the middle ground is no easy feat. Interesting, Barton (2015) poses an idea of community comments, a type of Wikipedia /open source contribution to find the balance.
8There are now empirical quantitative studies suggesting that simplified materials can enhance comprehension and, in particular, creating a more positive user experience – see Passera S, Haapio H, Barton T-D (2013)
9Jenkins, J (2011) where the lawyers role to eliminate word to reduce the document from 40 pages to 20.
10Like this phrase
11https://en.wikipedia.org/wiki/Sabermetrics
12See also See also: https://www.theguardian.com/technology/2019/sep/24/ai-equal-with-human-experts-in-medical-diagnosis-study-finds
13http://www.blackwellpublishing.com/content/BPL_Images/Journal_Samples/YREV0044-0124~89~4~544/545.PDF