Tag: Technology

Posted on

Automated Review of Contracts: Some Thoughts on LawGeex’s AI-Versus-Humans Study

I’m a contract-drafting guy, but I have to acknowledge that drafting contracts might not be the most annoying part of the day-to-day contracts process.

Assume that Acme does ten deals with ten different companies in which it drafts the contracts using its templates. Then assume that it does those deals using the other guy’s drafts. Odds are that in the second scenario Acme would end up doing a lot more work than it would in the first scenario—producing a draft using a template you’re familiar with is likely to take less time than reviewing a draft produced by the other side.

So it’s not surprising that software aimed at reviewing the other side’s drafts should now be attracting attention. The two names I’m familiar with are LegalSifter and LawGeex. They’re welcome innovations.

LawGeex recently disseminated this study that made the following claim:

In a landmark study, 20 experienced US-trained lawyers were pitted against the LawGeex Artificial Intelligence algorithm. The 40-page study details how AI has overtaken top lawyers for the first time in accurately spotting risks in everyday business contracts.

That prompted a bunch of hyperventilating articles, including this one.

I’m sure the reported results reflect what happened. I know something about confidentiality agreements, having spent around a year building an automated one. (It’s described in this LinkedIn article.) So I had a look at the LawGeex study, and it prompted the following thoughts. My intention isn’t to criticize, but to offer some context.

Humans Are Fallible

Yes, humans are fallible. In terms of contract drafting, my presumption is that everything is bad, and I’ve offered on this blog many examples of that. When that’s not the case, I’m pleasantly surprised. So I would expect a comparable dynamic to apply when it comes to review. But the circumstances of LawGeex’s study are a worst-case scenario. If a company is paying attention, it would give those reviewing confidentiality agreements some sort of checklist against which to measure what they’re reviewing. By contrast, those taking part in LawGeex’s test had to rely only in their experience and native wit.

Granularity

Second, the issues flagged by LawGeex are very broad. For example, one issue was presence of a no-soliciting provision. In my automated confidentiality agreement, the no-soliciting provision is customizable up the wazoo, starting with whether it covers just hiring or both hiring and soliciting. Simply flagging no-soliciting provisions doesn’t get one very far.

Necessarily Limited Scope

Third, inevitably, LawGeex’s list of issues wasn’t comprehensive. To select an example at random, it doesn’t include flagging instances of the word proprietary, something I wrote about in this 2010 post. And their list doesn’t cover general drafting issues, such as whether something should be expresses as a condition and not as an obligation, and I doubt it ever will. That’s why review by software should support review by a person, not replace it.

What Comes Next

Fourth, spotting issues is the first part of what the technology does. Then LawGeex suggests edits based on a company’s pre-defined legal policies. I’d be interested to know the level of detail it offers, in terms of both what it reads and the suggestions it offers. But that would require a demo. For purposes of this post, I’m just looking at their study.

Whom Do You Trust?

And fifth, my biggest question about the new crop of “AI” technologies isn’t the technology per se, it’s the humanoid expertise it incorporates. That concern applies to all services that address contract content. I’m toying with the slogan “Editorial expertise is the new black box.” In the case of services that offer contract templates, if I don’t know who prepared a template, I’m not going to trust it. Even if I do know, I’ll be skeptical unless given good reason not to be. Relying on someone’s contract language requires a leap of faith, so I know that I have to not only be an expert but also appear to be an expert. The same goes for services that assist with review.

To its credit, LawGeex identifies the “team of prestigious law professors and veteran lawyers” that prepared the list of issues that forms the basis for the test. But I happened to spot that the heading for one of the identified issues was “Exclusion—Public Domain.” That’s a little worrisome: as I note in this 2010 post, the phrase in the public domain “has no bearing on how widely available any given information is. Instead, it means that the information isn’t protected by intellectual-property rights and so can be used by anyone free of charge. That would represent an irrationally narrow exclusion from the definition of ‘Confidential Information’ ….” So LawGeex’s team flubbed by using that phrase, albeit just in a heading. Might they have missed other stuff? Just as those performing an old-fashioned review are likely to be fallible, the experts giving instructions to AI might be fallible too.

Most people shouldn’t find that sort of problem disconcerting, as most of us would be grateful to have the benefit of the collective expertise of LawGeex’s team, even if they’re fallible.

Other Kinds of Contracts

It’s not surprising that LawGeex’s report features confidentiality agreements. They’re the cockroach of the contract world—ubiquitous, annoying, and apparently indestructible. And you see the same issues in contract after contract. It will be interesting to see how LawGeex and its competitors do when it comes to reviewing more fluid kinds of contracts.

This category of product has the potential to make contract review quicker and more effective. Let’s see whether the technology and the underlying expertise are up to the job. And, to quote this post, let’s see whether the intended users give a ****.

Posted on

The Adams/Cummins Debate on Sweeping Away Traditional Contracting: Ken Says “No”

Tim Cummins is the head of the International Association for Contract and Commercial Management (IACCM). Over the years I’ve discussed with Tim the future of contracting. After a recent exchange of emails, we decided try a more formal approach. We agreed to address on our respective blogs the following proposition: We want new technologies to sweep away traditional contracting, so we can have faster, more efficient, and more cost-effective contracting. In this post on his blog, Tim gives a hearty “Yes” to this proposition; below, I give an equally hearty “No”! We invite you to chime in, whether on Tim’s blog, my blog, LinkedIn, or Twitter.

***

For most companies, the contracts process is a problem. It’s too slow, too expensive, and too often it results in disputes and other unsatisfactory business outcomes.

Tim Cummins and I agree that change is desperately needed, but we have different ideas about what that change should look like. Tim wants cutting-edge technology to sweep away the old order. I want change that’s driven by human expertise, with technology—and relatively unglamorous technology at that—only helping to scale up that expertise. So I say “No” to the proposition under debate.

The Problems

Let’s start by considering the different stages in the contracts process and the problems associated with each.

  • Drafting: Contracts use archaic, bloated, legalistic prose; drafters rely on misbegotten conventional wisdom; deal terms don’t make sense, express inefficient deal mechanics, or are too one-sided; and contracts are cumbersome because drafters are too risk-averse.
  • Reviewing counterparty drafts: It’s time-consuming, and it’s made worse by drafts exhibiting shortcomings listed in “Drafting,” above; and improvised review has the potential for human error.
  • Negotiation (before and after drafting): It’s time-consuming, and it’s made worse by drafts exhibiting shortcomings listed in “Drafting,” above; and lawyers are prone to “negotiation theater”—haggling over legal issues that are peripheral to the deal.
  • Signing: It’s an administrative nuisance.
  • Monitoring performance: It’s an administrative nuisance, and it’s made worse by drafts exhibiting shortcomings listed in “Drafting,” above; and improvised monitoring has the potential for human error.

Not a Technology-Only Solution

It’s clear from this summary that the biggest source of dysfunction in the contracts process is shortcomings in what contracts say and how they say it. And the harsh reality is that because of an insurmountable garbage-in-garbage-out problem, technology can only ever play an ancillary role in fixing that.

The technology that Tim appears most interested in—artificial intelligence, natural language processing, advanced analytics—can tell you how contracts address specific issues, but it can’t tell you what’s clearest and what makes most sense for a given transaction. Good luck drawing any conclusions about the implications of, say, efforts provisions or the phrase represents and warrants from letting technology loose on a bunch of contracts.

The Foundation

Instead, we need to offer a new model, one based on old-fashioned expertise.

The first step toward a rational contracts process is implementing comprehensive guidelines for clear contract prose. My book A Manual of Style for Contract Drafting, now in its fourth edition, is the only work that fits the bill. It’s aimed at an international readership, it has sold tens of thousands of copies, and no one has seriously challenged its authoritativeness. If you make decisions regarding contract language without consulting it, it’s likely that you’re copy-and-pasting, relying on flimsy conventional wisdom, or improvising. That’s unlikely to lead to optimal contract prose.

But a comprehensive set of guidelines for clear contract prose isn’t enough. In a copy-and-paste world, it’s unrealistic to expect those who work with contracts to stop the contracts machinery, dismantle it, retool it, then put it back together. Furthermore, it’s unrealistic to expect countless companies and law firms to each tackle that task and devote to it the necessary resources. And they wouldn’t have access to the necessary expertise—contract drafting is best left to specialists. There are precious few of those, as I explain in this post.

Template Library

But I’m not about to recommend that we train an army of contract-drafting specialists. Instead, what’s required is a public, subscription-based library of templates prepared by contract-drafting specialists who follow a comprehensive set of guidelines and are assisted by subject-matter experts. If automated, the templates would allow a considerable amount of customization. (Technology for automating templates has been around for years, in various forms.) No such library yet exists, at least not one as ambitious as what I have in mind.

Instead of just building templates, parking them online, and waiting for people to use them, it would make sense to work with companies, law firms, and trade groups to identify where there’s a real need for templates that would allow them to quickly create contracts that are clearer, more relevant, and more cost-effective than anything they could build on their own.

Such a library would also make ancillary technology more effective. For example, technology aimed at speeding review of counterparty drafts has to be told what to look for. That information could readily be gleaned from templates in the library.

Prospects for Change

It would be rash to assume that any initiative to improve contracts would bring sweeping change to what is a precedent-driven corner of a notoriously conservative profession. In any process of change, the laggards might well outnumber the adopters. But give people something that makes their lives easier, saves time and money, makes them more competitive, and reduces risk, and there’s a good chance they’ll use it.

Contract managers should consider being more assertive about seeking change this kind of change. I’ve noticed two recurring themes in commentary coming from the contract-management community: first, that lawyers are responsible for screwing up the contracts process, and second, that technology will save us. Both threads are present in Tim’s post in favor of the proposition under debate.

Those themes suggest that the contract-management community has ceded to lawyers primary responsibility for what’s in a contract. That’s unwarranted. Yes, lawyers might have some additional years of education, and they’re notorious for poaching on others’ turf. But nothing about contracts mandates that lawyers call the shots. (See this blog post.) Contract provisions that require lawyer expertise—primarily those addressing dispute resolution—tend to be a peripheral part of the deal. Anyone who is familiar with deal mechanics and is an informed consumer of contract language can call the shots, whether they’re a lawyer or a contract manager.

If the contract-management community would like to take greater responsibility for contract language without succumbing to the dysfunction of traditional contract language, a template library of the sort I have in mind would be the way to go.