Lawyers’ Job Security in a Near Future World of AI, the Law’s “Reasonable Man Myth” and “Bagley Two” – Part One

January 15, 2017

bad-robotDoes the inevitable triumph of AI robots over human reason and logic mean that the legal profession is doomed? Will Watson be the next generation’s lawyer of choice? I do no think so and have written many articles on why, including two last year: Scientific Proof of Law’s Overreliance On Reason: The “Reasonable Man” is Dead and the Holistic Lawyer is Born; and The Law’s “Reasonable Man,” Judge Haight, Love, Truth, Justice, “Go Fish” and Why the Legal Profession Is Not Doomed to be Replaced by Robots. In the Reasonable Man article I discussed how reasonability is the basis of the law, but that it is not objective. It depends on many subjective factors, on psychology. In the Scientific Proof article I continued the argument and argued:

The Law’s Reasonable Man is a fiction. He or she does not exist. Never has, never will. All humans, including us lawyers, are much more complex than that. We need to recognize this. We need to replace the Law’s reliance on reason alone with a more realistic multidimensional holistic approach.

Scientific Proof Article

brain_gears_NOTo help make my argument in the Scientific Proof article I relied on the analysis of Thomas H. Davenport and Julia Kirby in Only Humans Need Apply: Winners and Losers in the Age of Smart Machines (Harper 2016) and on the scientific work of Dan Ariely, a Professor of Psychology and Behavioral Economics at Duke University.

I cite to Only Humans Need Apply: Winners and Losers in the Age of Smart Machines to support my thesis:

Although most lawyers in the profession do not know it yet, the non-reasoning aspects of the Law are its most important parts. The reasoning aspects of legal work can be augmented. That is certain. So will other aspects, like reading comprehension. But the other aspects of our work, the aspects that require more than mere reason, are what makes the Law a human profession. These job functions will survive the surge of AI.

If you want to remain a winner in future Law, grow these aspects. Only losers will hold fast to reason. Letting go of the grip of the Reasonable Man, by which many lawyers are now strangled, will make you a better lawyer and, at the same time, improve your job security.

Also see Dean Gonsowski, A Clear View or a Short Distance? AI and the Legal Industry; and, Gonsowski, A Changing World: Ralph Losey on “Stepping In” for e-Discovery, (Relativity Blog).

Professor Ariely has found from many experiments that We’re All Predictably Irrational. In my article, Scientific ProofI point my readers to his many easily accessible video talks on the subject. I consider the implication of Professor Ariely’s research on the law:

Our legal house needs a new and better foundation than reason. We must follow the physicists of a century ago. We must transcend Newtonian causality and embrace the more complex, more profound truth that science has revealed. The Reasonable Man is a myth that has outlived its usefulness. We need to accept the evidence, and move on. We need to develop new theories and propositions of law that confirm to the new facts at hand. Reason is just one part of who we are. There is much more to us then that: emotion, empathy, creativity, aesthetics, intuition, love, strength, courage, imagination, determination – to name just a few of our many qualities. These things are what make us uniquely human; they are what separate us from AI. Logic and reason may end up being the least of our abilities, although they are still qualities that I personally cherish. …

Davinci_whole_manSince human reason is now known to be so unreliable, and is only a contributing factor to our decisions, on what should we base our legal jurisprudence? I believe that the Reasonable Man, now that he is known to be an impossible dream, should be replaced by the Whole Man. Our jurisprudence should be based on the reality that we are not robots, not mere thinking machines. We have many other faculties and capabilities beyond just logic and reason. We are more than math. We are living beings. Reason is just one of our many abilities.

So I propose a new, holistic model for the law. It would still include reason, but add our other faculties. It would incorporate our total self, all human attributes. We would include more than logic and reason to judge whether behavior is acceptable or not, to consider whether a resolution of a dispute is fair or not. Equity would regain equal importance.

A new schemata for a holistic jurisprudence would thus include not just human logic, but also human emotions, our feelings of fairness, our intuitions of what is right and just, and multiple environmental and perceptual factors. I suggest a new model start simple and use a four-fold structure like this, and please note I keep Reason on top, as I still strongly believe in its importance to the Law.

4-levels-Holistic_Law_pyramid

My Scientific Proof article included a call to action, the response to which has been positive:

The legal profession needs to take action now to reduce our over-reliance on the Myth of the Reasonable Man. We should put the foundations of our legal system on something else, something more solid, more real than that. We need to put our house in order before it starts collapsing around us. That is the reasonable thing to do, but for that very reason we will not start to do it until we have better motivation than that. You cannot get people to act on reason alone, even lawyers. So let us engage the other more powerful motivators, including the emotions of fear and greed. For if we do not evolve our work to focus on far more than reason, then we will surely be replaced.

cyborg-lawyer

AI can think better and faster, and ultimately at a far lower cost. But can AI reassure a client? Can it tell what a client really wants and needs. Can AI think out of the box to come up with new, creative solutions. Can AI sense what is fair? Beyond application of the rules, can it attain the wisdom of justice. Does it know when rules should be bent and how far? Does it know, like any experienced judge knows, when rules should be broken entirely to attain a just result? Doubtful.

I go on to make some specific suggestions, just to start the dialogue, and then closed with the following:

We must move away from over-reliance on reason alone. Our enlightened self-interest in continued employment in the rapidly advancing world of AI demands this. So too does our quest to improve our system of justice, to keep it current with the rapid changes in society.

Where we must still rely on reason, we should at the same time realize its limitations. We should look for new technology based methods to impose more checks and balances on reason than we already have. We should create new systems that will detect and correct the inevitable errors in reason that all humans make – lawyers, judges and witnesses alike. Bias and prejudice must be overcome in all areas of life, but especially in the justice system.

Computers, especially AI, should be able to help with this and also make the whole process more efficient. We need to start focusing on this, to make it a priority. It demands more than talk and thinking. It demands action. We cannot just think our way out of a prison of thought. We need to use all of our faculties, especially our imagination, creativity, intuition, empathy and good faith.

Reasonable Man Article

Reasonable_man_cloudTo help make my argument in the earlier blog, The Law’s “Reasonable Man,” Judge Haight, Love, Truth, Justice, “Go Fish” and Why the Legal Profession Is Not Doomed to be Replaced by Robots, I quoted extensively from an Order Denying Defendant’s Motion for Protective Order. The order arose out of a routine employment discrimination case. Bagely v. Yale, Civil Action No. 3:13-CV-1890 (CSH) (Doc. 108) (order dated April 27, 2015). The Order examined the “reasonability” of ESI accessibility under Rule 26(b)(2)(B) and the “reasonable” efforts requirements under Rule 26(b). I used language of that Bagley Order to help support my argument that there is far more to The Law than mere reason and logic. I also argued that this is a very good thing, for otherwise lawyers could easily be replaced by robots.

Another e-discovery order was entered in Bagley on December 22, 2016. Ruling On Plaintiff’s Motion To Compel. Bagely v. Yale, Civil Action No. 3:13-CV-1890 (CSH). Bagley Two again provokes me to write on this key topic. This second order, like the first, was written by Senior District Judge Charles S. Haight, Jr.. The eighty-six year old Judge Haight is becoming one of my favorite legal scholars because of his excellent analysis and his witty, fairly transparent writing style. This double Yale graduate has a way with words, especially when issuing rulings adverse to his alma mater. He is also one of the few judges that I have been unable to locate an online photo of, so use your imagination, which, by the way, is another powerful tool that separates us from AI juiced robots.

Lady JusticeI pointed out in the Reasonable Man article, and it bears repetition, that I am no enemy of reason and rationality. It is a powerful tool in legal practice, but it is hardly our only tool. It is one of many. The “Reasonable Man” is one of the most important ideas of Law, symbolized by the balance scales, but it is not the only idea. In fact, it is not even the most important idea for the Law. That honor goes to Justice. Lady Justice holding the scales of reason is the symbol of the Law, not the scales alone. She is usually depicted with a blindfold on, symbolizing the impartiality of justice, not dependent on the social status or position of the litigants.

My view is that lawyer reasoning should continue in all future law, but should augmented by artificial intelligence. With machines helping to rid us of hidden biases in all human reason, and making that part of our evaluation easier and more accurate, we are free to put more emphasis on our other lawyer skills, on the other factors that go into our evaluation of the case. These include our empathy, intuition, emotional intelligence, feelings, humor, perception (including lie detection), imagination, inventiveness and sense of fairness and justice. Reason is only one of many human capacities involved in legal decision making.

In Reasonable Man article I analyzed the first Bagley Order to help prove that point:

Bagley shows that the dividing line between what is reasonable and thus acceptable efforts, and what is not, can often be difficult to determine. It depends on a careful evaluation of the facts, to be sure, but this evaluation in turn depends on many subjective factors, including whether one side or another was trying to cooperate. These factors include all kinds of prevailing social norms, not just cooperativeness. It also includes personal values, prejudices, education, intelligence, and even how the mind itself works, the hidden psychological influences. They all influence a judge’s evaluation in any particular case as to which side of the acceptable behavior line a particular course of conduct falls.

In close questions the subjectivity inherent in determinations of reasonability is obvious. This is especially true for the attorneys involved, the ones paid to be independent analysts and objective advisors. People can, and often do, disagree on what is reasonable and what is not. They disagree on what is negligent and what is not. On what is acceptable and what is not.

All trial lawyers know that certain tricks of argument and appeals to emotion can have a profound effect on a judge’s resolution of these supposedly reason-based disagreements. They can have an even more profound affect on a jury’s decision. (That is the primary reason that there are so many rules on what can and cannot be said to a jury.)

lady_justice_not_blindIn spite of practical knowledge by the experienced, the myth continues in our profession that reasonability exists in some sort of objective, platonic plane of ideas, above all subjective influences. The just decision can be reached by deep, impartial reasoning. It is an article of faith in the legal profession, even though experienced trial lawyers and judges know that it is total nonsense, or nearly so. They know full well the importance of psychology and social norms. They know the impact of cognitive biases of all kinds, including, for example, hindsight biasSee Roitblat, The Schlemiel and the Schlimazel and the Psychology of Reasonableness (Jan. 10, 2014, LTN) (link is to republication by a vendor without attribution) (“tendency to see events that have already occurred as being more predictable than they were before they actually took place“); Also see Rimkus v Cammarata, 688 F. Supp. 2d 598 (S.D. Tex. 2010) (J. Rosenthal) (“It can be difficult to draw bright-line distinctions between acceptable and unacceptable conduct in preserving information and in conducting discovery, either prospectively or with the benefit (and distortion) of hindsight.” emphasis added); Pension Committee of the University of Montreal Pension Plan, et al. v. Banc of America Securities, LLC, et al., 685 F. Supp. 2d 456 (S.D.N.Y. Jan. 15, 2010 as amended May 28, 2010) at pgs. 463-464 (J. Scheindlin) (‘That is a judgment call that must be made by a court reviewing the conduct through the backward lens known as hindsight.” emphasis added).

In my conclusion to Reasonable Man article I summarized my thoughts and tried to kick off further discussion of this topic:

The myth of objectivity and the “Reasonable Man” in the law should be exposed. Many naive people still put all of their faith in legal rules and the operation of objective, unemotional logic. The system does not really work that way. Outsiders trying to automate the law are misguided. The Law is far more than logic and reason. It is more than the facts, the surrounding circumstances. It is more than evidence. It is about people and by people. It is about emotion and empathy too. It is about fairness and equity. It’s prime directive is justice, not reason.

That is the key reason why AI cannot automate law, nor legal decision making. Judge Charles (“Terry”) Haight could be augmented and enhanced by smart machines, by AI, but never replaced. The role of AI in the Law is to improve our reasoning, minimize our schlemiel biases. But the robots will never replace lawyers and judges. In spite of the myth of the Reasonable Man, there is far more to law then reason and facts. I for one am glad about that. If it were otherwise the legal profession would be doomed to be replaced by robots.

Bagley Two

Now let us see how Judge Haight once again helps prove the Reasonable Man points by his opinion in Bagley Two. Ruling On Plaintiff’s Motion To Compel (December 22, 2016), Bagely v. Yale, Civil Action No. 3:13-CV-1890 (CSH). In this opinion the reasonability of defendant Yale’s preservation efforts was considered in the context of a motion to compel discovery. His order again reveals the complexity and inherent subjectivity of all human reason. It shows that there are always multiple factors at work in any judge’s decision beyond just thought and reason, including an instinct born out of long experience for fairness and justice. Once again I will rely primarily on Judge Haight’s own words. I do so because I like the way he writes and because you need to read his original words to appreciate what I am talking about. But first, let me set the stage.

Reasonable_guageYale sent written preservation notices to sixty-five different people, which I know from thousands of matters is a very large number of custodians to put on hold in a single-plaintiff discrimination case. But Yale did so in stages, starting on March 1, 2013 and ending on August 7, 2014. Eight different times over this period they kept adding people to their hold list. The notices were sent by Jonathan Clune, a senior associate general counsel of Yale University. The plaintiff argued that they were too late in adding some of the custodians and otherwise attacked the reasonability of Yale’s efforts.

The plaintiff was not seeking sanctions yet for the suspected unreasonable efforts, they were seeking discovery from Yale as to details of these efforts. Specifically they sought production of: (1) the actual litigation hold notices; (2) the completed document preservation computer survey forms that were required to be returned to the Office of General Counsel by each Litigation Hold Recipient; and, (3) an affidavit detailing the retention and production for all non-ESI documents collected from each of the Litigation hold Recipients.

Yale opposed this discovery claiming any more information as to its preservation efforts was protected from discovery under the attorney-client privilege and attorney work product protection.  Yale also argued that even if the privileges did not apply here, the discovery should still be denied because to obtain such information a party must first provide convincing proof that spoliation in fact occurred. Yale asserted that the plaintiff failed to provide sufficient proof, or even any proof, that spoliation had in fact occurred.

Here is the start of Judge Haight’s evaluation of the respective positions:

Mr. Clune’s litigation hold notices stressed that a recipient’s failure to preserve pertinent documents could “lead to legal sanctions” against Yale. Clune was concerned about a possible sanction against Yale for spoliation of evidence. While Clune’s notices did not use the term, “spoliation” is a cardinal litigation vice, known by that name to trial lawyers and judges, perhaps unfamiliar to academics unable to claim either of those distinctions. Clune’s notices made manifest his concern that a trial court might sanction Yale for spoliation of evidence relevant to the University SOM’s decision not to reappoint Bagley to its faculty.

skull_bones_yaleNote the jab at academics. By the way, in my experience his observation is correct about the cluelessness of most law professors when it comes to e-discovery. But why does Judge Haight take the time here to point that out? This case did not involve the Law School. It involved the business school professors and staff (as you would expect). It is important to know that Judge Haight is a double Yale graduate, both undergraduate and law school. He graduated from Yale Law in 1955. He was even a member of Yale’s infamous of Skull and Bones society. (What does 322 really mean? Eulogia?) Perhaps there are some underlying emotions here? Judge Haight does seem to enjoy poking Yale, but he may do that in all his cases with Yale out of an eccentric kind of good humor, like a friendly shoulder punch. But I doubt it.

To be continued … 


Five Tips To Avoid Mistakes In Electronic Document Review

January 9, 2017

5-Tips_ReviewThese tips are based on a long life of litigation legal practice, including thousands of document reviews going back to 1978. I have seen hundreds of mistakes over the years, especially in the last decade when my work as a lawyer has been limited to electronic discovery. Many of these blunders were made by “the other side.” Some were funny and made me smile, others were not and led to motions of all kinds. Keeping it real, I have made my own fair share of errors too. Those lessons were painful, but are now deeply etched. No doubt I would have made many more errors, but for the generous guidance provided by more senior and experienced attorneys that I have had the very good fortune to work with. It is with this great debt in mind that I offer up these tips.

Click here to download a Word version. [An earlier version of this article was published last year.)

Some Mistakes are Funny

Gloat_SimpsonsOn the funny side of observing document review mistakes, I will never forget the time, not too long ago, where the other side produced documents to us with the most important ones placed together up front. That was a surprising electronic zipped production to open. It was fairly obvious what had happened. The highly relevant documents were not mixed-in as they should have been with the other more plebeian merely relevant documents. Instead, the hot documents were all together at the front of the production with the lowest numbers. (Sixth tip – never do that!) Our team laughed at the error, as we easily and quickly found lots of great stuff to help our case. Still, we kept a discrete silence and did not gloat. (Seventh tip – Never do that either, at least not in front of them!)

Opposing counsel, who later became a friend, admitted the error to me months after the case settled. He found out what happened a few days too late. Even he chuckled as to how inadvertently “nice” they were. As is often the case, the mistake did not really matter in the end. We would have recognized the hot documents anyway. As usual when errors happen in e-discovery, he blamed the vendor. They almost always get blamed for mistakes, but, the truth is, vendors are just tools of the attorneys (no offense dear vendors, tools are important). The attorneys are almost always the ones ultimately responsible for screw-ups.

Lessons of History

Clarence Darrow and William Jennings BryanThe five tips shared here are born out of a long history of document review. How relevant could past legal practice be you might ask? In 1980, just like today, document discovery was and still is a search for documents and communications that have probative value. The tools and document forms have changed, but not the basic tasks. The federal rules have changed too, but not that much, and the ethics of an attorney controlled discovery system, not at all.

Discovery has always been a search to determine what really happened, to sort out the conflicting stories and find evidence for use at trial. Legal counsel never creates facts. That is called falsification of evidence and is a crime. Attorneys just find the facts and then do the best they can with them; make them look as good as possible by legal argument and clever presentation. The discovery effort has always been a fairly cooperative one between attorneys. It has always been a question of trust but verify. Conversely, there have always been a few slime balls in the Bar who do not get that, but that is what judges (and Bar ethics committees) are for, and they soon sniff out the weasels. All things evolve and change, but some basic patterns remain the same.

By the early nineties I sometimes had to look beyond paper files and investigate computers for possible evidence. That occasionally happened in trade-secret cases, much like today. Forensics was fairly easy back then. My favorite ESI search and review tool was Norton Utilities, which I had been using since the mid-eighties. Like most computer lawyers around those days, as we were called, I was by necessity a DOS master, and, until around 1997, a one man IT department for my law firm. It only took a few hours a week to do that for my then twenty person law firm, along with the help of an outside “computer repairman.” I would always learn a lot from those guys.

DOS_Screen

compuserve_FTPThe frequency of document reviews that included computer files increased somewhat in the early nineties as law firm clients began using more technologies. By then most corporations and many individuals began to rely on computers for work, although almost nobody but a few techno-nerds used email, electronic messaging and pre-Internet online communities. (I was considered an odd-ball hobbyist for using electronic messaging with CompuServeThe SourceThe Well, etc. in the mid to late eighties, and the Internet since 93-94 with Mosaic, then NetScape.) Instead, facsimile machines were the rage at that time, and they just generated more paper discovery.

Although the presence, or not, of computer files was a discovery issue in trade-secret and non-compete cases in the early 90s, electronic communications discovery was still not a factor. The adoption of tech by businesses and lawyers seemed slow to me then, and still seems slow today. (When will companies and law firms adopt the AI technologies that have been readily available for years now?)

Discovery of computer files, as e-discovery was then called, started to take off in the late nighties as corporate email finally became popular. It was part of the public’s discovery on the Internet. I had the opportunity back in 1996 to write a chapter on Internet law for the then popular book by Macmillan (Que), Special Edition, Using the Internet (3rd Ed. 1996), which is incredibly still sold on Amazon.

Using_Internet_96

My chapter in the book was the first after the introduction and was titled by the editors “Your Cyber Rights and Responsibilities: Law and Etiquette.” I still smile when I see how they tasked me not only with explaining all of the Law of the Internet, but also proper online etiquette. I tried to address the legal issues in 52 pages (I pretty much ignored the etiquette part), including discussion of all of the key cases of the day. I covered things like free speech, online agreements, privacy rights, crime, security and cryptology (I even included a coded message, which surprisingly, the editor decrypted and then made me clean up). These are all still hot issues.

When businesses started using the Internet too, the discovery and review of electronic information really started to take off. That is when electronic document review was truly born. That is also when the first e-discovery vendors like Kroll and Attenex (now FTI) started to become large national organizations.

By early turn of the century potential evidence in the form of computer files and emails were multiplying like tribbles. The amount of electronic  evidence started to explode. It has been a dangerous avalanche of e-discovery overload ever since. The needle in the haystack problem was born that still challenges document review today. See Document Review and Predictive Coding: Video Talks – Part One.

Like several others I sensed the danger in the information explosion, saw how it was overwhelming discovery and making it too expensive. For that reason in 2006, again like several others (although I was the only one in Florida), I stopped practicing as a commercial litigator and limited my work to e-discovery only. Since that time electronic document reviews have been front and center in my practice. To be honest, I have not even seen an original paper document in discovery since that time, although I have heard they still exist. (Other attorneys have shown me their paper cuts to prove it. What a dangerous job paper document reviews can be.)

Five Videos Explain the Five Tips

The five tips shared here are rooted in the ancient history of paper productions, and pre-vendor computer file search, but are designed for current electronic practices and post 2015 amended rules of procedure. After a lifetime of work in this area, there are more tips I could provide, and will do so in the future, I’m sure, but these are the ones that occur to me today. The videos below explain these five tips and how you can implement them.

In this opening eleven-minute video I share what may be the most important tip of all, the avoidance of time pressures and resultant hurried activities.

Tip # 1 – Never Put Yourself in a Time Bind – Be Proactive

________

5-Tips_Review_ETHICSThe next video explains the second tip, Ethics. It is always important to do the right thing, including the production of requested relevant documents that will harm your client and their case.  Ethics is document review, like in all other areas of legal practice, indeed, like all other areas of life, is imperative, not discretionary. My thanks to the legal mentors in my past who drilled this into me from my first day out of law school. Any success I have enjoyed in my career I owe, at least in part, to their good influences.

slippery_slopeCall this Ethics advice the Boy Scout tip if you wish, but it really works to avoid a panoply of errors, including potentially career-ending ones. It also helps you to sleep at night and have a clean conscience. The slippery slopes of morality are where the worst errors are made in all legal tasks, but this is particularly true in document review. Discovery in our system is run by lawyers, not judges, magistrates, or special masters. It is based on lawyers faithful conduct and compliance with the rules, including the all-important rules requiring the voluntary production of evidence harmful to a client (a notion strange to many legal systems outside of the U.S.).

Lawyers know the rules, even if their clients do not, and it is critical that they follow them earnestly, holding up against all pressures and temptations. At the end of the day, your reputation and integrity are all that you have, so compromising your ethics is never an acceptable alternative. The Rules of Professional Conduct must be the guiding star of all legal practice, including electronic document review. It is your job as a lawyer to find the evidence and argue it’s meaning; never to hide it. This video is a reminder of a core truth of lawyer obligations as officers of the court.

Tip #2 – Ethics and Electronic Discovery

For more of Losey’s thoughts on ethics and e-discovery, seeLawyers Behaving Badly: Understanding Unprofessional Conduct in e-Discovery, 60 Mercer L. Rev. 983 (Spring 2009); Mancia v. Mayflower Begins a Pilgrimage to the New World of Cooperation, 10 Sedona Conf. J. 377 (2009 Supp.); e-Discovery for Everyone, Chapters 15-19, (ABA, 2017).

focus2Our third tip is Focused Concentration, which was mentioned in passing in the Part One video on Time, and also tips four and five, on Worms and Check Again. The Focus tip is based on my own experiences in cultivating the ability to concentrate on legal work, or anything else. It is contra to the popular, but erroneous notion, a myth really, that you can multi-task and still do each task efficiently. Our brain does not work that way. See Eg. Crenshaw, The Myth of Multitasking: How “Doing It All” Gets Nothing Done; and the work of neuroscientist Daniel J Levitin, who has found the only exception is adding certain background music. All document reviewers who wear headsets, myself included, know this exception very well.

Tip #3 – Focused Concentration

Steve-Jobs-zenFor more on quality control and improved lifestyle by focused attention and other types of meditation, see my earlier video blog, Document Review and Predictive Coding: Video Talks – Part Six, especially the 600 word introduction to that video that includes information on the regular meditation practices of Supreme Court Justice Stephen Breyer, among others. See A Word About Zen Meditation. This practice helped Steve Jobs, and helps Justice Breyer and countless others. It could help you too. Also see these excellent online services, Insight Timer  and Mindfulnes App. These practices will, at the very least, allow for more focused attention to what you are doing, including document review, and thus greatly reduce mistakes.

The next Worms tip is a simple technical one, unique to e-discovery, where Worm is an acronym that means write once, read many times. I prefer to make productions on write-only or recordable only CDs, aka, CD-R, or DVD-R, and not by file transfers. I do not want to use a CD-RW, or DVD-RW meaning one that is rewritable.

Tip #4 – Use WORMS to Produce

Speaking of WORMs, did you know that the SEC requires all broker-dealers to preserve its records for three years in a format that prevents alteration? That means our Write Once Read Many times format. SEC Interpretation: Electronic Storage of Broker-Dealer Records, 17 CFR Part 241 [Release No. 34-47806] (5/12/13).

On December 21, 2016, twelve large broker-dealer firms agreed to pay fines totaling $14.4 million to the Financial Industry Regulatory Authority (FINRA) over allegations, in FINRA’s words, that “they failed to preserve electronic records in a WORM format that couldn’t be altered.” This has to be the all time most expensive “can of worms.”

The fifth tip of Check Again, has to do with the importance of redundancy in quality control, subject only to proportionality considerations, including the tip to spot check your final production CD. I discuss briefly the tendency of lawyers to be trapped by paralysis by analysis, and why we are sometimes considered deal killers by business people because we focus so much on risk avoidance and over-think things. There has to be a proportional limit on the number and cost of double-checks in document review. I also mention in the fifth tip my Accept of Zero Error and ei-Recall checks, which are quality assurance efforts that we make in larger document review projects.

Tip #5 – Check Again

____________

These are five tips to help everyone doing electronic document review. They are not necessarily the “top five,” but they are all important. We suggest you drill these five best practices into your document review team.

For more information on best practices of document review see these three periodically updated resources:

 

 



Top Twenty-Two e-Discovery Opinions of 2016: Number One

January 1, 2017

Here is the e-Discovery Team’s most interesting e-discovery opinion of 2016: Hyles v. New York City, No. 10 Civ. 3119 (AT)(AJP), 2016 WL 4077114 (S.D.N.Y. Aug. 1, 2016)!!!!! The author is well-known e-discovery expert, Judge Andrew Peck of the SDNY.

ONE – Hyles v. New York City

judge_andy_peckThis opinion, like that of Number Two, Dynamo Holdings, is on the e-Discovery Team’s favorite topic, predictive coding. Admittedly, that had a lot to do with the Team’s pick of Hyles as this year’s most interesting e-discovery opinion. Hyles v. New York City, No. 10 Civ. 3119 (AT)(AJP), 2016 WL 4077114 (S.D.N.Y. Aug. 1, 2016). So too did the fact that it was written by Judge Peck. He is known for his excellent legal analysis, especially on the legal search topics. The Hyles opinion was not only spot on, it had the clarity and writing quality demanded of any opinion to be ranked one in 2016.

Although Hyles did not make new law, it clarified existing law on predictive coding. We like that. The specific issue addressed in Hyles has been discussed before, but never squarely ruled on by Judge Peck. In Hyles the plaintiff wanted to require the defendant City (i.e., the responding party) to use TAR (technology assisted review, aka predictive coding), instead of the method the City preferred of keyword searching. As expected, Judge Peck ruled that a party cannot be forced to do predictive coding, even if it is a better method than what the party wants to do, in this case, keyword search; and, even if the party’s preferred method had not yet started. As expected, the reason for this ruling was old Sedona Principle Six.

We liked the Hyles opinion, over Dynamo Two, because Hyles does not include descriptions of cockamamy methods of predictive coding, like Dynamo does. Instead, Hyles involves a more basic methodology, one faced by most e-discovery practitioners today, not just predictive coding specialists, on how to cull down the ESI universe subject to review for relevance by: (1) custodian priority; and, (2) date range. This is a predictive coding case that covers pre-predictive coding methods. It that sense Hyles is like Judge Peck’s other classic legal search opinion from the pre-predictive coding era, Gross ConstructionWilliam A. Gross Constr. Assocs., Inc. v. Am. Mutual Mfrs. Ins. Co., 256 F.R.D. 134, 134 (S.D.N.Y. 2009).

Here is how Judge Peck described counsels efforts to agree upon a method to cull the universe of documents to be reviewed for relevance.

As to date range, the parties agreed on a start date of September 1, 2005 but disagreed on the end date. … After hearing the parties’ arguments at the conference, the Court ruled that the end date would be April 30, 2010 (when defendant Patricoff was reassigned from her First Deputy Commissioner position), without prejudice to Hyles seeking documents or ESI from a later period, if justified, on a more targeted inquiry basis.

Notice how Judge Peck ruled, but without prejudice for the requesting party to come back later, if need be. Most discovery rulings on issues like that should be open-ended, with the idea that any follow-up requests must be narrow and focused, and thus relatively inexpensive to fulfill.

Judge Peck makes the same kind of ruling at to the total number of custodians whose ESI must be reviewed.

As to custodians, the City agreed to search the files of nine custodians (including Hyles), but not six additional custodians that Hyles requested. (7/18/16 Ltr. at 5, 7.) The Court ruled that discovery should be staged, by starting with the agreed upon nine custodians (Hyles, Stark, Patricoff and six others). After reviewing the production from the nine custodians, if Hyles could demonstrate that other custodians had relevant, unique and proportional ESI, the Court would consider targeted searches from such other custodians.[1]

Here is how Judge Peck quickly frames the dispute that the parties brought to him for resolution. (All record citations omitted.)

After the parties had initial discussions about the City using keywords, Hyles’ counsel consulted an ediscovery vendor and proposed that the City should use TAR as a “more cost-effective and efficient method of obtaining ESI from Defendants.”  The City declined, both because of cost and concerns that the parties, based on their history of scope negotiations, would not be able to collaborate to develop the seed set for a TAR process.

Andrew J. PeckJudge Peck began his analysis as you might expect by agreeing with the plaintiff that “in general, TAR is cheaper, more efficient and superior to keyword searching.” Then he set out the legal precedent history embodying his thinking on predictive coding and whether a party should be required to use TAR against their will.

In March 2009, the “dark ages” in terms of ediscovery advances, this Court described problems with keywords and the need for “careful thought, quality control, testing, and cooperation with opposing counsel in designing search terms or `keywords.'” William A. Gross Constr. Assocs., Inc. v. Am. Mutual Mfrs. Ins. Co., 256 F.R.D. 134, 134 (S.D.N.Y. 2009) (Peck, M.J.). Further elaborating on the deficiencies of keyword searching, my seminal Da Silva Moore decision in 2012 approved the use of predictive coding, aka TAR, in appropriate cases. Da Silva Moore v. Publicis Groupe, 287 F.R.D. 182, 190-91, 193 (S.D.N.Y. 2012) (Peck, M.J.). In again approving the use of TAR in 2015, I wrote that “the case law has developed to the point that it is now black letter law that where the producing party wants to utilize TAR for document review, courts will permit it.” Rio Tinto PLC v. Vale S.A., 306 F.R.D. 125, 127 (S.D.N.Y. 2015) (Peck, M.J.).[3] Dicta in a footnote in Rio Tinto stated that “[i]n contrast, where the requesting party has sought to force the producing party to use TAR, the courts have refused.” Rio Tinto PLC v. Vale S.A., 306 F.R.D. at 127 n.1. “The Court note[d], however, that in [the cited] cases the producing party had spent over $1 million using keyword search (in Kleen) or keyword culling followed by TAR (in Biomet), so it is not clear what a court might do if the issue were raised before the producing party had spent any money on document review.” Rio Tinto PLC v. Vale S.A., 306 F.R.D. at 127 n.1. Since the search methodology issue arose in this case before the City spent much, if any, money on searching for responsive ESI, this case squarely raises the issue of whether the requesting party can have the Court force the responding party to use TAR.

The plaintiff also argued that since parties should cooperate in discovery the City should cooperate and use the best technology available to find relevant evidence. Judge Peck rejected this argument as follows:

Hyles’ counsel is correct that parties should cooperate in discovery. I am a signatory to and strong supporter of the Sedona Conference Cooperation Proclamation, and I believe that parties should cooperate in discovery. See William A. Gross Constr. Assocs., Inc. v. Am. Mfrs. Mut. Ins. Co., 256 F.R.D. at 136; Rio Tinto PLC v. Vale S.A., 306 F.R.D. at 129 n.6. The December 1, 2015 Advisory Committee Notes to amended Fed. R. Civ. P. 1 emphasized the need for cooperation. Cooperation principles, however, do not give the requesting party, or the Court, the power to force cooperation or to force the responding party to use TAR.

His comment that a Court does not have the “power to force cooperation” is also interesting and somewhat controversial. Perhaps it is a matter of semantics, but I have seen many judges “order” parties to cooperate. Still, his second point on cooperation is that the doctrine cooperation does not require a responding party to use TAR if they do not want to. Cooperation does not mean capitulation.

Judge Peck then goes on to articulate the main reason that a judge should not ordinarily force a party to use a particular tool or technique to mine client data for useful evidence. That should be the litigant’s independent duty and the court should not interfere without cause. This is part of what is known as The Sedona Conference Principle Six as is well explained by Judge Peck in Hyles.

It certainly is fair to say that I am a judicial advocate for the use of TAR in appropriate cases. I also am a firm believer in the Sedona Principles, particularly Principle 6, which clearly provides that:

Responding parties are best situated to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own electronically stored information.

The Sedona Principles: Second Edition, Best Practices Recommendations & Principles for Addressing Electronic Document Production, Principle 6 (available at http://www.TheSedonaConference.org).

Under Sedona Principle 6, the City as the responding party is best situated to decide how to search for and produce ESI responsive to Hyles’ document requests. Hyles’ counsel candidly admitted at the conference that they have no authority to support their request to force the City to use TAR. The City can use the search method of its choice. If Hyles later demonstrates deficiencies in the City’s production, the City may have to re-do its search.[4] But that is not a basis for Court intervention at this stage of the case.

Notice how the decision to not compel the use of TAR is without prejudice. The plaintiff can revisit the request by demonstrating deficiencies in the defendant’s production.

Judge Peck then quotes with approval the recent Dynamo Two opinion where Judge Buch held that it was not the court’s business to dictate to attorneys how to do document review, and again relied on Sedona Six. Here is Judge Peck’s concluding words.

Here, too, it is not up to the Court, or the requesting party (Hyles), to force the City as the responding party to use TAR when it prefers to use keyword searching. While Hyles may well be correct that production using keywords may not be as complete as it would be if TAR were used (7/18/16 Ltr. at 4-5), the standard is not perfection, or using the “best” tool (see 7/18/16 Ltr. at 4), but whether the search results are reasonable and proportional. Cf. Fed. R. Civ. P. 26(g)(1)(B).

To be clear, the Court believes that for most cases today, TAR is the best and most efficient search tool. That is particularly so, according to research studies (cited in Rio Tinto), where the TAR methodology uses continuous active learning (“CAL”), which eliminates issues about the seed set and stabilizing the TAR tool. The Court would have liked the City to use TAR in this case. But the Court cannot, and will not, force the City to do so. There may come a time when TAR is so widely used that it might be unreasonable for a party to decline to use TAR. We are not there yet. Thus, despite what the Court might want a responding party to do, Sedona Principle 6 controls. Hyles’ application to force the City to use TAR is DENIED.

Our view of the most interesting opinion of 2916. The e-Discovery Team agrees with Judge Peck that “for most cases today, TAR is the best and most efficient search tool.” To be clear, however, our agreement is predicated upon the TAR tool being used properly. The method of TAR matters. The e-Discovery Team would much rather work on a well-run, well-designed keyword search project, than a mismanaged, poorly designed predictive coding project.

We think the wise words of Judge Facciola in O’Keefe in 2008 about angels have been too early forgotten:

Whether search terms or ‘keywords’ will yield the information sought is a complicated question involving the interplay, at least, of the sciences of computer technology, statistics and linguistics…. Given this complexity, for lawyers and judges to dare opine that a certain search term or terms would be more likely to produce information than the terms that were used is truly to go where angels fear to tread. This topic is clearly beyond the ken of a layman.

peck_facciola

United States v. O’Keefe, 537 F. Supp. 2d 14 (D.C. 2008). What Judge Facciola said about keyword search is as true today as when written. When it comes to legal search today using active machine learning, which is the true meaning of TAR, the expertise required is even greater. Predictive coding requires special skills and a unique knowledge set to do right. It is clearly beyond the ken of almost all attorneys practicing law today. We do not see this gap narrowing, not because the education is not available, but because most lawyers are disinterested. For this reason the competency gap is widening and the problem noted by Judge Facciola in 2008 is still alive and well today.

LoveIn spite of this competency gap, and the stupid fearlessness of many trial lawyers, those who do not even know what they do not know, courts continue to approve the use of TAR carte blanche, with no requirement of expert assistance or use of proven methodologies. For that reason our agreement with Judge Peck on the superiority of predictive coding must be qualified. Still, we agree, because when active machine learning is done right it is a thing of beauty, far more effective than keywords in all but the simplest projects. Why I Love Predictive Coding: Making document review fun with Mr. EDR and Predictive Coding 3.0. (e-discovery team, 2/14/16).

We also agree with Judge Peck’s speculation that there may come a time when a court forces the use of best practices by recalcitrant lawyers. Judge Peck may even reverse himself on this point before TAR is more widely used, especially if  Sedona Principle Six is revised or shown to be inapplicable to a particular case. Is it really true, as Principle Six asserts, that “Responding parties are best situated to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own electronically stored information.” They may be in the best position for preservation, but are they for search and production? Legal search, especially active machine learning, is a speciality well beyond the IT capabilities and skills of responding parties. Legal search does not require special knowledge of the data itself, it requires special knowledge of the procedures, methodologies, and technologies for electronic document review, including especially predictive coding. This is knowledge possessed by e-discovery specialists, by lawyers who specialize in legal search, not by litigant’s IT departments.

It is hard to see how Principle Six applies to choice of document review software and feature utilization. We need to think this through and have a vigorous debate on the continued application of Principle Six to document review defensibility. See Ball, Craig, Sedona Principle Six: Overdue for an Overhaul (10/10/14) (“It’s time to deep six Sedona Six.“)

In the not too remote future, when Hyles is someday reversed (perhaps by Judge Peck himself) and a party is ordered to use TAR, we expect (hope) that the opinion will also specify the particular method or methods of use of TAR. Otherwise, the order is too general to have any meaning. You might as well order an attorney to use a computer to do document review. There are many, many ways to do TAR. Most of them are wrong. Any attorney angel should fear to tread TAR without the help of experts.

If ESI continues to grow more complicated, and the volumes of data continue to explode, then in the future legal search that includes predictive coding may well be the only way document discovery can be conducted. Litigation lawyers of the future may still do depositions, motion practice, trials and the like, but it is unlikely they will also continue to do large volume document review. They will leave that to active machine learning experts. The improvements we see in the use of artificial intelligence and easier-to-use software will help expand the group of experts, the specialists in document review, but it is likely to bring it within the reach of the general litigator.

The Bar is already faced with a large competency gap. A new type of legal work is emerging to fill that competency gap, a new job, where specialists in AI enhanced evidence search handle all document discovery. A dual track for trial preparation is emerging. One group of lawyers will be concerned with electronic document discovery and another group will handle all of the other litigation tasks.

Looking a little further into the future, we expect courts may eventually turn over the entire ESI search and production process over to neutral expert specialists serving as discovery masters (or something like that). That may well be the best means for the just, speedy and efficient resolution of most law suits.

 


%d bloggers like this: