Lawyers’ Job Security in a Near Future World of AI, the Law’s “Reasonable Man Myth” and “Bagley Two” – Part Two

January 22, 2017

This is the second and concluding section to the two-part blog, Lawyers’ Job Security in a Near Future World of AI, the Law’s “Reasonable Man Myth” and “Bagley Two.” Click here to read Part One.

Robot_handshake

___________

Next consider Judge Haight’s closing words to the opinion dated December 22, 2016, Ruling On Plaintiff’s Motion To Compel; Bagely v. Yale, Civil Action No. 3:13-CV-1890 (CSH):

However, requiring this additional production, or a further deposition in case of need, is in keeping with a governing objective of the Federal Rules of Civil Procedure: “By requiring disclosure of all relevant information, the discovery rules allow ultimate resolution of disputed issues to be based on full and accurate understanding of true facts.” 6 Moore’s Federal Practice § 26.02 (Matthew Bender 3d ed.). 6

__________

6 While Yale may not welcome the measurement of its obligations in the case at bar by these principles, it is worth recalling that the treatise’s principal initial author, James Wm. Moore, was a towering figure on the faculty of Yale Law School. In his preface to the first edition (1938), Professor Moore referred to his effort “at all times to accord to the Rules the interpretation which is most likely to attain the general objective of the new practice: the settlement of litigation on the merits.” That is the interpretation this Ruling attempts to adopt.

william_moore_prof_yale

Prof. Moore (1905-1994)

Poor Yale. Moore’s Federal Practice is one of the most cited treatises in the law. James W. Moore was the author of the 34-volume Moore’s Federal Practice (2d ed., 1948) and the three-volume Moore’s Manual: Federal Practice & Procedure (1962). He was also the Sterling Professor Emeritus of Law at Yale University, where he taught for 37 years. Who else but Yale can have anything in Moore’s thirty-four volume treatise held against them personally? Seems kind of funny, but I am sure Yale’s attorneys were not laughing.

Getting back to the case and Judge Haight’s decision. Aside from showing the malleability and limits of reason, Bagley Two provides some important new precedent for e-discovery, namely his rulings on privilege and the discoverability of a party’s preservation efforts. Judge Haight starts by repeating what is now established law, that a party’s preservation efforts are not satisfied by mere issuance of a notice, that a whole process is involved and the process must be reasonable. He then goes on to provide a pretty good list of the facts and circumstances that should be considered to determine reasonability.

[A] party’s issuance of a litigation hold notice does not put an end to the party’s obligation to preserve evidence; it is, rather, the first in a series of related steps necessary to ensure that preservation. As Magistrate Judge Francis aptly observed in Mastr Adjustable Rate Mortgages Trust 2006 v. UBS Real Estate Securities Inc., 295 F.R.D. 77, 85 (S.D.N.Y. 2013): “A litigation hold is not, alone, sufficient; instead compliance must be monitored.”

In spoliation cases involving litigation hold notices, one can discern from Second Circuit and district court opinions a number of decisive questions:

1. When did a party’s duty to preserve evidence arise?
2. Did the party issue a litigation hold notice in order to preserve evidence?
3. When did the party issue a litigation hold notice, in relation to the date its duty to preserve the evidence arose?
4. What did the litigation hold notice say?
5. What did recipients of the litigation hold notice do or say, in response to or as result of, the notice?
6. After receiving recipients’ responses to the litigation hold notice, what further action, if any, did the party giving the notice take to preserve the evidence?

Questions 2 through 6 are entirely fact-specific to a given case. Question 1 is a mixed question of law and fact, whose legal element the Second Circuit defined in Fujitsu Ltd. v. Federal Express Corp., 247 F.3d 423, 436 (2d Cir. 2001): “The obligation to preserve evidence arises when the party has notice that the evidence is relevant to litigation or when a party should have known that the evidence may be relevant to future litigation.”

In the case at bar, I am unable to accept Yale’s argument that the litigation hold notices it issued about Bagley and the recipients’ responses to the notices are immune from discovery because (in the absence of proof that spoliation had in fact occurred) such documents “are subject to the attorney-client and to work product privileges,” Defendants’ Brief [Doc. 192], at 3. That contention is something of a stretch. … . Assuming that all of Clune’s litigation hold notices were sent to employees of Yale, Clune was in effect communicating with his client. However, the predominant purpose of that communication was to give recipients forceful instructions about what they must do, rather than advice about what they might do. 3

I like the list of six key facts to consider to weigh the reasonability of preservation efforts, especially the last one. But my primary point here is the malleability of reason in classifying the notice as unprotected. A letter from in-house counsel telling employees that the law requires them to preserve is not advice entitled to privilege protection? It’s predominant purpose was instead unprotected instructions? The language of the litigation hold notices was earlier quoted in the opinion. It’s language included the following:

[A]ll members of the Yale faculty and staff who have information in their possession or control relating or referring in any way to Professor Bagley, her employment and teaching at SOM, or the circumstances relating to the non-renewal of her faculty appointment (collectively “this Matter”) have a legal obligation to preserve that information. The law imposes this obligation to prevent the loss of potential evidence during litigation. You must preserve and retain, and not alter, delete, remove, discard or destroy, directly or indirectly, any information concerning this Matter. Failure to preserve information could seriously undermine Yale’s legal position and lead to legal sanctions.

The lawyer’s letter tells employees that they “have a legal obligation to preserve,” and the legal consequences if they do not. Yet this letter is not advice because the predominant purpose is just an unprotected instruction? That is the holding.

mental_impressionsJudge Haight gets rid of work product protection too.

As for the work product doctrine, it “is not actually a privilege, but rather a qualified immunity from discovery,” codified in Fed. R. Civ. P. Rule 26(b)(3), whose purpose “is to protect an attorney’s mental processes so that the attorney can analyze and prepare for the client’s case without interference from an opponent.” 6 Moore’s Federal Practice, § 26.70[1] (Matthew Bender 3d ed.). 4 That purpose is not implicated by the present exercise.

__________

4 Fed. R. Civ. P. 26 (b)(3) of Civil Procedure protects from disclosure those materials which reveal “the mental impressions, conclusions, opinions, or legal theories of a party’s attorney.” See also In re Steinhardt Partners, L.P., 9 F.3d 230, 234 (2d Cir. 1993) (“At its core, the work product doctrine shelters the mental processes of the attorney, providing a privileged area within which he can analyze and prepare his client’s case.”) (quoting United States v. Nobles, 422 U.S. 225, 238 (1975)) (emphasis added).

I do not agree with Judge Haight on this aspect of his ruling. I think both work product and attorney client apply to these particular notices and his “reasoning”on this issue is wrong. I do, however, agree with his final ruling requiring production. I think the protections had been waived by the circumstances and actions of defense counsel, which, by the way, they were correct in doing. I think the waiver on their part was necessary. Judge Haight also mentioned waiver, but as dicta alternative grounds in footnote three:

3 The Court also notes that to the extent that Yale’s litigation hold notices included the text of the exemplar provided to Plaintiff as “document preservation notices,” that text has already been revealed publicly in this case, so that secrecy or privilege relating to that language was destroyed or waived. See Doc. 191-1, Ex. F.

triggerJudge Haight then looks at the question of when Yale’s duty to preserve commenced. Recall Yale kept adding custodians in eight stages. The first were pre-litigation notices. They were made, I note, after Yale’s lawyer mental processes told him that litigation was reasonably likely. The last were made after suit was filed, again based on the lawyer’s mental processes causing him to believe that these additional witnesses might have relevant evidence. The mental processes of Plaintiff’s attorneys led them to believe that all of the notices, including the pre-litigation notices, were sent too late and thus spoliation was likely. Here is Judge Haight’s analysis of the trigger issue:

When, during the course of this melancholy chain of events, should Yale have known that evidence pertinent to Bagley’s reappointment might be relevant to future litigation? That is a crucial question in spoliation analysis. A state of reasonable anticipation clearly antedates the actual filing of a complaint; in Fujitsu, 247 F.3d at 436, the Second Circuit was careful to couple actual present and possible future litigation as catalysts of equal strength for the preservation of evidence.

Bagley has not yet formally moved for spoliation sanctions, and so the question is not yet before me for decision, but some preliminary, non-binding observations may be made. The record previously made in the case shows that Bagley’s personal distress and institutional disapproval and distrust grew throughout the winter and spring of 2012 (the last year of her five-year appointment), so that when on May 24, 2012, Dean Snyder told Bagley that she would not be reappointed, it would not be irrational to suppose that Bagley might soon transform herself from disheartened academic to vengeful litigant. In fact, Bagley filed an internal discrimination complaint against Yale during the following month of June 2012 (which had the effect of bringing Provost Salovey out of the wings and onto the stage).

Predictable_IrrationalNote the Judge’s use of the phrase not be irrational to suppose. What is the impact of hindsight bias on this supposedly objective, rational analysis? Bagley’s later actions made it obvious that she would sue. She did sue. The law suit has been very contentious. But was it really all that obvious back in 2012 that Yale would end up in the federal courthouse? I personally doubt it, but, admit it is a close judgment call. We lawyers say that a lot. All that phrase really means is that reason is not objective. It is in the eye of the beholder.

Judge Haight then wraps up his analysis in Bagley Two.

What happened in this case is that Yale identified 65 individuals who might have evidence relevant to Bagley’s denial of reappointment, and issued them litigation hold notices in eight separate batches, a process that took a considerable amount of time. The first nine notices were sent nine months after Snyder told Bagley she would not be reappointed. The last was sent eight months after Bagley filed this action. To characterize the pace of this notification process as culpable or even negligent would be premature on the present record, but it is fair to say that it was leisurely, to an extent making it impossible to dismiss as frivolous Bagley’s suggestion that she might move for a spoliation sanction. The six questions outlined supra arise in this case, and the factors pertinent to resolving them include an unreasonable delay in issuing the notices and a subsequent failure to implement and monitor the recipients’ responses. Judge Sweet said in Stimson that the Second Circuit has left open “the question of whether a sufficiently indefensible failure to issue a litigation hold could justify an adverse inference on its own,” and an additional factor would be “the failure to properly implement the litigation hold even after it was issued.” 2016 WL 54684, at *6. These are legitimate questions in the case at bar. Bagley is entitled to discovery with respect to them. 5 (footnote citations omitted)

I certainly agree with Judge Haight on all of those points and law. Those factual circumstances do justify the modest amount of discovery requested by the plaintiff in this motion.

gavelNow we get to the actual Order on the pending motion to compel:

Therefore I conclude that in the circumstances of this case, Bagley’s “Motion to Compel” [Doc. 190] is GRANTED. Bagley is entitled to examine the litigation hold notices issued by Yale, and the responsive survey forms that notice recipients returned to Yale. These documents bear directly upon the questions courts identify as dispositive in spoliation cases. Bagley is entitled to discovery in these areas, in order to discern the merit or lack of merit of a formal claim for spoliation claim. To the extent that Yale objects to production of these documents on the grounds of privilege or the work product doctrine, the objections are OVERRULED.

For the same reasons, Bagley is also entitled to an affidavit from a Yale officer or employee (not a notice recipient or recipients) which describes what non-ESI documents Yale received from notice recipients and what was done with them. On a spoliation claim, Bagley will ultimately bear the burden of showing that pertinent evidence was destroyed or rendered unavailable. This discovery may cast light on that disputed issue. Yale may prefer not to have to produce that information; Yale’s counsel miss no opportunity to remind the Court how much discovery effort the case has previously required.

Judge Haight then ended his opinion with the previously quoted zinger regarding Yale’s famous law Professor Moore. This zinger and comments about Yale’s leisurely efforts and Yale counsel’s missing no opportunities to remind the court tell a story of their own. It shows the emotional undertone. So too does his earlier noted comment about “spoliation” being a cardinal litigation vice, well known to practicing attorneys and judges, but “perhaps unfamiliar” to academics. I suspect this goes beyond humor.

Artificial Intelligence and the Future of Employment

robot_whispererI am sure legal reason will improve in the future and become less subjective, less subject to hidden irrationalities and prejudices. By using artificial intelligence our legal doctrines and decision making can be improved, but only if the human judges remain in charge. The same comment goes for all attorneys. In fact, it applies to all current employment.

The doom and gloom futurists disagree. They think AI will replace humans at their jobs, not empower them. They envision a future of cold automation, not man-machine augmentation. They predict wide-spread unemployment with a loss of half of our current employment. An University of Oxford study predicted that almost half of all U.S. jobs could be lost to automation in the next twenty years. Even the influential World Economic Forum predicts predicts that Five Million jobs could be lost by 2020. Five Million Jobs by 2020: the Real Challenge of the Fourth Industrial Revolution. Also seeThe Future of Jobs: Employment, Skills and Workforce Strategy for the Fourth Industrial Revolution (World Economic Forum, Jan. 2016).

A contrary view “augmentation” oriented group predicts the opposite, that at least as many new jobs will be created as lost. This is a subject of hot debate. See EgArtificial intelligence will save jobs, not destroy them (World Economic Forum, 1/19/17). Readers know I am in the half-full camp.

James Bessen: Law Prophet of the Future of Employment

james_bessonMany are like me and have an overall positive outlook, including James Bessen, an economist  and Lecturer in Law at the Boston University School of Law. Jim Bessen, who was a good hacker with an entrepreneurial background (he created the first WYSIWYG desktop publishing software), has researched the history of computer use and employment since 1980. Jim’s research has shown that for those who can keep up with technology, there will be new jobs to replace the ones lost. Bessen, How Computer Automation Affects Occupations: Technology, Jobs & Economics, Boston University School of Law Law & Economics Working Paper No. 15-49 (1/16/16). He also found that wages in occupations that use computers grow faster, not slower:

[B]ecause higher wage occupations use computers more, computer use tends to increase well-paid jobs and to decrease low-paid jobs. Generally, computer use is associated with a substantial reallocation of jobs, requiring workers to learn new skills to shift occupations.

Also see the article in The Atlantic magazine by Bessen, The Automation Paradox: When computers start doing the work of people, the need for people often increases, (The Atlantic, 1/19, 2016) where he said:

…workers will have greater employment opportunities if their occupation undergoes some degree of computer automation. As long as they can learn to use the new tools, automation will be their friend.

This is certainly consistent with what I have seen in the legal profession since I started practice in 1980.

james_bessenJames Bessen has also written a book on this, Learning by Doing: The Real Connection Between Innovation, Wages, and Wealth. (Yale U. Press 2015). In this book, Bessen, in his words:

… looks at both economic history and the current economy to understand how new technology affects ordinary workers and how society can best meet the challenges it poses.

He notes that major new technologies always require new human work skills and knowledge, and that today, as before, they are slow and difficult to develop. He also makes the observation, which is again consistent with my own experience as a tech-lawyer, that relevant technical knowledge “develops slowly because it is learned through experience, not in the classroom.” In his analysis that is because the new knowledge is not yet standardized. I agree. This is one reason my work has been focused on the standardization of the use of active machine learning in the search for electronic evidence; see for example Predictive Coding 4.0 and my experiments at the TREC conference on predictive coding methods sponsored by the National Institute of Standards and Technology. Also see: Electronic Discovery Best Practices. In spite of my efforts on standards and best practices for e-discovery, we are still in the early, rapidly changing, non-standardized stage of new technology. Bessen argues that employer policies and government policies should encourage such on-the-job learning and perfection of new methods.

Jim Bessen’s findings are starting to be discussed by many who are now concerned with the impact of AI on employment. See for instance, Andrea Willige’s article in the World Economic Forum concerning Davos for 2017Two reasons computers won’t destroy all the jobs (“jobs don’t disappear, they simply move up the skills and wage ladder. For workers to move up the ranks, they must acquire the necessary skillset.”).

Standardization v. On-the-Job Training

Moving on up requires new employment skills. It requires workers who can step-in, step-up, step-aside, step-narrowly, or step-forward. Only Humans Need Apply; Dean Gonsowski, A Clear View or a Short Distance? AI and the Legal Industry, and, Gonsowski, A Changing World: Ralph Losey on “Stepping In” for e-Discovery, (Relativity Blog) (Interview with references to the the 5-steps described in Only Humans Need Apply). Unless and until standardization emerges, and this is taught in a classroom, the new skills will be acquired by on-the-job learning only, sometimes with experienced trainers, but more often self-taught by trial and error.

Borg_Ralph_headI have been working on creating the perfect, standard method for electronic document review using predictive coding since Da Silva Moore. I have used trial and error and on-the-job learning, buttressed by spending a month a year over the last five years on scientific research and experiments with my own team (remember my Borg experiments and videos?) and with TREC, EDI and Kroll Ontrack. Borg Challenge: Report of my experimental review of 699,082 Enron documents using a semi-automated monomodal methodology (a five-part written and video series comparing two different kinds of predictive coding search methods); Predictive Coding Narrative: Searching for Relevance in the Ashes of EnronEDI-Oracle Study: Humans Are Still Essential in E-Discovery (LTN Nov., 2013); e-Discovery Team at TREC 2015 Total Recall Track, Final ReportTREC 2016 Total Recall Track NOTEBOOK.

predictive_coding_4-0_simpleAfter years we have finally perfected and standardized a highly effective method for document review using predictive coding. We call it Predictive Coding 4.0. This method is complete, well-tested, proven and standardized for my team, but not yet accepted by the industry. Unfortunately, industry acceptance of one lawyer’s method is very difficult (impossible?) in the highly competitive, still young and emerging field of electronic document review. I create a standard because I have to in my work, not because I unrealistically expect the industry to adopt it. The industry is still too young for that. I will continue with my on-the-job training, content with that, just as Bessen, Davenport and Kirby observe is the norm for all new technologies. Someday a standard will be generally accepted and taught in classrooms, but we are far from it.

Conclusion

There is more going on in Bagley Two than objective reason, even assuming such a thing exists. Experienced attorneys can easily read between the lines. Reasoned analysis is just the tip of the iceberg, or top of the pyramid, as I envisioned in the new model for Holistic Law outlined in my prior article, Scientific Proof.

There is far more to Senior District Judge Charles S. Haight, Jr., than his ability to be logical and apply reason to the facts. He is not just a “thinking machine.” He has wisdom from decades on the bench. He is perceptive, has feelings and emotions, good intuitions and, we can see, a sense of humor. The same holds true for most judges and lawyers, perhaps even law professors. We are all human and have many other capacities beyond what robots can be trained to do.

Jason_Ralph_RobotReason is just one of the things that we humans do, and, as the work of Professor Ariely has shown, it is typically full of holes and clouded by hidden bias. We need the help of computers to get reason done right, to augment our logic and reasoning skills. Do not try to compete with, nor exclude robots from tasks involving reason. You will ultimately lose that battle. Instead, work with the robots. Invite them in, but remain in control of the processes; use the AI’s abilities to enhance and enlarge your own.

I am sure legal reason will improve in the future and become less subjective. This will happen when more lawyers Step-In as discussed in Davenport and Kirby, Only Humans Need Apply and Dean Gonsowski, A Clear View or a Short Distance? AI and the Legal Industry, and A Changing World: Ralph Losey on “Stepping In” for e-Discovery

alex_hafezMany of us have stepped-in, to use Davenport and Kirby’s language, to manage the use of TAR and AI in document review, not just me. Consider, for instance attorney Alexander Hafez, currently a “Solutions Engineer” for FTI. He was the only other attorney featured in Only Humans Need Apply. Alex bootstrapped his way from minimum wage contract document reviewer, to his current large vendor consultant “step-in” job, by, in the book’s words, “educational bricolage” composed of on-the-job learning and “a specialized course of two and some autodidactic reading.” Id. pg. 144. There are thousands of lawyers in e-Discovery doing quite well in today’s economy. The use of AI and other advanced technologies is now starting to appear in other areas of the law too, including contract review, analysis and construction. See eg. Kira Systems, Inc.

Great-Depression_LitigatorsAs the other areas of the Law become as enhanced and augmented as e-discovery, we will see new jobs open up for the steppers. Old mechanistic law jobs will be replaced. That is for sure. There will be jobs lost in the legal economy. But if Davenport, Kirby and Bessen are correct, and I for one think they are, new, better paying jobs will be created to replace them. Still, for most luddite lawyers, young and old, who are unable to adapt and learn new technologies, the impact of AI on the Law could be devastating. 

Only the tech-savvy will be able to move up the skill and wage ladder by stepping-in to make the technology work right. I attained the necessary skill set to do this with legal technology by teaching myself, by “hacking around” with computers. Yes, it was difficult, but I enjoyed this kind of learning. My story of on the job self-learning is very common. Thus the name of Bessen’s book, Learning by DoingOthers might do better in a more structured learning environment, such as a school, but for the fact there currently is none for this sort of thing, at least in the Law. It falls between the cracks of law school and computer science. For now the self-motivated, self-learners will continue to lead the way.

brad_smith_microsoftNot only do we need to improve our thinking with machines, we need to contribute our other talents and efforts. We need to engage and expand upon the qualities of our job that are most satisfying to us, that meet our human nature. This uniquely human work requires what is sometimes called “soft skills.” This primarily includes the ability for good interpersonal communication, but also such things as the ability to work collaboratively, to adapt to a new set of demands, and to solve problems on the fly. Legal counseling is a prime example according to the general counsel of Microsoft, Brad Smith. Microsoft’s Top Lawyer Toasts Legal Secretaries (Bloomberg Law, 1/18/17). The top lawyer, once CEO of Microsoft, also opined:

Individuals need to learn new skills to keep pace, and this isn’t always easy.  Over the next decade this could become more daunting still, as technology continues to change rapidly.  There is a broadening need for new technical skills and stronger soft skills.  The ability – and opportunity – to continue learning has itself become more important.

Brad Smith, Constructing a Future that Enables all Americans to Succeed, (Dept. of Commerce guest blog, 11/30/16).

The Wikipedia article on “soft skills” lists ten basic skills as compiled by Heckman and Kautz, Hard Evidence on Soft Skills, Labour Econ. 2012 Aug 1; 19(4): 451–464.

  • Communication – oral, speaking capability, written, presenting, listening.
  • Courtesy – manners, etiquette, business etiquette, gracious, says please and thank you, respectful.
  • Flexibility – adaptability, willing to change, lifelong learner, accepts new things, adjusts, teachable.
  • Integrity – honest, ethical, high morals, has personal values, does what’s right.
  • Interpersonal skills – nice, personable, sense of humor, friendly, nurturing, empathetic, has self-control, patient, sociability, warmth, social skills.
  • Positive attitude – optimistic, enthusiastic, encouraging, happy, confident.
  • Professionalism – businesslike, well-dressed, appearance, poised.
  • Responsibility – accountable, reliable, gets the job done, resourceful, self-disciplined, wants to do well, conscientious, common sense.
  • Teamwork – cooperative, gets along with others, agreeable, supportive, helpful, collaborative.
  • Work ethic – hard working, willing to work, loyal, initiative, self-motivated, on time, good attendance.

soft-skills_cartoon

As Brad Smith correctly observed, the skills and tasks needed to keep pace with technology include these kinds of soft skills as well as new technological know-how, things like the best methods to implement new predictive coding software. The tasks, both soft and technical, are generally not overly repetitive and typically require some creativity, imagination, flexibility and inventiveness and, in my view, the initiative to exceed original parameters.

cute_robotA concerned lawyer with real empathy who counsels fellow humans is not likely to be replaced anytime soon by a robot, no matter how cute. There is no substitute for caring, human relationships, for comforting warmth, wit and wisdom. The calm, knowledgeable, confident presence of a lawyer who has been through a problem many times before, and assures you that they can help, is priceless. It brings peace of mind, relaxation and trust far beyond the abilities of any machine.

Stepping-in is one solution for those of us who like working with new technology, but for the rest of humanity, soft-skills are now even more important. Even us tech-types need to learn and improve upon our soft skills. The team approach to e-discovery, which is the basic premise of this e-Discovery Team blog, does not work well without them.

ralph_17_pallate_knife_2Brad Smith’s comment on the need for continued learning is key for everyone who wants to keep working in the future. It is the same thing that Bessen, Davenport and Kirby say. Continued learning is one reason I keep writing. It helps me to learn and may help others to learn too, as part of their “autodidactic reading” and “educational bricolage.” (How else would I learn those words?) According to Bessen’s, Davenport and Kirby’s research most of the key skills needed to keep pace can only be learned on-the-job and are usually self-taught. That is one reason online education is so important. It makes it easier than ever for otherwise isolated people to have access to specialized knowledge and trainers.


Five Tips To Avoid Mistakes In Electronic Document Review

January 9, 2017

5-Tips_ReviewThese tips are based on a long life of litigation legal practice, including thousands of document reviews going back to 1978. I have seen hundreds of mistakes over the years, especially in the last decade when my work as a lawyer has been limited to electronic discovery. Many of these blunders were made by “the other side.” Some were funny and made me smile, others were not and led to motions of all kinds. Keeping it real, I have made my own fair share of errors too. Those lessons were painful, but are now deeply etched. No doubt I would have made many more errors, but for the generous guidance provided by more senior and experienced attorneys that I have had the very good fortune to work with. It is with this great debt in mind that I offer up these tips.

Click here to download a Word version. [An earlier version of this article was published last year.)

Some Mistakes are Funny

Gloat_SimpsonsOn the funny side of observing document review mistakes, I will never forget the time, not too long ago, where the other side produced documents to us with the most important ones placed together up front. That was a surprising electronic zipped production to open. It was fairly obvious what had happened. The highly relevant documents were not mixed-in as they should have been with the other more plebeian merely relevant documents. Instead, the hot documents were all together at the front of the production with the lowest numbers. (Sixth tip – never do that!) Our team laughed at the error, as we easily and quickly found lots of great stuff to help our case. Still, we kept a discrete silence and did not gloat. (Seventh tip – Never do that either, at least not in front of them!)

Opposing counsel, who later became a friend, admitted the error to me months after the case settled. He found out what happened a few days too late. Even he chuckled as to how inadvertently “nice” they were. As is often the case, the mistake did not really matter in the end. We would have recognized the hot documents anyway. As usual when errors happen in e-discovery, he blamed the vendor. They almost always get blamed for mistakes, but, the truth is, vendors are just tools of the attorneys (no offense dear vendors, tools are important). The attorneys are almost always the ones ultimately responsible for screw-ups.

Lessons of History

Clarence Darrow and William Jennings BryanThe five tips shared here are born out of a long history of document review. How relevant could past legal practice be you might ask? In 1980, just like today, document discovery was and still is a search for documents and communications that have probative value. The tools and document forms have changed, but not the basic tasks. The federal rules have changed too, but not that much, and the ethics of an attorney controlled discovery system, not at all.

Discovery has always been a search to determine what really happened, to sort out the conflicting stories and find evidence for use at trial. Legal counsel never creates facts. That is called falsification of evidence and is a crime. Attorneys just find the facts and then do the best they can with them; make them look as good as possible by legal argument and clever presentation. The discovery effort has always been a fairly cooperative one between attorneys. It has always been a question of trust but verify. Conversely, there have always been a few slime balls in the Bar who do not get that, but that is what judges (and Bar ethics committees) are for, and they soon sniff out the weasels. All things evolve and change, but some basic patterns remain the same.

By the early nineties I sometimes had to look beyond paper files and investigate computers for possible evidence. That occasionally happened in trade-secret cases, much like today. Forensics was fairly easy back then. My favorite ESI search and review tool was Norton Utilities, which I had been using since the mid-eighties. Like most computer lawyers around those days, as we were called, I was by necessity a DOS master, and, until around 1997, a one man IT department for my law firm. It only took a few hours a week to do that for my then twenty person law firm, along with the help of an outside “computer repairman.” I would always learn a lot from those guys.

DOS_Screen

compuserve_FTPThe frequency of document reviews that included computer files increased somewhat in the early nineties as law firm clients began using more technologies. By then most corporations and many individuals began to rely on computers for work, although almost nobody but a few techno-nerds used email, electronic messaging and pre-Internet online communities. (I was considered an odd-ball hobbyist for using electronic messaging with CompuServeThe SourceThe Well, etc. in the mid to late eighties, and the Internet since 93-94 with Mosaic, then NetScape.) Instead, facsimile machines were the rage at that time, and they just generated more paper discovery.

Although the presence, or not, of computer files was a discovery issue in trade-secret and non-compete cases in the early 90s, electronic communications discovery was still not a factor. The adoption of tech by businesses and lawyers seemed slow to me then, and still seems slow today. (When will companies and law firms adopt the AI technologies that have been readily available for years now?)

Discovery of computer files, as e-discovery was then called, started to take off in the late nighties as corporate email finally became popular. It was part of the public’s discovery on the Internet. I had the opportunity back in 1996 to write a chapter on Internet law for the then popular book by Macmillan (Que), Special Edition, Using the Internet (3rd Ed. 1996), which is incredibly still sold on Amazon.

Using_Internet_96

My chapter in the book was the first after the introduction and was titled by the editors “Your Cyber Rights and Responsibilities: Law and Etiquette.” I still smile when I see how they tasked me not only with explaining all of the Law of the Internet, but also proper online etiquette. I tried to address the legal issues in 52 pages (I pretty much ignored the etiquette part), including discussion of all of the key cases of the day. I covered things like free speech, online agreements, privacy rights, crime, security and cryptology (I even included a coded message, which surprisingly, the editor decrypted and then made me clean up). These are all still hot issues.

When businesses started using the Internet too, the discovery and review of electronic information really started to take off. That is when electronic document review was truly born. That is also when the first e-discovery vendors like Kroll and Attenex (now FTI) started to become large national organizations.

By early turn of the century potential evidence in the form of computer files and emails were multiplying like tribbles. The amount of electronic  evidence started to explode. It has been a dangerous avalanche of e-discovery overload ever since. The needle in the haystack problem was born that still challenges document review today. See Document Review and Predictive Coding: Video Talks – Part One.

Like several others I sensed the danger in the information explosion, saw how it was overwhelming discovery and making it too expensive. For that reason in 2006, again like several others (although I was the only one in Florida), I stopped practicing as a commercial litigator and limited my work to e-discovery only. Since that time electronic document reviews have been front and center in my practice. To be honest, I have not even seen an original paper document in discovery since that time, although I have heard they still exist. (Other attorneys have shown me their paper cuts to prove it. What a dangerous job paper document reviews can be.)

Five Videos Explain the Five Tips

The five tips shared here are rooted in the ancient history of paper productions, and pre-vendor computer file search, but are designed for current electronic practices and post 2015 amended rules of procedure. After a lifetime of work in this area, there are more tips I could provide, and will do so in the future, I’m sure, but these are the ones that occur to me today. The videos below explain these five tips and how you can implement them.

In this opening eleven-minute video I share what may be the most important tip of all, the avoidance of time pressures and resultant hurried activities.

Tip # 1 – Never Put Yourself in a Time Bind – Be Proactive

________

5-Tips_Review_ETHICSThe next video explains the second tip, Ethics. It is always important to do the right thing, including the production of requested relevant documents that will harm your client and their case.  Ethics is document review, like in all other areas of legal practice, indeed, like all other areas of life, is imperative, not discretionary. My thanks to the legal mentors in my past who drilled this into me from my first day out of law school. Any success I have enjoyed in my career I owe, at least in part, to their good influences.

slippery_slopeCall this Ethics advice the Boy Scout tip if you wish, but it really works to avoid a panoply of errors, including potentially career-ending ones. It also helps you to sleep at night and have a clean conscience. The slippery slopes of morality are where the worst errors are made in all legal tasks, but this is particularly true in document review. Discovery in our system is run by lawyers, not judges, magistrates, or special masters. It is based on lawyers faithful conduct and compliance with the rules, including the all-important rules requiring the voluntary production of evidence harmful to a client (a notion strange to many legal systems outside of the U.S.).

Lawyers know the rules, even if their clients do not, and it is critical that they follow them earnestly, holding up against all pressures and temptations. At the end of the day, your reputation and integrity are all that you have, so compromising your ethics is never an acceptable alternative. The Rules of Professional Conduct must be the guiding star of all legal practice, including electronic document review. It is your job as a lawyer to find the evidence and argue it’s meaning; never to hide it. This video is a reminder of a core truth of lawyer obligations as officers of the court.

Tip #2 – Ethics and Electronic Discovery

For more of Losey’s thoughts on ethics and e-discovery, seeLawyers Behaving Badly: Understanding Unprofessional Conduct in e-Discovery, 60 Mercer L. Rev. 983 (Spring 2009); Mancia v. Mayflower Begins a Pilgrimage to the New World of Cooperation, 10 Sedona Conf. J. 377 (2009 Supp.); e-Discovery for Everyone, Chapters 15-19, (ABA, 2017).

focus2Our third tip is Focused Concentration, which was mentioned in passing in the Part One video on Time, and also tips four and five, on Worms and Check Again. The Focus tip is based on my own experiences in cultivating the ability to concentrate on legal work, or anything else. It is contra to the popular, but erroneous notion, a myth really, that you can multi-task and still do each task efficiently. Our brain does not work that way. See Eg. Crenshaw, The Myth of Multitasking: How “Doing It All” Gets Nothing Done; and the work of neuroscientist Daniel J Levitin, who has found the only exception is adding certain background music. All document reviewers who wear headsets, myself included, know this exception very well.

Tip #3 – Focused Concentration

Steve-Jobs-zenFor more on quality control and improved lifestyle by focused attention and other types of meditation, see my earlier video blog, Document Review and Predictive Coding: Video Talks – Part Six, especially the 600 word introduction to that video that includes information on the regular meditation practices of Supreme Court Justice Stephen Breyer, among others. See A Word About Zen Meditation. This practice helped Steve Jobs, and helps Justice Breyer and countless others. It could help you too. Also see these excellent online services, Insight Timer  and Mindfulnes App. These practices will, at the very least, allow for more focused attention to what you are doing, including document review, and thus greatly reduce mistakes.

The next Worms tip is a simple technical one, unique to e-discovery, where Worm is an acronym that means write once, read many times. I prefer to make productions on write-only or recordable only CDs, aka, CD-R, or DVD-R, and not by file transfers. I do not want to use a CD-RW, or DVD-RW meaning one that is rewritable.

Tip #4 – Use WORMS to Produce

Speaking of WORMs, did you know that the SEC requires all broker-dealers to preserve its records for three years in a format that prevents alteration? That means our Write Once Read Many times format. SEC Interpretation: Electronic Storage of Broker-Dealer Records, 17 CFR Part 241 [Release No. 34-47806] (5/12/13).

On December 21, 2016, twelve large broker-dealer firms agreed to pay fines totaling $14.4 million to the Financial Industry Regulatory Authority (FINRA) over allegations, in FINRA’s words, that “they failed to preserve electronic records in a WORM format that couldn’t be altered.” This has to be the all time most expensive “can of worms.”

The fifth tip of Check Again, has to do with the importance of redundancy in quality control, subject only to proportionality considerations, including the tip to spot check your final production CD. I discuss briefly the tendency of lawyers to be trapped by paralysis by analysis, and why we are sometimes considered deal killers by business people because we focus so much on risk avoidance and over-think things. There has to be a proportional limit on the number and cost of double-checks in document review. I also mention in the fifth tip my Accept of Zero Error and ei-Recall checks, which are quality assurance efforts that we make in larger document review projects.

Tip #5 – Check Again

____________

These are five tips to help everyone doing electronic document review. They are not necessarily the “top five,” but they are all important. We suggest you drill these five best practices into your document review team.

For more information on best practices of document review see these three periodically updated resources:

 

 



Top Twenty-Two e-Discovery Opinions of 2016: Number One

January 1, 2017

Here is the e-Discovery Team’s most interesting e-discovery opinion of 2016: Hyles v. New York City, No. 10 Civ. 3119 (AT)(AJP), 2016 WL 4077114 (S.D.N.Y. Aug. 1, 2016)!!!!! The author is well-known e-discovery expert, Judge Andrew Peck of the SDNY.

ONE – Hyles v. New York City

judge_andy_peckThis opinion, like that of Number Two, Dynamo Holdings, is on the e-Discovery Team’s favorite topic, predictive coding. Admittedly, that had a lot to do with the Team’s pick of Hyles as this year’s most interesting e-discovery opinion. Hyles v. New York City, No. 10 Civ. 3119 (AT)(AJP), 2016 WL 4077114 (S.D.N.Y. Aug. 1, 2016). So too did the fact that it was written by Judge Peck. He is known for his excellent legal analysis, especially on the legal search topics. The Hyles opinion was not only spot on, it had the clarity and writing quality demanded of any opinion to be ranked one in 2016.

Although Hyles did not make new law, it clarified existing law on predictive coding. We like that. The specific issue addressed in Hyles has been discussed before, but never squarely ruled on by Judge Peck. In Hyles the plaintiff wanted to require the defendant City (i.e., the responding party) to use TAR (technology assisted review, aka predictive coding), instead of the method the City preferred of keyword searching. As expected, Judge Peck ruled that a party cannot be forced to do predictive coding, even if it is a better method than what the party wants to do, in this case, keyword search; and, even if the party’s preferred method had not yet started. As expected, the reason for this ruling was old Sedona Principle Six.

We liked the Hyles opinion, over Dynamo Two, because Hyles does not include descriptions of cockamamy methods of predictive coding, like Dynamo does. Instead, Hyles involves a more basic methodology, one faced by most e-discovery practitioners today, not just predictive coding specialists, on how to cull down the ESI universe subject to review for relevance by: (1) custodian priority; and, (2) date range. This is a predictive coding case that covers pre-predictive coding methods. It that sense Hyles is like Judge Peck’s other classic legal search opinion from the pre-predictive coding era, Gross ConstructionWilliam A. Gross Constr. Assocs., Inc. v. Am. Mutual Mfrs. Ins. Co., 256 F.R.D. 134, 134 (S.D.N.Y. 2009).

Here is how Judge Peck described counsels efforts to agree upon a method to cull the universe of documents to be reviewed for relevance.

As to date range, the parties agreed on a start date of September 1, 2005 but disagreed on the end date. … After hearing the parties’ arguments at the conference, the Court ruled that the end date would be April 30, 2010 (when defendant Patricoff was reassigned from her First Deputy Commissioner position), without prejudice to Hyles seeking documents or ESI from a later period, if justified, on a more targeted inquiry basis.

Notice how Judge Peck ruled, but without prejudice for the requesting party to come back later, if need be. Most discovery rulings on issues like that should be open-ended, with the idea that any follow-up requests must be narrow and focused, and thus relatively inexpensive to fulfill.

Judge Peck makes the same kind of ruling at to the total number of custodians whose ESI must be reviewed.

As to custodians, the City agreed to search the files of nine custodians (including Hyles), but not six additional custodians that Hyles requested. (7/18/16 Ltr. at 5, 7.) The Court ruled that discovery should be staged, by starting with the agreed upon nine custodians (Hyles, Stark, Patricoff and six others). After reviewing the production from the nine custodians, if Hyles could demonstrate that other custodians had relevant, unique and proportional ESI, the Court would consider targeted searches from such other custodians.[1]

Here is how Judge Peck quickly frames the dispute that the parties brought to him for resolution. (All record citations omitted.)

After the parties had initial discussions about the City using keywords, Hyles’ counsel consulted an ediscovery vendor and proposed that the City should use TAR as a “more cost-effective and efficient method of obtaining ESI from Defendants.”  The City declined, both because of cost and concerns that the parties, based on their history of scope negotiations, would not be able to collaborate to develop the seed set for a TAR process.

Andrew J. PeckJudge Peck began his analysis as you might expect by agreeing with the plaintiff that “in general, TAR is cheaper, more efficient and superior to keyword searching.” Then he set out the legal precedent history embodying his thinking on predictive coding and whether a party should be required to use TAR against their will.

In March 2009, the “dark ages” in terms of ediscovery advances, this Court described problems with keywords and the need for “careful thought, quality control, testing, and cooperation with opposing counsel in designing search terms or `keywords.'” William A. Gross Constr. Assocs., Inc. v. Am. Mutual Mfrs. Ins. Co., 256 F.R.D. 134, 134 (S.D.N.Y. 2009) (Peck, M.J.). Further elaborating on the deficiencies of keyword searching, my seminal Da Silva Moore decision in 2012 approved the use of predictive coding, aka TAR, in appropriate cases. Da Silva Moore v. Publicis Groupe, 287 F.R.D. 182, 190-91, 193 (S.D.N.Y. 2012) (Peck, M.J.). In again approving the use of TAR in 2015, I wrote that “the case law has developed to the point that it is now black letter law that where the producing party wants to utilize TAR for document review, courts will permit it.” Rio Tinto PLC v. Vale S.A., 306 F.R.D. 125, 127 (S.D.N.Y. 2015) (Peck, M.J.).[3] Dicta in a footnote in Rio Tinto stated that “[i]n contrast, where the requesting party has sought to force the producing party to use TAR, the courts have refused.” Rio Tinto PLC v. Vale S.A., 306 F.R.D. at 127 n.1. “The Court note[d], however, that in [the cited] cases the producing party had spent over $1 million using keyword search (in Kleen) or keyword culling followed by TAR (in Biomet), so it is not clear what a court might do if the issue were raised before the producing party had spent any money on document review.” Rio Tinto PLC v. Vale S.A., 306 F.R.D. at 127 n.1. Since the search methodology issue arose in this case before the City spent much, if any, money on searching for responsive ESI, this case squarely raises the issue of whether the requesting party can have the Court force the responding party to use TAR.

The plaintiff also argued that since parties should cooperate in discovery the City should cooperate and use the best technology available to find relevant evidence. Judge Peck rejected this argument as follows:

Hyles’ counsel is correct that parties should cooperate in discovery. I am a signatory to and strong supporter of the Sedona Conference Cooperation Proclamation, and I believe that parties should cooperate in discovery. See William A. Gross Constr. Assocs., Inc. v. Am. Mfrs. Mut. Ins. Co., 256 F.R.D. at 136; Rio Tinto PLC v. Vale S.A., 306 F.R.D. at 129 n.6. The December 1, 2015 Advisory Committee Notes to amended Fed. R. Civ. P. 1 emphasized the need for cooperation. Cooperation principles, however, do not give the requesting party, or the Court, the power to force cooperation or to force the responding party to use TAR.

His comment that a Court does not have the “power to force cooperation” is also interesting and somewhat controversial. Perhaps it is a matter of semantics, but I have seen many judges “order” parties to cooperate. Still, his second point on cooperation is that the doctrine cooperation does not require a responding party to use TAR if they do not want to. Cooperation does not mean capitulation.

Judge Peck then goes on to articulate the main reason that a judge should not ordinarily force a party to use a particular tool or technique to mine client data for useful evidence. That should be the litigant’s independent duty and the court should not interfere without cause. This is part of what is known as The Sedona Conference Principle Six as is well explained by Judge Peck in Hyles.

It certainly is fair to say that I am a judicial advocate for the use of TAR in appropriate cases. I also am a firm believer in the Sedona Principles, particularly Principle 6, which clearly provides that:

Responding parties are best situated to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own electronically stored information.

The Sedona Principles: Second Edition, Best Practices Recommendations & Principles for Addressing Electronic Document Production, Principle 6 (available at http://www.TheSedonaConference.org).

Under Sedona Principle 6, the City as the responding party is best situated to decide how to search for and produce ESI responsive to Hyles’ document requests. Hyles’ counsel candidly admitted at the conference that they have no authority to support their request to force the City to use TAR. The City can use the search method of its choice. If Hyles later demonstrates deficiencies in the City’s production, the City may have to re-do its search.[4] But that is not a basis for Court intervention at this stage of the case.

Notice how the decision to not compel the use of TAR is without prejudice. The plaintiff can revisit the request by demonstrating deficiencies in the defendant’s production.

Judge Peck then quotes with approval the recent Dynamo Two opinion where Judge Buch held that it was not the court’s business to dictate to attorneys how to do document review, and again relied on Sedona Six. Here is Judge Peck’s concluding words.

Here, too, it is not up to the Court, or the requesting party (Hyles), to force the City as the responding party to use TAR when it prefers to use keyword searching. While Hyles may well be correct that production using keywords may not be as complete as it would be if TAR were used (7/18/16 Ltr. at 4-5), the standard is not perfection, or using the “best” tool (see 7/18/16 Ltr. at 4), but whether the search results are reasonable and proportional. Cf. Fed. R. Civ. P. 26(g)(1)(B).

To be clear, the Court believes that for most cases today, TAR is the best and most efficient search tool. That is particularly so, according to research studies (cited in Rio Tinto), where the TAR methodology uses continuous active learning (“CAL”), which eliminates issues about the seed set and stabilizing the TAR tool. The Court would have liked the City to use TAR in this case. But the Court cannot, and will not, force the City to do so. There may come a time when TAR is so widely used that it might be unreasonable for a party to decline to use TAR. We are not there yet. Thus, despite what the Court might want a responding party to do, Sedona Principle 6 controls. Hyles’ application to force the City to use TAR is DENIED.

Our view of the most interesting opinion of 2916. The e-Discovery Team agrees with Judge Peck that “for most cases today, TAR is the best and most efficient search tool.” To be clear, however, our agreement is predicated upon the TAR tool being used properly. The method of TAR matters. The e-Discovery Team would much rather work on a well-run, well-designed keyword search project, than a mismanaged, poorly designed predictive coding project.

We think the wise words of Judge Facciola in O’Keefe in 2008 about angels have been too early forgotten:

Whether search terms or ‘keywords’ will yield the information sought is a complicated question involving the interplay, at least, of the sciences of computer technology, statistics and linguistics…. Given this complexity, for lawyers and judges to dare opine that a certain search term or terms would be more likely to produce information than the terms that were used is truly to go where angels fear to tread. This topic is clearly beyond the ken of a layman.

peck_facciola

United States v. O’Keefe, 537 F. Supp. 2d 14 (D.C. 2008). What Judge Facciola said about keyword search is as true today as when written. When it comes to legal search today using active machine learning, which is the true meaning of TAR, the expertise required is even greater. Predictive coding requires special skills and a unique knowledge set to do right. It is clearly beyond the ken of almost all attorneys practicing law today. We do not see this gap narrowing, not because the education is not available, but because most lawyers are disinterested. For this reason the competency gap is widening and the problem noted by Judge Facciola in 2008 is still alive and well today.

LoveIn spite of this competency gap, and the stupid fearlessness of many trial lawyers, those who do not even know what they do not know, courts continue to approve the use of TAR carte blanche, with no requirement of expert assistance or use of proven methodologies. For that reason our agreement with Judge Peck on the superiority of predictive coding must be qualified. Still, we agree, because when active machine learning is done right it is a thing of beauty, far more effective than keywords in all but the simplest projects. Why I Love Predictive Coding: Making document review fun with Mr. EDR and Predictive Coding 3.0. (e-discovery team, 2/14/16).

We also agree with Judge Peck’s speculation that there may come a time when a court forces the use of best practices by recalcitrant lawyers. Judge Peck may even reverse himself on this point before TAR is more widely used, especially if  Sedona Principle Six is revised or shown to be inapplicable to a particular case. Is it really true, as Principle Six asserts, that “Responding parties are best situated to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own electronically stored information.” They may be in the best position for preservation, but are they for search and production? Legal search, especially active machine learning, is a speciality well beyond the IT capabilities and skills of responding parties. Legal search does not require special knowledge of the data itself, it requires special knowledge of the procedures, methodologies, and technologies for electronic document review, including especially predictive coding. This is knowledge possessed by e-discovery specialists, by lawyers who specialize in legal search, not by litigant’s IT departments.

It is hard to see how Principle Six applies to choice of document review software and feature utilization. We need to think this through and have a vigorous debate on the continued application of Principle Six to document review defensibility. See Ball, Craig, Sedona Principle Six: Overdue for an Overhaul (10/10/14) (“It’s time to deep six Sedona Six.“)

In the not too remote future, when Hyles is someday reversed (perhaps by Judge Peck himself) and a party is ordered to use TAR, we expect (hope) that the opinion will also specify the particular method or methods of use of TAR. Otherwise, the order is too general to have any meaning. You might as well order an attorney to use a computer to do document review. There are many, many ways to do TAR. Most of them are wrong. Any attorney angel should fear to tread TAR without the help of experts.

If ESI continues to grow more complicated, and the volumes of data continue to explode, then in the future legal search that includes predictive coding may well be the only way document discovery can be conducted. Litigation lawyers of the future may still do depositions, motion practice, trials and the like, but it is unlikely they will also continue to do large volume document review. They will leave that to active machine learning experts. The improvements we see in the use of artificial intelligence and easier-to-use software will help expand the group of experts, the specialists in document review, but it is likely to bring it within the reach of the general litigator.

The Bar is already faced with a large competency gap. A new type of legal work is emerging to fill that competency gap, a new job, where specialists in AI enhanced evidence search handle all document discovery. A dual track for trial preparation is emerging. One group of lawyers will be concerned with electronic document discovery and another group will handle all of the other litigation tasks.

Looking a little further into the future, we expect courts may eventually turn over the entire ESI search and production process over to neutral expert specialists serving as discovery masters (or something like that). That may well be the best means for the just, speedy and efficient resolution of most law suits.

 


%d bloggers like this: