The Importance of Witness Interviews: ‘What Happens in Vegas Shouldn’t Stay in Vegas’

September 16, 2018

A discovery order in Vegas shows the importance of witness interviews and what can happen when you take a cavalier attitude towards preservation. Small v. University Medical CenterCase No. 2:13-cv-0298-APG-PAL (D.C. Nev., 9/9/18) (FLSA class action seeking unpaid wages for skipped meal breaks). The lengthy order is entitled Report and Recommendation and Final Findings of Fact and Conclusions of Law and imposes severe sanctions on the defendant. The order proves, when it comes to e-discovery at least, what happens in Vegas doesn’t stay in Vegas. The truth does and should come out, including where’s the electronic evidence. Interviews are a good way to find out what really happened.

This is a long blog – 5,122 words – but it is still a lot shorter than the 123 page Short opinion, which is not short at all. I counted, it is 48,980 words. Not that I’m complaining, but it is one of the longest discovery orders I have ever read. It has many good instructional elements. Specialists should probably read and skim the whole opinion.

The Short opinion also has the distinction of having taken longer to prepare than any other discovery order I have ever read – FOUR YEARS! Can you imagine any decision taking that long? I am sure there were good reasons, but still. That is a full presidential term.

First Steps of e-Discovery: Prepare and Preserve

The FLSA suit arose from a DOL investigation that faulted the defendant employer hospital, UMC, for failing to keep “accurate records” of the time worked. UMC’s alleged records failures continued after it was sued. They failed to give timely preservation notices and failed to interview key custodians. That’s a failure of the first two legal tasks a lawyer is required to do in Electronic Discovery Best Practices (EDBP), steps two and three (step one is prepare). See EDBP.com (detail shown above right with all ten legal activities shown below); also see: Favro, Phillip, Vegas Court Spotlights the Importance of Custodian Interviews with New ESI Sources (LegalTech News 8/30/18) (further discussed below); John Patzakis, Three Key eDiscovery Preservation Lessons from Small v. University Medical Center (Next Generation eDiscovery Blog, 9/12/18).

Judge Peggy Leen’s Order

Magistrate Judge Peggy A. Leen is the learned judge who wrote the opinion in Small v. University Medical Center, Report and Recommendation and Final Findings of Fact and Conclusions of Law. The order affirms and implements most of the recommendations of the Special Master for e-Discovery appointed several years ago in this case, Daniel Garrie.

The Special Master’s Report was issued four years earlier on August 18, 2014, two years after the suit was filed in July 2012. The Report was notable for characterization of defendant’s discovery misconduct as so egregious as to “shock the conscience” and make “a mockery of the orderly administration of justice.” It was a long, complicated report.

When she completed her work she ruled in large part for the plaintiffs and  sanctioned the defendant:

VI. THE COURT’S FINDINGS AND CONCLUSIONS

The court has personally conducted a thorough review of the record prior to the special master’s appointment and the record of the proceedings conducted by the special master. The record before the court and the record developed by the special master amply supports his findings that UMC destroyed evidence by failing to identify, preserve, collect, process, and search multiple repositories of information relevant to the parties’ claims and defenses.

UMC failed to preserve several different types of ESI, including an estimated 26,000 text messages and 38,000 documents from a shared drive “containing human resources, corporate compliance, employee grievance, payroll, and DOL investigation data.” The documents lost include important policy and procedure manuals regarding meal breaks and compensation. Relevant ESI on laptops, desktops and local drives were not preserved until some 18 months into the litigation. UMC also failed to comply with multiple discovery orders, leading to the plaintiffs’ motions for sanctions.

Judge Leen did not follow the recommendation of the Special Master to impose a sanction of default judgment in favor of 613 class members on the Fair Labor Standards Act claims. Instead, she imposed a permissive adverse inference jury instruction, along with monetary sanctions. These jury instructions can have a profound impact on the jury, but not as  strong as a mandatory adverse inference instruction. The mandatory instruction almost always leads to a verdict against the spoliating party. The permissive kind of instruction imposed here still gives a defendant like UMC a chance. The sanctioned party can still prevail with a jury on the merits of the case, albeit a slim chance. Here is the specific language that Judge Leen suggested be used at trial with the jury:

2. UMC is sanctioned in the form of an instruction to the jury that the court has found UMC failed to comply with its legal duty to preserve discoverable information, failed to comply with its discovery obligations, and failed to comply with a number of the court’s orders. The instruction will provide that these failures resulted in the loss or destruction of some ESI relevant to the parties’ claims and defenses and responsive to plaintiffs’ discovery requests, and that the jury may consider these findings with all other evidence in the case for whatever value it deems appropriate.

Careful study of the long opinion shows a very practical, albeit unstated reason for Judge Leen to make this concession. It made her order much harder to appeal; some would say appeal-proof. (After you put four years into something you want it to last.) That is because near the end of the process at one of the hearings Judge Leen was able to get defendant’s own attorney to concede that an adverse inference jury instruction would be appropriate. You do not see that happen very often. But this attorney apparently saw the writing on the wall from the comments the judge was making and realized that accepting a permissive inference was the best they could hope for and certainly a lot better than default judgments for all 613 class members.

Here is Judge Leen’s explanation of how this admission came about.

During oral argument on its objections to the special master’s R & R, counsel for UMC stated “I’m not even going to tell you that I don’t think we shouldn’t be sanctioned.” (Hr’g Tr. 24:28-25:1, Oct. 21, 2014, ECF No. 229.) When asked what sanction he felt was appropriate based on the developed record, UMC’s counsel suggested that an adverse inference jury instruction would be appropriate. (Tr. 25:4-10.)

Here we see a wise and experienced judge in action. Too bad Peggy Leen retires in 2019.

Judge Leen had good reason under the law to hesitate to enter default judgments on 613 claims, effectively ending the cases except to determine the amount of damages, all without any hearing on the merits of the claims. Entry of the  lesser sanction of a permissive instruction was consistent with Judge Leen’s analysis of Rule 37(b) on sanctions for violation of court orders.

[T]he court cannot conclude that UMC’s multiple discovery failures and failure to comply with the court’s orders threatens to interfere with the rightful decision of this case on the merits.

The lesser sanction was also consistent with her analysis of 2015 revisions to Rule 37(e) on sanctions for ESI spoliation, Rule 1 on just-speedy-inexpensive, and Rule 26(b)(1) on proportionality. Here is Judge Leen’s well-accepted analysis of 37(e):

To summarize, the court may impose sanctions against UMC under the current version of Rule 37(e) only if it finds: (1) UMC failed to preserve ESI “that should have been preserved” in anticipation or conduct of litigation; (2) the information was lost because UMC failed to take reasonable steps to preserve it; (3) the ESI cannot be restored or replaced; and (4) the plaintiffs were prejudiced by the loss. If all of these prerequisites are met, the court may issue sanctions no greater than necessary to cure the prejudice caused by the loss. Only if the court finds UMC acted with intent to deprive may the court impose the most severe sanctions.

Judge Leen then applied the law to the facts.

The court has found that UMC failed to preserve ESI that should have been preserved in anticipation of litigation, and throughout the course of this litigation. The court has also found that the information was lost because UMC failed to take reasonable steps to preserve it. Thousands of text messages on UMC Blackberry devices were lost and cannot be restored. Tens of thousands of files from the Q-Drive were lost and cannot be restored prior to December 2013. . . .

However, the special master’s extraordinary expertise and persistence resulted in restoration, remediation, and production of a great deal of relevant and discoverable ESI. The special master was able to direct restoration of the time tracking systems UMC failed to disclose until near the end of special master proceedings. Fortunately, Jackie Panzeri, UMC’s payroll manager who described herself as a “pack rat” that “keeps documents forever” had a lot of documents on her personal drive and several archives full of emails she did not delete or modify. She was involved in the DOL investigation from the beginning and saved both documents collected and produced to the DOL and for this case. The court is also mindful that ESI is stored in multiple locations and that modified or lost data from the seven key custodians is likely to be found in other locations. . . .

Although the court finds plaintiffs have been prejudiced by the loss of data from key repositories and custodians, the loss has not threatened to interfere with the rightful decision of the case on its merits given the large volume of ESI the special master was able to ensure that UMC produced. For these reasons, the court finds that lesser sanctions are appropriate, proportional, and no greater than necessary to cure the prejudice caused by the loss of ESI uncovered by the special master.

As you can see, hope springs eternal. Judge Leen’s still thinks that the now lost ESI from the seven key custodians is likely to be found in other locations. 

I doubt the Special Master Garrie would share the same optimism. He has already called defendant’s conduct a mockery of the orderly administration of justice. In his Report the Special Master said he has “serious doubts that UMC can complete discovery in a defensible manner going forward without increased candor to the Court and their own counsel, and more competent technical assistance.’ Well, maybe they will change. If not, and Judge Leen is wrong and the missing ESI is not found, then Judge Lee or her successor might reconsider and upgrade the sanction to a mandatory adverse inference. Special Master Garrie may yet get his way.

Defendant’s Threshold Errors

The quotes below from Small summarize the key factual findings of defendants’ threshold errors, the ones that lead to most of the others (emphasis added), much like a domino effect. To me these are the most important errors made and you should study Judge Leen’s words here closely.

D. UMC Executives Failed to Accept Responsibility for Ensuring that ESI was Preserved and Failed to Notify Key Custodians and IT Staff to Preserve, and Prevent Loss, or Destruction of Relevant, Responsive ESI

The record amply supports the special master’s findings that UMC had no policy for issuing a litigation hold, and that no such hold was issued for the first eight months of this litigation until after Mr. Espinoza was deposed on April 8, 2013, and was asked about UMC’s response to plaintiff’s August 6, 2012 preservation letter. The special master accurately found that UMC executives were unaware of their preservation duties, ignored them altogether, or at best addressed them “in the hallway in passing.” . . .

The special master’s finding that UMC executives failed to accept responsibility for their legal duty to preserve is amply supported in the record. UMC executives and counsel failed to communicate with and provide adequate instructions to the department heads and IT personnel of repositories containing discoverable ESI to prevent the loss or destruction of potentially relevant ESI. . . .

There is no evidence in the record, and UMC does not suggest there is any, that current or former counsel gave instructions to UMC to suspend business as usual to prevent the destruction, deletion or modification of ESI responsive to plaintiffs’ discovery requests. . . .

It is also undisputed that UMC’s prior and current counsel failed to conduct timely custodian interviews. Custodian interviews were not conducted until well into the special master proceedings when it became apparent they had not been done. The special master required the interviews to be conducted a second time because the initial custodian interviews conducted by counsel were inadequate. . . .

There is ample support in the record that UMC executives displayed a cavalier attitude about their preservation obligations addressing them in passing, and that UMC executives repeatedly took the position in declarations and testimony that responsibility for preservation was someone else’s job. . . .

The special master correctly found that current and former counsel failed to conduct timely custodian interviews to identify individuals with discoverable information and key repositories of discoverable ESI.

The record in this matter is very complex and voluminous. That is why the Special Master Report and the Order by Judge Leen are so lengthy; 123 pages for the order alone. Suffice it to say, if witness interviews of key custodians been conducted when they should have, shortly after suit was filed, a great deal of relevant evidence that ultimately was lost could have been saved. The Special Master’s detailed findings make that obvious. The lost-files could have been identified and preserved unaltered. Lines of responsibility to comply with legal preservation obligations could have been clarified and enforced. Had these interviews been conducted, and the ESI found quickly, the relevant ESI could have been bulk-collected and the evidence saved from spoliation.

As it is, the actions and mistakes of defendant here have severely weakened their case. That’s what can easily happen when a company has a cavalier attitude to compliance with their legal obligation to preserve potentially relevant ESI.

Eight Failed Challenges to the Special Master’s Report

Judge Leen considered and rejected eight challenges to the Special Master’s report that were raised by the defendant employer, UMC:

  1. Competence and Impartiality of the Special Master, Daniel Garrie.
  2. UMC’s Failure to Comply with the Court’s Orders to Preserve and Produce ESI.
  3. UMC’s Failure Have a Preservation Policy or Litigation Hold Policy and Failure to Timely Implement One.
  4. UMC’s Executives Failure to Accept Responsibility for Ensuring that ESI was Preserved and Failure to Notify Key Custodians and IT Staff to Preserve, and Prevent Loss, or Destruction of Relevant, Responsive ESI.
  5. UMC’s Failure to Disclose the Existence of Relevant ESI Repositories, Including Multiple Timekeeping Systems and the Q-Drive Until Late in the Special Master Proceedings.
  6. UMC Modified, Lost, Deleted and/or Destroyed ESI Responsive to Plaintiffs’ Discovery Requests.
  7. UMC’s Failure to Comply with its Legal Duty to Preserve, Failure to Put in Place a Timely Litigation Hold, Failure to Comply with Multiple Court Orders to Preserve and Produce Responsive ESI, and Loss and Destruction of Responsive ESI (1) Necessitated the Appointment of a Special Master, (2) Caused Substantial Delay of these Proceedings, and (3) Caused Plaintiffs to Incur Needless Monetary Expenses.
  8. The Special Master Correctly Concluded UMC Repeatedly Misrepresented the Completeness of its Production of Documents Produced to DOL; However, UMC Was Not Ordered to Produce Kronos Payroll Data in Spreadsheet Format.

Defendants failed in their challenges to the Special Master’s findings, including the threshold challenge to Special Master Dan Garrie’s competence. Ouch! Garrie is a Senior Managing Partner of Law & Forensics. He has written numerous articles and books on law, technology and e-discovery. See eg. D. Garrie & Yoav Griver. Dispute Resolution and E-Discovery, Thomson Reuters (2nd ed. 2013). Garrie earned a Masters degree in computer science at Brandeis University before going on to law school. A challenge to his expertise was misplaced.

The challenge did not go over well with the supervising Judge who studied his work more closely than anyone. After emphatically rejecting the hospital arguments, Judge Peggy Leen stated:

The court has conducted a de novo review of all of the special master proceedings and finds that he was professional and courteous, if occasionally frustrated by testimony displaying a lack of appreciation of UMC’s legal duties to preserve and produce responsive ESI. He was repeatedly told by UMC executives and employees that they did not know about their duty to preserve, had not learned about their preservation obligations from counsel, did not know what a litigation hold was, and had not explored relevant repositories of information responsive to plaintiffs’ discovery requests.

Bench Slap of Defendant’s Attorneys

With a background like that it is not surprising that the Special Master uncovered so much evidence of incompetence and malfeasance in preserving evidence. Judge Leen held: (emphasis added)

UMC was on notice that its timekeeping, time systems, payroll policies, and procedures were relevant to this litigation. UMC also knew it was unable to document that employees were being compensated for actual time worked. Both UMC and its former and current counsel failed to comply with UMC’s legal duty to suspend routine document retention/destruction policies to ensure the preservation of relevant documents. UMC failed to communicate the need to preserve relevant documents and ESI to employees in possession or likely to be in possession of discoverable information, or for that matter to communicate this duty even to “key players.” UMC and its counsel failed to identify, locate, and maintain information relevant to specific, predictable, and identifiable claims involved in this litigation.

Note that Judge Leen goes out of her way to include the defendant and its lawyers in the blame, both its  prior attorneys and its present attorneys. All of these attorneys failed in the “legal duty to suspend routine document retention/destruction policies to ensure the preservation of relevant documents.” In situations of shared blame like this the attorneys involved are sometimes personally sanctioned along with the client, but this has not happen here. Judge Leen did make several sharp comments against the defendants lawyers, includi9ng this finding:

UMC’s current counsel blamed former counsel and their ESI consultants for the delay in producing responsive ESI. Counsel for UMC advised the court at the hearing on June 25, 2013, that the client did not have any real understanding of what MPP had done or what data had been collected. This representation turned out to be false. . . . Thus, the representation UMC’s current counsel made to the court that the client did not have any real idea of what prior counsel had done regarding ESI collection was patently false. In the light most favorable to current counsel, they did not ask the right questions of the individuals involved in the initial collection. The people involved in the process— MPP, its vendors and consultants, and the IT personnel at UMC who did the collection of ESI from 26 custodians—were simply not asked until after the special master was appointed and made the appropriate inquiries.

You do not see comments like that very often. Basically the judge is saying you lied to me and I cannot trust you. Again, more conscience shocking conduct by these attorneys, well outside the norm of accepted behavior.

Importance and Art of Custodian Interviews

The interviews that eventually were taken under the Special Master’s order and supervision show that critical evidence could have been saved from routine destruction, if the interviews been done at the time the suit was filed, not years later. The interviews would have ensured that preservation notices were properly given, understood and followed, and the right ESI was collected and effectively searched. See William A. Gross Construction Associates, Inc. v. American Manufacturers Mutual Insurance Co., 256 F.R.D. 134, 136 (S.D.N.Y. 2009) (custodian interviews to assist also in keyword search formulation).

It is important to note that the custodian interviews in Small had to be done twice. The attorneys botched the first attempt at witness interviews. They were ordered to do it again. I am not surprised. Many people underestimate the complexity and sophistication of interviews in cases like this. They also underestimate the wiliness of custodians and tendency of some of them to evade questions.

It is very difficult for most attorneys to conduct an interview on the subject of information storage, IT systems, company document storage systems, email, texts, other personal messaging, social media, personal computers, phones, other devices and  software programs used. Questions on these subjects are very different from questions on the merits of a case. A good custodian interview requires special technical knowledge and skills, which, unfortunately, most lawyers still lack. Too bad, because witness interviews are so very important to big cases with complex, messy ESI systems.

Philip Favro, an expert consultant for Driven, Inc., makes this point well in his excellent article on Small:

Fulsome custodian interviews are essential for ensuring that relevant electronically stored information (ESI) is preserved. Such interviews are characterized by exhaustive questioning on any number of topics including traditional and newer sources of ESI.

Properly conducted, custodian interviews should provide counsel with a thorough understanding of the nature and types of relevant information at issue in the litigation, together with the sources where that information is located. If custodian interviews are neglected or deficient, parties are vulnerable to data loss and court sanctions. The Small v. University Medical Center case is instructive on these issues.

Vegas Court Spotlights the Importance of Custodian Interviews with New ESI Sources (LegalTech News 8/30/18).

Phil’s explanation of some of the facts behind the Special Master ordered redo of the interviews shows how difficult some custodian interviews can be, especially when they want to hide something from the lawyers:

Once conducted, the interviews were deemed insufficient by the special master and (later on) the court. In its order, the court spotlighted some of the evasive answers that UMC’s custodians provided. For example, UMC’s director of human resources disclosed the existence of only one relevant timekeeping application despite having approved the use of other timekeeping systems for certain employees. UMC argued that its HR director was only obligated to disclose the timekeeping application he actually used:

[The custodian] did not use those applications himself and therefore had no obligation to disclose these systems in custodian interviews ordered by the special master because a “custodian interview is aimed at uncovering the applications, systems, programs, data with which the actual custodian interfaces.” (emphasis added).

The court decried this limited notion of a custodian interview, observing that it failed to satisfy UMC’s “legal obligation to identify, locate, maintain, and produce relevant information.”

In Small they never did any custodian interviews until after the case blew up and a Special Master was appointed. Even when interviews were finally conducted by defense counsel, they did a poor job; they were not well-informed of the client IT systems and were not “tough enough” with the interviewees. They seemed to be easily deceived and accepted evasive, incomplete answers. You must cross-examine and be the devils advocate for effective interviews, especially when the custodian is evasive.

Favro recommends:

Interviews should go beyond cursory questioning and focus instead on identifying all sources of relevant information. Nor should they be limited to safe topics like “where can relevant messages be found in your email account” or “where are relevant documents stored on your laptop.” Interviews should now include questions regarding the existence of information exchanged through new communications media or stored in online locations . . .

There is an art to interviews like this. The witnesses have to be comfortable telling you the truth, the full truth, without fear of reprisals. Assurances of confidentiality and witness protection can be a good tongue loosener, but do not mislead them. Remind them who you represent, typically at the very beginning.

Trust, friendliness and rapport are important in interviews, but fear has its place too. I like to tell the witness up front how important it is for them to be fully truthful and candid. A short, but stern formal reminder can go a long way if delivered properly. Since interviews are usually not under oath this is especially important. Some formality is important as part of the tongue-loosening process. Moreover, interviews like this are typically done one-on-one with no court reporter and no written statement for the witness to read and sign at the end. An interview is just two people talking, one asking all of the questions, preferably face-to-face and preferably in the witnesses office with their computer equipment at the ready to show you something, if need be.

To encourage full honesty and to help get at the truth I also sometimes inform a witness that they will likely be deposed and subject to intense cross-exam by opposing counsel. (I might possibly exaggerate the adversaries capabilities from time to time.) I point out how it will all be under oath and penalty of perjury. Then I start my role of the devils advocate, saying these are the kind of questions you will be asked, and then tear into them and make sure the story is straight and the memory not too patchy. Hey, do not get mad at me for pressing on you; these are the kind of questions you can expect and we have to be prepared. That works. Fear can be a powerful motivator of truth. So can good cross-exam. The carrot and stick approach is usually effective.

Another important guardian of truth is for the questioning attorneys to be able to look the witness in the eye and follow exactly what they are saying; full technical understanding of the ESI questions. Do not speak the language? Too technical? Then bring a translator, an expert. Do not allow the witness to speak over your head. They may well be bs-ing you. Nodding your head at everything said, even when you do not understand, is a natural lawyer tendency that you must fight against. Do not be afraid to ask stupid questions. When it comes to technical interviews of any kind I interrupt and ask questions all of the time. Much of the language used in tech and e-discovery is vague and subject to multiple meanings. You need to ask questions. Only a fool is afraid to ask questions for fear of seeming foolish.

Good interviews are a best practice to start e-discovery off right and protect clients from wasted expense and unnecessary risks. See the fine article on point by Kelly Twigger, 5 Things A Great Custodian Interview Can Do For Your Case And Your Budget (Above The Law, 6/27/17).

Proper custodian interviews require skill and training. They require the attorney or paralegal doing the interview to have a basic understanding of technology, communications software and social media. It can be challenging in some situations and even advanced practitioners need a good detailed outline to do it right. Make sure your law firm or law department has a good ESI custodian interview outline. I suggest having both a short and long form. These help even experienced lawyers to make sure they do not forget to ask something.

Expert consultants like Kelly Twigger of ESI Attorneys can help you to prepare good outlines and other tools. They can also do the most challenging tasks for you, such as prepare custom Preservation Notices, conduct Custodian Interviews, supervise ESI Collection, attend the 26(f) conference and prepare an ESI discovery plan, and ultimately, document search, review and production. An e-discovery expert can make it far easier and less expensive to stay current with the many technical-legal issues in the field.

A custodian interview can provide a wealth of information to help lawyers to find and save important evidence, but only if done properly by skilled legal practitioners. Do not risk the judge ordering a redo. Make sure you do a proper interview of the key custodians as soon as possible

Conclusion

Small shows what can happen when you take a cavalier attitude towards ESI preservation and interviews. Small v. University Medical CenterCase No. 2:13-cv-0298-APG-PAL, Report and Recommendation and Final Findings of Fact and Conclusions of Law dated August 9, 2018. Preservation errors at the beginning of a case can easily cascade into serious negligence and ESI destruction. This often results in sanctions motions and discovery about discovery. That diverts everyone from the merits of the case. In Small the sanctions not only included a permissive inference jury instruction, but also monetary sanctions, amount yet to be determined. What happened to the defendant in federal court in Vegas in Small is something that you should fear and loathe ever happening to you.

Proper timely custodian interviews could have prevented the loss of data in Small, could have prevented any sanctions. We all know that what happens in Vegas does not stay in Vegas, at least not when discovery in a law suit is concerned. The truth will come out as it should. This is especially true in a case like Small with misconduct that shocks the conscience in a mockery of justice, as Special Master Dan Garrie put it back in 2014.

Early custodian Interviews are an important, well-accepted best-practice, especially in a large matter like Small v. UMC. Interviews are the third step in the ten-step best practices of Electronic Discovery shown below. Electronic Discovery Best Practices (EDBP.com). They are one of three important activities that attorneys must perform in every law suit to preserve potential electronic evidence (shown in blue in the diagram below): hold notices, interviews and ESI collections.

See EDBP on Preservation.

In a large firm like mine, which only does Labor and Employment law, you can use one of the specialists in e-discovery to assist in these tasks, at least until you become proficient on your own. Specialists in large firms are usually experienced attorneys that now limit their work to e-discovery. (I recommend against specializing too early, but some are able to do it effectively.) In my firm there is only one full-time specialist, me, but I have over fifty attorney liaisons to assist. They have special training in e-discovery and are the go-to e-discovery lawyers for their office (we have 50), but they spend most of their time in employment litigation and other services outside of e-discovery. Other large firms have more full-time e-discovery specialists, but fewer part-time specialists. I decided to try to spread out the knowledge.

One of the things a specialists do, full or part-time, is help to create and update good standard witness interview question outlines for use by other attorneys in the firm. For instance, I have both a long and short form that I recently updated. Your firm probably has something similar. If not, do it now. Better late than never.

If you are in a smaller firm and do not have a full-time specialist in your ranks, then you should consider retaining an outside specialist as co-counsel in larger e-discovery matters. They can help you to save on overall costs and, most importantly, prevent a disaster like Small v. University Medical Center from ever darkening your door.

 

 

 


“Save Everything” and Eventually You Will Not Be Able to Find Anything: The Sedona Conference Principles and Commentary on Defensible Disposition

August 13, 2018

If you are a data hoarder, an information pack-rat that saves everything, you will eventually drown in your own data and die. Maybe not literally killed, mind you, but figuratively. Maybe not you personally, but your enterprise, your group, your project, your network. Too much information can render you and your enterprise intellectually paralyzed, cut off and seriously misinformed or uninformed. Saving it all is physically and logistically difficult, if not possible. Even if you could, keeping it all would impede your search, making it hard to find the information you need, when you need it. I address these issues this week in my review of a new commentary by The Sedona Conference Principles and Commentary on Defensible Disposition (August 2018).

Information overload is better than physical death I know, but still very bad in today’s Google world. You end up not being able to find the information you need, when you need it. That makes it hard to determine what really happened. It allows lies and liars to fester and grow. We are now seeing firsthand in the U.S. where this can lead. It is not good. It has put the whole world into a precarious situation. We need the truth to thrive as a culture; not smoke and mirrors, not conman games. A culture built on lies is a cancer. It is a deadly disease, especially for the Law, which depends on truth, on evidence, on real facts, to attain the goal of Justice.

Saving Too Much

Over-retention is the enemy of effective, efficient search. The more ESI there is to search, the more difficult the search. There can be exceptions to this rule, but for the most part it is true. That makes a “save everything” ESI policy an enemy of search. It interferes with the ability to find the information needed, which in my case is electronic evidence in legal proceedings, when it is needed. It is important for these information needs be filled quickly and completely.

Search is powerful. That is my field. The more data the better, is often true, but not always. It depends on the data and its effective life, how long a particular type of data is of any use to anyone. Big data allows for detection of patterns that would otherwise not be seen. This analysis takes CPU power. The advances in this area have been fantastic. We have the processing power, as well as the cheap storage, but our search and retrieval software has not otherwise kept up with the data explosion in volume and complexity. Predictive coding software and other AI applications have come a long way, but are still sometimes confused by the volume, variety and complexity of useless data that plagues most company IT systems.

Retrieval of specific documents and metadata takes time and specialized human skills. The more worthless data in a collection, such as spam, the greater the number false positives in a search, no matter how powerful the algorithms or skilled the searcher. Vast volumes of data make searches longer to execute and less precise. The more noise in the data, the more difficult to hear the signal. That is a fundamental law of information.

With high data volumes you can often still find the signal, the relevant documents that you need in large chaotic data collections, but it takes time and special tools and skills. There are often too many false positives in searches of data collections containing too much spam-like, useless data. Although search is strong, search alone is inadequate to meet the needs of most organizations. They also need data destruction and retention policies that govern all information. That is one reason why the success of information governance depends on data disposition.

An organization should save as much as it needs, but not too much, and also not too little. It is a Goldilocks situation. If you do not save data, you can never find it. If you save too little, then what you later need might not be there to be found. But if you save too much, you may never be able to find what you need. The signal may be in the collection to be found, in plain view, but hidden in the vast numbers, the noise of spam and other irrelevancies.

Search v. Destroy

I have debated Information Governance leaders for years the importance of search versus file destruction. I was pretty much the only advocate for search over disposition. I favored retention over destruction in most close cases, but I had a cost and proportionality overlay. I am reminded, for instance, of my debate with Jason Baron on the subject at the IQPC 10th Anniversary of Information Governance and eDiscovery, where he managed to quote Churchill at the end and won the debate hands-down. e-Disco News, Knowledge and Humor: What’s Happening Today and Likely to Happen Tomorrow (e-Discovery Team, June 7, 2015); Information Governance v Search: The Battle Lines Are Redrawn (e-Discovery Team, Feb. 8, 2015).

I did not consider it a fair debate because of Jason’s very successful pandering to the jury during his closing argument with a quote by Churchill from his speech, We Shall Fight on the Beaches. That’s the one about never surrendering in the fight against “the odious apparatus of Nazi rule” (sadly, this exhortation still has legs today in the US).

The debate was “unfair” primarily because this was an IG conference. Everybody in IG is pro-destruction and values disposition over search. I think most IG leaders go too far, that they are trigger happy to kill data. I pointed out in my debates that once a file is deleted, it cannot be found, no matter how good your filing, no matter how good your search (forensic recovery issues aside).

I am pro-search and think that the importance of management of ESI by filing and disposition is somewhat overblown. I think search is king, not data deletion. Still, even in my most strident of debates and pro-search arguments, I never advocated for the retention of all data. I always assumed that some file disposition was required and accepted that as a given. I was not a save everything and search advocate. I advocated for both, search and destroy. I advocated for more retention than most, but have never argued to retain everything.

There is a common core of agreement that some ESI should be deleted, that all data should not be saved. The disagreement is on how much data to save. How does a person or company know what is the “just right” data destruction policy for that company? There is agreement among experts that there is no one-size-fits-all solution, so custom work is required. Different retention and destruction policies should apply depending on the company and the particularities of their data universe. Many IG specialists advise clients on the custom fit they need. It involves careful investigation of the company, its data and activities, including law suits and other investigations.

The Sedona Conference  Principles and Commentary on Defensible Disposition

Kevin Brady

Kevin Brady

These IG specialists, and the companies they serve, now have an excellent new resource tool to analyze and custom-fit data destruction policies. The Sedona Conference Principles and Commentary on Defensible Disposition (August 2018 Public Comment Version) (Editors-in-Chief, Kevin F. Brady and Dean Kuckelman). I highly recommend this new and excellent work by The Sedona Conferences. My commendations to the Drafting Team: Lauren A. Allen, Jesse Murray, Ross Gotler, Ken Prine, Logan J. Herlinger, David C. Shonka, Mark Kindy; the Drafting Team Leaders: Tara Emory and Becca Rausch; the Staff Editor: Susan McClaim, and Editors-in-Chief, Kevin F. Brady and Dean Kuckelman. Please send to them any comments you may have.

The Commentary begins in usual Sedona fashion by articulation of basic principles and comments tied to principles. The cases and legal authorities cited in all Commentaries by The Sedona Conference are excellent. This commentary on data disposition is no exception. I commend it for your detailed study and reference. Free download here from The Sedona Conference.

The Principles are:

PRINCIPLE 1.    Absent a legal retention or preservation obligation, organizations may dispose of their information.

Comment 1.a.   An organization should, in the ordinary course of business, properly dispose of information that it does not need.

Comment 1.b.   When designing and implementing an information disposition program, organizations should consider the obligation to preserve information that is relevant to the claims and defenses and proportional to the needs of any pending or anticipated litigation.

Comment 1.c. When designing and implementing an information disposition program, organizations should consider the obligation to preserve information that is relevant to the subject matter of government inquiries or investigations that are pending or threatened against the organization.

Comment 1.d.   When designing and implementing an information disposition program, organizations should consider applicable statutory and regulatory obligations to retain information.

PRINCIPLE 2.    When designing and implementing an information disposition program, organizations should identify and manage the risks of over-retention.

Comment 2.a.   Information has a lifecycle, including a time when disposal is beneficial.

Comment 2.b. To determine the “right” time for disposal, risks and costs of retention and disposal should be evaluated.

PRINCIPLE 3.    Disposition should be based on Information Governance policies that reflect and harmonize with an organization’s information, technological capabilities, and objectives.

Comment 3.a.   To create effective information disposition policies, organizations should establish core components of an Information Governance program, which should reflect what information it has, when it can be disposed of, how it is stored, and who owns it.

Comment 3.b. An organization should understand its technological capabilities and define its information objectives in the context of those capabilities.

Document Disposition and Information Governance

The Sedona Conference Principles and Commentary on Defensible Disposition builds upon Sedona’s earlier work, the Sedona Conference Commentary on Information Governance (Oct. 2014). Principle 6 of the Commentary on Information Governance provides the following guidance to organizations:

The effective, timely, and consistent disposal of physical and electronic information that no longer needs to be retained should be a core component of any Information Governance program. The Sedona Conference, Commentary on Information Governance, 15 SEDONA CONF. J. 125, 146 (2014) (“Information Governance” is “an organization’s coordinated, interdisciplinary approach to satisfying information compliance requirements and managing information risks while optimizing information value.” Id. at 126).

The Comment to Principle 6 goes on to explain:

It is a sound strategic objective of a corporate organization to dispose of information no longer required for compliance, legal hold purposes, or in the ordinary course of business. If there is no legal retention obligation, information should be disposed as soon as the cost and risk of retaining the information is outweighed by the likely business value of retaining the information. . . . Typically, the business value decreases and the cost and risk increase as information ages. Id. at 147.

The Sedona Conference concluded in 2018 that this 2014 advice, and similar advice from other sources, has not been followed by most organizations. instead, they continue to struggle to make “effective disposition decisions.” The group in Principles and Commentary on Defensible Disposition concluded in its Introduction that this struggle was caused by many factors, but identified the three main problems:

[T]he incorrect belief that organizations will be forced to “defend” their disposition
actions if they later become involved in litigation. Indeed, the phrase “defensible disposition” suggests that organizations have a duty to defend their information disposition actions. While it is true that organizations must make “reasonable and good faith efforts to retain information that is relevant to claims or defenses,” that duty to preserve information is not triggered until there is a “reasonably anticipated or pending litigation” or other legal demands for records. The Sedona Principles, Third Edition: Best Practices, Recommendations & Principles for Addressing Electronic Document Production, 19 SEDONA CONF. J. 1, 51, Principle 5, 93 (2018).

Another factor in the struggle toward effective disposition of information is the difficulty in appreciating how such disposition reduces costs and risks.

Lastly, many organizations struggle with how to design and implement effective disposition as part of their overall Information Governance program.

The Principles and Commentary on Defensible Disposition attempt to address these three factors and provide guidance to organizations, and the professionals who counsel organizations, on developing and implementing an effective disposition program.

Disposition Challenges

The Sedona Conference Principles and Commentary on Defensible Disposition (August, 2018) concludes by identifying the main challenges to data deletion.

  1. Unstructured Information.
  2. Mergers and Acquisitions.
  3. Departed, Separated, or Former Employees
  4. Shared File Sites
  5. Personally Identifiable Information (“PII”)
  6. Law Firms, eDiscovery Vendors, and Adversaries
  7. In-House Legal Departments
  8. Hoarders (my personal favorite)
  9. Regulations
  10. Cultural Change and Training

There are more, I am sure, but this is a good top ten list to start. I only wish they had included more discussion of these top ten.

Conclusion

Search is still more important for me than destroy. I prefer Where’s Waldo over Kill Waldo! I have not changed my position on that. But neither has mainstream Information Governance. They still disagree with my emphasis on Search. But everyone agrees that we should do both: Search and Destroy. Even I do not want companies to save all of their data. Some data should be destroyed.

I agree with mainstream IG that saving everything forever is not a viable information governance policy, no matter how many resources you also put into ESI search and retrieval. I have never said that you should rely solely on search, just that you should give Search more importance and, when in doubt, that you should save more documents than less. The Search and Destroy argument has always been one of a matter of degree and balance, not whether there should be no destruction at all. The difficult questions involve what should be saved and for how long, which are traditional information management problems.

Where to draw the line on destruction is the big question for everyone. The answer is always company specific, even project specific. It involves questions of varying retention times, files type and custodian analysis. When it comes down to specific decisions, and close questions, I generally favor retention. What may appear to be useless today, may prove to be relevant evidence tomorrow. I hate not being able to prove my case because all of the documents have already been deleted. Then it is just one person’s word against another. IG experts, who usually no longer litigate, or never litigated, do not like my complaints. They are eager to kill, to purge and destroy data. I am more inclined to save and search, but not save too much. It is a question of balance.

Data destruction – the killing of data – can, if done properly, make the search for relevant content much easier. Some disposition of obviously irrelevant, spam and otherwise useless information makes sense on every level. It helps all users of the IT system. It also helps with legal compliance. Too much destruction of data, too aggressive, and you may end up deleting information that you were required by law to keep. You could lose a law suit because of one mistake in a data disposition decision. Where do you draw the line between save and delete? What is the scope of a preservation duty? What files types should be retained? What retention times should apply? How much is too much? Not enough?

The questions go on and on and there is no one right answer. It all depends on the facts and circumstances of the organization and its data. The new Sedona Conference Principles and Commentary on Defensible Disposition is an important new guide to help IT lawyers and technologists to craft custom answers to these questions.

 


Spoliated Schmalz: New Sanctions Case in Chicago That Passes-Over a Mandatory Adverse Inference

March 30, 2018

I am writing today, on Jewish Passover, regarding an opinion that AI found for me, namely a standing search of all of Pacer using Lex Machina. My AI assistant found a sanctions order for spoliation in an employment law case in Chicago. Schmalz v. Village Of North Riverside, et al, No. 1:2013cv08012 – (N.D. Ill., March 23, 2018). The opinion is a Report and Recommendation by U.S. Magistrate Judge Mary Rowland, a rising star in the e-discovery world. Her writing and analysis of Rule 37(e) is excellent. I can assure you that, despite its name, it is not at all schmaltzy. The sanctions granted were good for the moving party, but could have been better, kind of like chicken soup without the matzo balls. Trust me, thanks to my wife, Molly Friedman, I am an expert on that culinary subject.

The lukewarm sanctions in Schmalz were entered against defendants for the inadvertent loss of fifty text messages when key custodian cell phones were not saved after suit was filed. There was no real dispute as to the spoliation, which Judge Rowland called gross negligence. The arguments were about the remedy, the sanctions, if any, to be entered for this spoliation.

The Plaintiff, Schmalz, asked for dispositive sanctions under Rule 37(e)(2), either striking the defense or an adverse inference instruction. Judge Rowland passed-over these harsh sanctions as over-kill. She found that was unnecessary to counteract the prejudice caused to the plaintiff by loss of the text messages. She reached this opinion based on her finding that intentional, or bad faith, destruction of evidence was not shown. Instead, the evidence proved that defendants failure to preserve a few cell phones within their control was gross-negligence, not bad faith. In other words, just doofuses, not bad guys. Judge Rowland did, however, enter sanctions, permitting the plaintiff to present evidence at trial concerning the negligent text message loss. The Order also allows plaintiff to argue to the jury that they should presume that the contents of these texts would be contrary to defense witnesses testimony. Judge Rowland also granted Plaintiff Schmalz a fee award.

At page six of the Schmalz opinion, Judge Rowland explains the background and how the spoliation was discovered by surprising deposition testimony of a key witness:

In his February 2016 deposition, Defendant Niemann revealed that he had “at least 50” text message communications before and after the election with Defendant Hermanek about the police department, who he would promote to the Commander position, why he did not want a lieutenant’s position, and about Plaintiff specifically.

Plaintiff issued a discovery request for the text messages identified in Defendant Neimann’s deposition. (Dkt. 168 at 6). Defendants answered that there were no texts to be produced because “neither defendant Hermanek nor defendant Niemann still possess their cell phones from that time period.” …

Defendants’ duty to preserve the text messages arose as early as August 2013 when they received a litigation hold letter. See (Dkt. 207-1). Further, Defendants admit that they failed to take any steps to preserve the text messages. See (Dkt. 179-2, at 2–3). Likewise, Defendants admit that the text messages have been lost and cannot be replaced by additional discovery as they have exhausted all efforts to retrieve the messages. See (Dkt. 179-2, at 2–3). Given that these predicate elements are met, the Court next determines whether Plaintiff is prejudiced from loss of the text messages. See Fed. R. Civ. P. 37(e)(1).

Judge Rowland then goes on to explain her analysis at pages eight and nine of her opinion.

These text messages are certainly relevant as they involve private communications between the primary defendants and decision-makers in the case during a critical time period, and the alleged subject matter of the text messages involve issues highly pertinent to the underlying claim, including promotions in the police department and the Plaintiff specifically.

Defendants’ argument that Plaintiff is not prejudiced because “there are other means to obtain the contents of the conversations from the defendants, including prior oral discovery and potential trial testimony,” (Def.’s Resp., Dkt. 196 at 5), is unavailing. “A party has the right to prosecute its case in the way it deems fit based on all available relevant evidence.” Larson v. Bank One Corp., No. 00 C 2100, 2005 WL 4652509, at *14 (N.D. Ill. Aug. 18, 2005); see also Hickman v. Taylor, 329 U.S. 495, 507, 67 S. Ct. 385, 392, 91 L. Ed. 451 (1947) (“Mutual knowledge of all the relevant facts gathered by both parties is essential to proper litigation.”). The content of text messages cannot be replaced simply by eliciting testimony from the Defendants, and by having Plaintiff accept that testimony rather than relying on the actual messages to use as they deem fit. Without the lost text messages, Plaintiff is deprived of the opportunity to know “the precise nature and frequency” of those pri-vate communications, which occurred during a critical time period. See Ronnie Van Zant, Inc. v. Pyle, 270 F. Supp. 3d 656, 670 (S.D.N.Y. 2017) (finding prejudice when text messages were lost and “the precise nature and frequency of those communications cannot be verified”). Accordingly, the Court finds that Plaintiff has suffered prejudice as a result of the spoliation of highly relevant text messages. …

Upon a finding of prejudice, a court may order “measures no greater than necessary to cure the prejudice.” Fed. R. Civ. P. 37(e)(1). Under Subdivision (e)(1), the court has much discretion to fashion an appropriate sanction, and “[t]he range of such measures is quite broad if they are necessary for this purpose.” Fed. R. Civ. P. 37(e), 2015 Amendment Advisory Committee Notes.

Judge Rowland then wraps up her analysis at page fifteen of the Schmalz opinion.

Because the Court finds that Defendants acted with gross negligence, which is insufficient to support a finding of intent as required under Rule 37(e)(2), the Court concludes that a lesser sanction under Rule 37(e)(1) is appropriate. … To address the prejudice resulting from Defendant’s spoliation of evidence, the Court recommends that the parties shall be allowed to present evidence to the jury regarding the destruction of the text messages and the likely relevance of the lost information; and that the jury shall be instructed that it may consider this information when making its decision. However, the jury shall not be given specific instructions on any presumption or inference based on the destruction of the text messages.

Conclusion

Do not be a caveman lawyer and forget the cell phones of key custodians. They may need to be preserved, depending on the facts. In Schmalz the cell phones contained key evidence. An interview of the witness at the beginning of the case should have revealed this important fact. The interview should have triggered appropriate preservation. That did not happen here. Judge Rowland found that defendant’s failure was a cave-man lawyer like mistake, grossly negligent and out of touch with 21st Century discovery. For that reason, what appears to have been an easily defendable case has become a nightmare. See Order of Judge Darrah in Schmalz dated October 28, 2016 ($60,000 settlement). The plaintiff can now put on a side-show at trial on cell phone negligence and missing messages. Although better to the defense than a mandatory adverse inference instruction, which is almost certainly a deathblow, this kind of testimony will distract from the otherwise questionable merits of the case.


Document Review and Proportionality – Part Two

March 28, 2018

This is a continuation of a blog that I started last week. Suggest you read Part One before this.

Simplified Six Step Review Plan for Small and Medium Sized Cases or Otherwise Where Predictive Coding is Not Used

Here is the workflow for the simplified six-step plan. The first three steps repeat until you have a viable plan where the costs estimate is proportional under Rule 26(b)(1).

Step One: Multimodal Search

The document review begins with Multimodal Search of the ESI. Multimodal means that all modes of search are used to try to find relevant documents. Multimodal search uses a variety of techniques in an evolving, iterated process. It is never limited to a single search technique, such as keyword. All methods are used as deemed appropriate based upon the data to be reviewed and the software tools available. The basic types of search are shown in the search pyramid.

search_pyramid_revisedIn Step One we use a multimodal approach, but we typically begin with keyword and concept searches. Also, in most projects we will run similarity searches of all kinds to make the review more complete and broaden the reach of the keyword and concept searches. Sometimes we may even use a linear search, expert manual review at the base of the search pyramid. For instance, it might be helpful to see all communications that a key witness had on a certain day. The two-word stand-alone call me email when seen in context can sometimes be invaluable to proving your case.

I do not want to go into too much detail of the types of searches we do in this first step because each vendor’s document review software has different types of searches built it. Still, the basic types of search shown in the pyramid can be found in most software, although AI, active machine learning on top, is still only found in the best.

History of Multimodal Search

Professor Marcia Bates

Multimodal search, wherein a variety of techniques are used in an evolving, iterated process, is new to the legal profession, but not to Information Science. That is the field of scientific study which is, among many other things, concerned with computer search of large volumes of data. Although the e-Discovery Team’s promotion of multimodal search techniques to find evidence only goes back about ten years, Multimodal is a well-established search technique in Information Science. The pioneer professor who first popularized this search method was Marcia J. Bates, and her article, The Design of Browsing and Berrypicking Techniques for the Online Search Interface, 13 Online Info. Rev. 407, 409–11, 414, 418, 421–22 (1989). Professor Bates of UCLA did not use the term multimodal, that is my own small innovation, instead she coined the word “berrypicking” to describe the use of all types of search to find relevant texts. I prefer the term “multimodal” to “berrypicking,” but they are basically the same techniques.

In 2011 Marcia Bates explained in Quora her classic 1989 article and work on berrypicking:

An important thing we learned early on is that successful searching requires what I called “berrypicking.” . . .

Berrypicking involves 1) searching many different places/sources, 2) using different search techniques in different places, and 3) changing your search goal as you go along and learn things along the way. . . .

This may seem fairly obvious when stated this way, but, in fact, many searchers erroneously think they will find everything they want in just one place, and second, many information systems have been designed to permit only one kind of searching, and inhibit the searcher from using the more effective berrypicking technique.

Marcia J. Bates, Online Search and Berrypicking, Quora (Dec. 21, 2011). Professor Bates also introduced the related concept of an evolving search. In 1989 this was a radical idea in information science because it departed from the established orthodox assumption that an information need (relevance) remains the same, unchanged, throughout a search, no matter what the user might learn from the documents in the preliminary retrieved set. The Design of Browsing and Berrypicking Techniques for the Online Search Interface. Professor Bates dismissed this assumption and wrote in her 1989 article:

In real-life searches in manual sources, end users may begin with just one feature of a broader topic, or just one relevant reference, and move through a variety of sources.  Each new piece of information they encounter gives them new ideas and directions to follow and, consequently, a new conception of the query.  At each stage they are not just modifying the search terms used in order to get a better match for a single query.  Rather the query itself (as well as the search terms used) is continually shifting, in part or whole.   This type of search is here called an evolving search.

Furthermore, at each stage, with each different conception of the query, the user may identify useful information and references. In other words, the query is satisfied not by a single final retrieved set, but by a series of selections of individual references and bits of information at each stage of the ever-modifying search. A bit-at-a-time retrieval of this sort is here called berrypicking. This term is used by analogy to picking huckleberries or blueberries in the forest. The berries are scattered on the bushes; they do not come in bunches. One must pick them one at a time. One could do berrypicking of information without  the search need itself changing (evolving), but in this article the attention is given to searches that combine both of these features.

I independently noticed evolving search as a routine phenomena in legal search and only recently found Professor Bates’ prior descriptions. I have written about this often in the field of legal search (although never previously crediting Professor Bates) under the names “concept drift” or “evolving relevance.” See Eg. Concept Drift and Consistency: Two Keys To Document Review Quality – Part Two (e-Discovery Team, 1/24/16). Also see Voorhees, Variations in Relevance Judgments and the Measurement of Retrieval Effectiveness, 36 Info. Processing & Mgmt  697 (2000) at page 714.

SIDE NOTE: The somewhat related term query drift in information science refers to a different phenomena in machine learning. In query drift  the concept of document relevance unintentionally changes from the use of indiscriminate pseudorelevance feedback. Cormack, Buttcher & Clarke, Information Retrieval Implementation and Evaluation of Search Engines (MIT Press 2010) at pg. 277. This can lead to severe negative relevance feedback loops where the AI is trained incorrectly. Not good. If that happens a lot of other bad things can and usually do happen. It must be avoided.

Yes. That means that skilled humans must still play a key role in all aspects of the delivery and production of goods and services, lawyers too.

UCLA Berkeley Professor Bates first wrote about concept shift when using early computer assisted search in the late 1980s. She found that users might execute a query, skim some of the resulting documents, and then learn things which slightly changes their information need. They then refine their query, not only in order to better express their information need, but also because the information need itself has now changed. This was a new concept at the time because under the Classical Model Of Information Retrieval an information need is single and unchanging. Professor Bates illustrated the old Classical Model with the following diagram.

The Classical Model was misguided. All search projects, including the legal search for evidence, are an evolving process where the understanding of the information need progresses, improves, as the information is reviewed. See diagram below for the multimodal berrypicking type approach. Note the importance of human thinking to this approach.

See Cognitive models of information retrieval (Wikipedia). As this Wikipedia article explains:

Bates argues that searches are evolving and occur bit by bit. That is to say, a person constantly changes his or her search terms in response to the results returned from the information retrieval system. Thus, a simple linear model does not capture the nature of information retrieval because the very act of searching causes feedback which causes the user to modify his or her cognitive model of the information being searched for.

Multimodal search assumes that the information need evolves over the course of a document review. It is never just run one search and then review all of the documents found in the search. That linear approach was used in version 1.0 of predictive coding, and is still used by most lawyers today. The dominant model in law today is linear, wherein a negotiated list of keyword is used to run one search. I called this failed method “Go Fish” and a few judges, like Judge Peck, picked up on that name. Losey, R., Adventures in Electronic Discovery (West 2011); Child’s Game of ‘Go Fish’ is a Poor Model for e-Discovery Search; Moore v. Publicis Groupe & MSL Group, 287 F.R.D. 182, 190-91, 2012 WL 607412, at *10 (S.D.N.Y. Feb. 24, 2012) (J. Peck).

The popular, but ineffective Go Fish approach is like the Classical Information Retrieval Model in that only a single list of keywords is used as the query. The keywords are not refined over time as the documents are reviewed. This is a mono-modal process. It is contradicted by our evolving multimodal process, Step One in our Six-Step plan. In the first step we run many, many searches and review some of the results of each search, some of the documents, and then change the searches accordingly.

Step Two: Tests, Sample

Each search run is sampled by quick reviews and its effectiveness evaluated, tested. For instance, did a search of what you expected would be an unusual word turn up far more hits than anticipated? Did the keyword show up in all kinds of documents that had nothing to do with the case? For example, a couple of minutes of review might show that what you thought would be a carefully and rarely used word, Privileged, was in fact part of the standard signature line of one custodian. All his emails had the keyword Privileged on them. The keyword in these circumstances may be a surprise failure, at least as to that one custodian. These kind of unexpected language usages and surprise failures are commonplace, especially with neophyte lawyers.

Sampling here does not mean random sampling, but rather judgmental sampling, just picking a few representative hit documents and reviewing them. Were a fair number of berries found in that new search bush, or not? In our example, assume that your sample review of the documents with “Privileged” showed that the word was only part of one person’s standard signature on every one of their emails. When a new search is run wherein this custodian is excluded, the search results may now test favorably. You may devise other searches that exclude or limit the keyword “Privileged” whenever it is found in a signature.

There are many computer search tools used in a multimodal search method, but the most important tool of all is not algorithmic, but human. The most important search tool is the human ability to think the whole time you are looking for tasty berries. (The all important “T” in Professor Bates’ diagram above.) This means the ability to improvise, to spontaneously respond and react to unexpected circumstances. This mean ad hoc searches that change with time and experience. It is not a linear, set it and forget it, keyword cull-in and read all documents approach. This was true in the early days of automated search with Professor Bates berrypicking work in the late 1980s, and is still true today. Indeed, since the complexity of ESI has expanded a million times since then, our thinking, improvisation and teamwork are now more important than ever.

The goal in Step Two is to identify effective searches. Typically, that means where most of the results are relevant, greater than 50%. Ideally we would like to see roughly 80% relevancy. Alternatively, search hits that are very few in number, and thus inexpensive to review them all, may be accepted. For instance, you may try a search that only has ten documents, which you could review in just a minute. You may just find one relevant, but it could be important. The acceptable range of number of documents to review in Bottom Line Driven Review will always take cost into consideration. That is where Step-Three comes in, Estimation. What will it costs to review the documents found?

Step Three: Estimates

It is not enough to come up with effective searches, which is the goal of Steps One and Two, the costs involved to review all of the documents returned with these searches must also be considered. It may still cost way too much to review the documents when considering the proportionality factors under 26(b)(1) as discussed in Part One of this article. The plan of review must always take the cost of review into consideration.

In Part One we described an estimation method that I like to use to calculate the cost of an ESI review. When the projected cost, the estimate, is proportional in your judgment (and, where appropriate, in the judge’s judgment), then you conclude your iterative process of refining searches. You can then move onto the next Step-Four of preparing your discovery plan and making disclosures of that plan.

Step Four: Plan, Disclosures

Once you have created effective searches that produce an affordable number of documents to review for production, you articulate the Plan and make some disclosures about your plan. The extent of transparency in this step can vary considerably, depending on the circumstances and people involved. Long talkers like me can go on about legal search for many hours, far past the boredom tolerance level of most non-specialists. You might be fascinated by the various searches I ran to come up with the say 12,472 documents for final review, but most opposing counsel do not care beyond making sure that certain pet keywords they may like were used and tested. You should be prepared to reveal that kind of work-product for purposes of dispute avoidance and to build good will. Typically they want you to review more documents, no matter what you say. They usually save their arguments for the bottom line, the costs. They usually argue for greater expense based on the first five criteria of Rule 26(b)(1):

  1. the importance of the issues at stake in this action;
  2. the amount in controversy;
  3. the parties’ relative access to relevant information;
  4. the parties’ resources;
  5. the importance of the discovery in resolving the issues; and
  6. whether the burden or expense of the proposed discovery outweighs its likely benefit.

Still, although early agreement on scope of review is often impossible, as the requesting party always wants you to spend more, you can usually move past this initial disagreement by agreeing to phased discovery. The requesting party can reserve its objections to your plan, but still agree it is adequate for phase one. Usually we find that after that phase one production is completed the requesting party’s demands for more are either eliminated or considerably tempered. It may well now to possible to reach a reasonable final agreement.

Step Five: Final Review

Here is where you start to carry out your discovery plan. In this stage you finish looking at the documents and coding them for Responsiveness (relevant), Irrelevant (not responsive), Privileged (relevant but privileged, and so logged and withheld) and Confidential (all levels, from just notations and legends, to redactions, to withhold and log. A fifth temporary document code is used for communication purposes throughout a project: Undetermined. Issue tagging is usually a waste of time and should be avoided. Instead, you should rely on search to find documents to support various points. There are typically only a dozen or so documents of importance at trial anyway, no matter what the original corpus size.

 

I highly recommend use of professional document review attorneys to assist you in this step. The so-called “contract lawyers” specialize in electronic document review and do so at a very low cost, typically in the neighborhood of $50 per hour.  The best of them, who may often command slightly higher rates, are speed readers with high comprehension. They also know what to look for in different kinds of cases. Some have impressive backgrounds. Of course, good management of these resources is required. They should have their own management and team leaders. Outside attorneys signing Rule 26(g) will also need to supervise them carefully, especially as to relevance intricacies. The day will come when a court will find it unreasonable not to employ these attorneys in a document review. The savings is dramatic and this in turn increases the persuasiveness of your cost burden argument.

Step Six: Production

The last step is transfer of the appropriate information to the requesting party and designated members of your team. Production is typically followed by later delivery of a Log of all documents withheld, even though responsive or relevant. The withheld logged documents are typically: Attorney-Client Communications protected from disclosure under the client’s privilege; or, Attorney Work-Product documents protected from disclosure under the attorney’s privilege. Two different privileges. The attorney’s work-product privilege is frequently waived in some part, although often very small. The client’s communications with its attorneys is, however, an inviolate privilege that is never waived.

Typically you should produce in stages and not wait until project completion. The only exception might be where the requesting party would rather wait and receive one big production instead of a series of small productions. That is very rare. So plan on multiple productions. We suggest the first production be small and serve as a test of the receiving party’s abilities and otherwise get the bugs out of the system.

Conclusion

In this essay I have shown the method I use in document reviews to control costs by use of estimation and multimodal search. I call this a Bottom Line Driven approach. The six step process is designed to help uncover the costs of review as part of the review itself. This kind of experienced based estimate is an ideal way to meet the evidentiary burdens of a proportionality objection under revised Rules 26(b)(1) and 32(b)(2). It provides the hard facts needed to be specific as to what you will review and what you will not and the likely costs involved.

The six-step approach described here uses the costs incurred at the front end of the project to predict the total expense. The costs are controlled by use of best practices, such as contract review lawyers, but primarily by limiting the number of documents reviewed. Although it is somewhat easier to follow this approach using predictive coding and document ranking, it can still be done without that search feature. You can try this approach using any review software. It works well in small or medium sized projects with fairly simple issues. For large complex projects we still recommend using the eight-step predictive coding approach as taught in the TarCourse.com.


%d bloggers like this: