Disproportionate Keyword Search Demands Defeated by Metric Evidence of Burden

June 10, 2018

The defendant in a complex commercial dispute demanded that plaintiff search its ESI for all files that had the names of four construction projects. Am. Mun. Power, Inc. v. Voith Hydro, Inc. (S.D. Ohio, 6/4/18) (copy of full opinion below). These were the four projects underlying the law suit. Defense counsel, like many attorneys today, thought that they had magical powers when it comes to finding electronic evidence. They thought that all, or most all, of the ESI with these fairly common project names would be relevant or, at the very least, worth examining for relevance. As it turns out, defense counsel was very wrong, most of the docs with keyword hits were not relevant and the demand was unreasonable.

The Municipal Power opinion was written by Chief Magistrate Judge Elizabeth A. Preston Deavers of the Southern District Court of Ohio. She reached this conclusion based on evidence of burden, what we like to call the project metrics. We do not know the total evidence presented, but we do know that Judge Deavers was impressed by the estimate that the privilege review alone would cost the plaintiff between $100,000 – $125,000. I assume that estimate was based on a linear review of all relevant documents. That is very expensive to do right, especially in large, diverse data sets with high privilege and relevance prevalence. Triple and quadruple checks are common and are built into standard protocols.

Judge Deavers ruled against the defense on the four project names keywords request, and granted a protective order for the plaintiff because, in her words:

The burden and expense of applying the search terms of each Project’s name without additional qualifiers outweighs the benefits of this discovery for Voith and is disproportionate to the needs of even this extremely complicated case.

The plaintiff made its own excessive demand upon defendant to search its ESI using a long list of keywords, including Boolean logic. The plaintiff’s keyword list was much more sophisticated than the defendants four name search demand. The plaintiff’s proposal was rejected by the defendant and the judge for the same proportionality reason. It kind of looks like tit for tat with excessive demands on both sides. But, it is hard to say because the negotiations were apparently focused on mere guessed-keywords, instead of a process of testing and refining – evolved-tested keywords.

Defense counsel responded to the plaintiff’s keyword demands by presenting their own metrics of burden, including the projected costs of redaction of confidential customer information. These confidentiality concerns can be difficult, especially where you are required to redact. Better to agree upon an alternative procedure where you withhold the entire document and log them with a description. This can be a less expensive alternative to redaction.

When reading the opinion below note how the Plaintiff’s opposition to the demand to review all ESI with the four project names gave specific examples of types of documents (ESI) that would have the names on them and still have nothing whatsoever to do with the parties claims or defenses, the so called “false positives.” This is a very important exercise that should not be overlooked in any argument. I have seen some pretty terrible precision percentages, sometimes as low as two percent.

Get your hands in the digital mud. Go deep into TAR if you need to. It is where the time warps happen and we bend space and time to attain maximum efficiency. Our goal is to attain: (1) the highest possible review speeds (files per hr), both hybrid and human; (2)  the highest precision (% of relevant docs); and, (3) the countervailing goal of total recall (% of relevant docs found). The recall goal is typically given the greatest weight, with emphasis on highly relevant. The question is how much greater weight to give recall and that depends on the total facts and circumstances of the doc review project.

Keywords are the Model T of legal search, but we all start there. It is still a very important skill for everyone to learn and then move on to other techniques, especially to active machine learning.

In some simple projects it can still be effective, especially if the user is highly skilled and the data is simple. It also helps if the data is well known to the searcher from earlier projects. See TAR Course: 8th Class (Keyword and Linear Review).

________________________

Below is the unedited full opinion (very short). We look forward to more good opinions by Judge Deavers on e-discovery.

__________

UNITED STATES DISTRICT COURT FOR THE SOUTHERN DISTRICT OF OHIO, EASTERN DIVISION. No. 2:17-cv-708

June 4, 2018

AMERICAN MUNICIPAL POWER, INC., Plaintiff, vs. VOITH HYDRO, INC., Defendant.

ELIZABETH A. PRESTON DEAVERS, UNITED STATES MAGISTRATE JUDGE. Judge Algenon L. Marbley.

MEMORANDUM OF DECISION

This matter came before the Court for a discovery conference on May 24, 2018. Counsel for both parties appeared and participated in the conference.

The parties provided extensive letter briefing regarding certain discovery disputes relating to the production of Electronically Stored Information (“ESI”) and other documents. Specifically, the parties’ dispute centers around two ESI-related issues: (1) the propriety of a single-word search by Project name proposed by Defendant Voith Hydro, Inc. (“Voith”) which it seeks to have applied to American Municipal Power, Inc.’s (“AMP”) ESI; 1 and (2) the propriety of AMP’s request that Voith run crafted search terms which AMP has proposed that are not limited to the Project’s name. 2 After careful consideration of the parties’ letter briefing and their arguments during the discovery conference, the Court concluded as follows:

  • Voith’s single-word Project name search terms are over-inclusive. AMP’s position as the owner of the power-plant Projects puts it in a different situation than Voith in terms of how many ESI “hits” searching by Project name would return. As owner, AMP has stored millions of documents for more than a decade that contain the name of the Projects which refer to all kinds of matters unrelated to this case. Searching by Project name, therefore, would yield a significant amount of discovery that has no bearing on the construction of the power plants or Voith’s involvement in it, including but not limited to documents related to real property acquisitions, licensing, employee benefits, facility tours, parking lot signage, etc. While searching by the individual Project’s name would yield extensive information related to the name of the Project, it would not necessarily bear on or be relevant to the construction of the four hydroelectric power plants, which are the subject of this litigation. AMP has demonstrated that using a single-word search by Project name would significantly increase the cost of discovery in this case, including a privilege review that would add $100,000 – $125,000 to its cost of production. The burden and expense of applying the search terms of each Project’s name without additional qualifiers outweighs the benefits of this discovery for Voith and is disproportionate to the needs of even this extremely complicated case.
  • AMP’s request that Voith search its ESI collection without reference to the Project names by using as search terms including various employee and contractor names together with a list of common construction terms and the names of hydroelectric parts is overly inclusive and would yield confidential communications about other projects Voith performed for other customers. Voith employees work on and communicate regarding many customers at any one time. AMPs proposal to search terms limited to certain date ranges does not remedy the issue because those employees still would have sent and received communications about other projects during the times in which they were engaged in work related to AMP’s Projects. Similarly, AMP’s proposal to exclude the names of other customers’ project names with “AND NOT” phrases is unworkable because Voith cannot reasonably identify all the projects from around the world with which its employees were involved during the decade they were engaged in work for AMP on the Projects. Voith has demonstrated that using the terms proposed by AMP without connecting them to the names of the Projects would return thousands of documents that are not related to this litigation. The burden on Voith of running AMP’s proposed search terms connected to the names of individual employees and general construction terms outweighs the possibility that the searches would generate hits that are relevant to this case. Moreover, running the searches AMP proposes would impose on Voith the substantial and expensive burden of manually reviewing the ESI page by page to ensure that it does not disclose confidential and sensitive information of other customers. The request is therefore overly burdensome and not proportional to the needs of the case.

1 Voith seeks to have AMP use the names of the four hydroelectric projects at issue in this case (Cannelton, Smithland, Willow and Meldahl) as standalone search terms without qualifiers across all of AMP’s ESI. AMP proposed and has begun collecting from searches with numerous multiple-word search terms using Boolean connectors. AMP did not include the name of each Project as a standalone term.

2 AMP contends that if Voith connects all its searches together with the Project name, it will not capture relevant internal-Voith ESI relating to the construction claims and defenses in the case. AMP asserts Voith may have some internal documents that relate to the construction projects that do not refer to the Project by name, and included three (3) emails with these criteria it had discovered as exemplars. AMP proposes that Voith search its ESI collection without reference to the Project names by using as search terms including various employee and contractor names together with a list of generic construction terms and the names of hydroelectric parts.

IT IS SO ORDERED.

DATED: June 4, 2018

/s/ Elizabeth A. Preston Deavers

ELIZABETH A. PRESTON DEAVERS

UNITED STATES MAGISTRATE JUDGE

 

 


Proportionality Analysis Defeats Motion for Forensic Examination

May 28, 2018

It is rare of a judge to change their mind after making a decision. It is rarer still for a judge to celebrate doing so in a written opinion for the world to see. But that is exactly what Magistrate Judge Jeffrey Cole has done in his opinion dated May 17, 2018 in Motorola Sols., Inc v. Hytera Communications Corp., No. 17 C 1973 (N.D. Ill.).

This celebration is one reason that Judge Cole’s Order denying Motorola’s motion for forensic inspection is so remarkable. Another is that it begins with a quote, a rare occurrence in judicial orders, one that I always like. The quote celebrates the better late than never philosophy of changing your mind to follow a new understanding, a personal wisdom. The quote is by the late, great Supreme Court Justice, Felix Frankfurter. Felix served as a judge on the Supreme Court from 1939 to 1962.  Before that he was, among other things, a Harvard Law Professor and co-founder of the American Civil Liberties Union. Here is the quote with which Cole begins his order:

“Wisdom too often never comes, and so one ought not to reject it merely because it comes late.”

Henslee v. Union Planters Nat. Bank & Trust Co., 335 U.S. 595, 600, 69 S. Ct. 290, 93 L. Ed. 259, 1949-1 C.B. 223 (1949)

(Frankfurter, J., dissenting)

Another unusual thing about Judge Cole’s order, and the real reason I am writing about it, is that the wisdom that came to him was from the doctrine of proportionality and Rule 26(b)(1). This was the basis for Judge Cole to deny plaintiff’s motion for a forensic inspection of defendant’s computers, in China no less.

District Court Judge Ronald Norgle had previously allowed the parties until October 6, 2017, to conduct discovery on the statute of limitations defense only and stayed all other discovery. The parties had one month in which to take discovery on a very limited topic of fraudulent concealment, which is a type of tolling within the doctrine of equitable estoppel of the limitations defense. Nothing else. After all, Motorola has waited almost ten years before filing a trade-secret theft suit against a Chinese corporation for allegedly stealing its radio wave technology. As Judge Cole colorfully described the situation (citations to record removed in all quotes) with a reference to Hannibal:

While the inquiry should have been uncomplicated, it has become a long, drawn out, pitched battle — one, in a rhetorical sense, to rival the Punic Wars — albeit without the elephants and the Alps and the sheer drama.

After that, the parties exchanged motions to compel repeatedly. Deadlines were extended, from one month to several. Thousands of pages of memoranda and exhibits were filed. 1  And, again, this was all over the supposedly limited discovery on a limited topic that ought to have taken little time and effort. The very nature of what occurred tends to sustain the all too prevalent observation that discovery has become more important than the actual case. Bell Atl. Corp. v. Twombly, 550 U.S. 544, 595, n.13, 127 S. Ct. 1955, 167 L. Ed. 2d 929 (2007); A.H. Robins Co. v. Piccinin, 788 F.2d 994, 1013 (4th Cir. 1986).

1 The filings and orders in this case which, again, is only in the preliminary stage of determining whether Motorola’s filing is timely, already cover more than 7,500 pages. See also n. 5, infra.

Five months into this “limited” discovery process Motorola asked to conduct a forensic examination of the computers of “key Hytera witnesses who have been involved in the use of Motorola’s confidential information and any relevant Hytera servers … on which Hytera has stored Motorola documents,” all of which are located in China. Motorola said it wanted to “begin with forensic inspection of the computers” of seven Hytera employees. Judge Cole said this request reminded him of Winston Churchill’s famous quip: “Now this is not the end. It is not even the beginning of the end.”

Flip Flopping Towards Wisdom

After several hearings Judge Cole was persuaded by the siren songs of plaintiff’s counsel from the well-known firm of Kirkland & Ellis LLP. They must have been very good orators and put on a compelling argument to support their motion. They convinced Judge Cole to allow them to begin a forensic examination process in China under elaborate Hague Convention procedures. Only after the hearings and oral decision to compel the inspection did Judge Cole realize the error of that decision. Judge Cole to his credit does not blame Kirkland and Ellis litigators for leading him astray. Following standard judicial protocol Judge Cole assumed full responsibility for the initial error:

Over the course of two lengthy hearings on March 21 and April 4, 2018, I tentatively concluded that forensic examination of Hytera’s computers would be appropriate, but only if the parties could arrive at a suitable protocol that would not, among other things, run afoul of Chinese law. As we discuss, infra at 5, that was a mistake. But the law frowns on relying on a blunder to gain an opportunistic advantage. Cf. Architectural Metal Systems, Inc. v. Consolidated Systems, Inc., 58 F.3d 1227, 1231 (7th Cir.1995); Market Street Associates; Packer Trading Co. v. CFTC, 972 F.2d 144, 150 (7th Cir 1992); Centex Construction v. James, 374 F.2d 921, 923 (8th Cir.1967). 2

2 We should not be understood as ascribing fault to plaintiff’s counsel. After all, in our adversary system, lawyers properly play a partisan role. Masias v. Secretary of Health and Human Svcs, 2009 U.S. Claims LEXIS 281, at *27 (Fed. Cl. 2009); Philips Medical Systems Intern. B.V. v. Bruetman, 8 F.3d 600, 606 (7th Cir.1993) (Posner, J.). See also Smith v. Robbins, 528 U.S. 259, 293, 120 S. Ct. 746, 145 L. Ed. 2d 756 (2000) (Souter, J., dissenting) (“a partisan scrutiny of the record and assessment of potential issues, goes to the irreducible core of the lawyer’s obligation to a litigant in an adversary system … .”). Mistakes are ultimately (and in most cases) the responsibility of the court.

Judge Cole went on to celebrate a jurists right to change their mind in order to get things right.

The scope of discovery that I was initially inclined to allow was, in the context of the present inquiry that had been narrowed by the district court to the limitations issue, overbroad. What is being sought goes beyond the issue of equitable tolling. In the end, Motorola’s counsel and I were talking about relevance to allegations in Motorola’s complaint. And so, well beyond the statute of limitations, by the end of the April 4 hearing, discovery was encompassing documents related to Motorola’s entire case.

As we have said, “all judges make mistakes,” Fujisawa Pharm. Co., 115 F.3d at 1339, and, when possible, it is best that judges put them right.

Proportionality Applied to Restrain Discovery

In the May 17, 2018 order Judge Cole found the wisdom to say no and forbid the forensic examination of the computers in China. He did so because he found that this discovery was “out of proportion with the needs of this case, as presently limited by the district court” and cited Rule 26 (b)(1), Federal Rules of Civil Procedure. Although I am sure that he heard extensive argument and evidence concerning the estimated costs and burdens imposed by the forensic exams, his decision did not focus on costs. Instead it focused on one of the other very important factors in 26(b)(1), “the importance of the discovery in resolving the issues.” Judge Cole realized that the computers in China could not possibly have information in them of any real relevance to equitable tolling of the statute of limitations defense.

At a minimum, even if relevant to the present limited issue, discovery of computers in China is not proportional to the importance of discovery in resolving the issues and the burden and expense of the proposed discovery manifestly outweighs its likely benefit to the very limited question of equitable tolling. Although the federal discovery rules are permissive, they are not, as Judge Moran wisely put it, “a ticket to an unlimited … exploration of every conceivable matter that captures an attorney’s interest.” Sapia v. Bd. of Educ. of the City of Chi., 2017 U.S. Dist. LEXIS 73153, 2017 WL 2060344, at *2 (N.D. Ill. 2017); see also Leibovitch v. Islamic Republic of Iran, 2018 U.S. Dist. LEXIS 31713, 2018 WL 1072567, at *11 (N.D. Ill. 2018). “[J]udges should not hesitate to exercise appropriate control over the discovery process.” Herbert v. Lando, 441 U.S. 153, 177, 99 S. Ct. 1635, 60 L. Ed. 2d 115 (1979). Failure to exercise that control results in needless and enormous costs to the litigants and to the due administration of justice.

Judge Cole also understood that a forensic inspection is a drastic remedy that requires good cause not shown by plaintiff here:

The original idea here was for a month or so of discovery focused on the very limited issue of the statute of limitations. While it is rare for parties to complete discovery even by dates chosen by their counsel, there can be no dispute that things have already gone far beyond what was intended and what was necessary in the statute of limitations portion of this case, in terms of time and scope. Now, Motorola wants things to go very much further. Forensic examination is generally regarded as a drastic step even in general discovery. See, e.g.,John B. v. Goetz, 531 F.3d 448, 460 (6th Cir. 2008) (“mere skepticism that an opposing party has not produced all relevant information is not sufficient to warrant drastic electronic discovery measures.”). As the court said in In re Ford Motor Company, 345 F.3d 1315 (11th Cir. 2003):

““In the absence of a strong showing that the responding party has somehow defaulted in this obligation, the court should not resort to extreme, expensive, or extraordinary means to guarantee compliance. Forensic inspection of computer hard drives is an expensive process, and adds to the burden of litigation for both parties, as an examination of a hard drive by an expert automatically triggers the retention of an expert by the responding party for the same purpose. Furthermore, examination of a hard drive inevitably results in the production of massive amounts of irrelevant, and perhaps privileged, information … . This court is therefore loathe to sanction intrusive examination of an opponent’s computer as a matter of course, or on the mere suspicion that the opponent may be withholding discoverable information.”

The Advisory Committee Notes to Rule 34 recognize that courts must use caution in evaluating requests to inspect an opposing party’s electronic devices or systems for ESI, in order to avoid unduly impinging on a party’s privacy interests:

Inspection or testing of certain types of electronically stored information or of a responding party’s electronic information system may raise issues of confidentiality or privacy. The addition of testing and sampling to Rule 34(a) with regard to documents and electronically stored information is not meant to create a routine right of direct access to a party’s electronic information system, although such access might be justified in some circumstances. Courts should guard against undue intrusiveness resulting from inspecting or testing such systems.

Fed. R. Civ. P. 34, Advisory Committee Notes—2006 Amendment (emphasis added). Likewise, the Sedona Principles urge general caution in this area:

Civil litigation should not be approached as if information systems were crime scenes that justify forensic investigation at every opportunity to identify and preserve every detail … . [M]aking forensic image backups of computers is only the first step of an expensive, complex, and difficult process of data analysis that can divert litigation into side issues and satellite disputes involving the interpretation of potentially ambiguous forensic evidence.

The Sedona Principles, supra, at 34, 47. 4

Conclusion

Judge Cole’s wrap up is wise and witty and something you may want to quote in many discovery disputes, especially the footnote:

Parties are entitled to a reasonable opportunity to investigate the relevant facts — and no more. Upjohn Company v. United States, 449 U.S. 383, 390, 101 S. Ct. 677, 66 L. Ed. 2d 584 (1981); Vakharia v. Swedish Covenant Hosp., 1994 U.S. Dist. LEXIS 2712, at *2 (N.D. Ill. 1994) (Moran, J.). Motorola has already had that reasonable opportunity and far more. What was intended to be a month-long process of discovery on a very limited issue has turned into a protracted affair in which Motorola has received 700,000 documents — nearly 3 million pages — over a period of eight months. 5

5 A ream of paper is 500 sheets, which is 2 inches tall. Three million pages is 6,000 reams, meaning that 3 million pages of discovery, which is about 1,000 feet high or 100 stories high. By any measure, that is extraordinary.

Yet, apparently for Motorola, it’s not enough. It now wants a forensic inspection of several computers in China — and it warns that that is only the “beginning.” What should have been limited discovery on a “straightforward [issue has] spiral[ed] out of control.” Montanez v. Simon, 755 F.3d 547, 552 (7th Cir. 2014). The time has come to say: “enough is enough.” Walker v. Sheahan, 526 F.3d 973, 981 (7th Cir. 2008). Eight months of “limited,” single-issue discovery are now at an end. Motorola’s motion for forensic inspection is denied.

Enough is enough. To go further would have been a disproportionate burden, especially considering the very narrow issue allowed in discovery. Judge Cole at first made a mistake, and then he changed his mind and made it right. He is a wise judge. I wish there were more like him. Except of course if you change your mind to rule against me! <‘_’>


Project Cost Estimation Is Key To Opposing ESI Discovery as Disproportionately Burdensome Under Rule 26(b)(1)

May 6, 2018

If you are opposing ESI discovery as over-burdensome under Rule 26(b)(1), then you MUST provide evidence of the economic burden of the requested review. You cannot just say it is over-burdensome. Even if it seems obvious, you must provide some metrics, some data, some hard evidence to back that up. That requires the ability to estimate the costs and burdens involved in a document review. In the old days, the nineties, almost every litigator could estimate the cost of a paper review. It was not a tough skill. But today, where large volumes of ESI are common, everything is much more complicated. Today you need an expert to accurately and reliably estimate the costs of various types of ESI reviews.

Requiring proof of burden is nothing new to the law, yet most lawyers today need outside help to do it, especially in large ESI projects. For example, consider the defense team of lawyers representing the City of Chicago and other defendants in a major civil rights case with lots of press, Mann v. City of Chicago, Nos. 15 CV 9197, 13 CV 4531, (N.D. Ill. Sept. 8, 2017); Chicago sued for ‘unconstitutional and torturous’ Homan Square police abuse (The Guardian, 10/19/15). They did not even attempt to estimate the costs of the review they opposed. They also failed or refused to hire an expert who could do that for them. Sine they had no evidence, not even an estimate, their argument under Rule 26(b)(1) failed miserably.

Mann v. City of Chicago: Case Background

The background of the case is interesting, but I won’t go into the fact details here; just enough to set up the discovery dispute. Plaintiffs in later consolidated cases sued the City of Chicago and the Chicago police alleging that they had been wrongfully arrested, detained and abused at “off the books” detention centers without access to an attorney. Aside from the salacious allegations, it does not look like the plaintiffs have a strong case. It looks like a fishing expedition to me, in more ways than one as I will explain. With this background, it seems to me that if defendants had made any real effort to prove burden here, they could have prevailed on this discovery dispute.

The parties agreed on the majority of custodians whose ESI would be searched, but, as usual, the plaintiffs’ wanted more custodians searched, including that of the mayor himself, Rahm Emanuel. The defendants did not want to include the mayor’s email in the review. They argued, without any real facts showing burden, that the Mayor’s email would be irrelevant (a dubious argument that seemed to be a throw-away) and too burdensome (their real argument).

Here is how Magistrate Judge Mary M. Rowland summarized the custodian dispute in her opinion:

Plaintiffs argue Mayor Emanuel and ten members of his senior staff, including current and former chiefs of staff and communications directors are relevant to Plaintiffs’ Monell claim. (Id. at 5).[2] The City responds that Plaintiffs’ request is burdensome, and that Plaintiffs have failed to provide any grounds to believe that the proposed custodians were involved with CPD’s policies and practices at Homan Square. (Dkt. 74 at 1, 6). The City proposes instead that it search the two members of the Mayor’s staff responsible for liasoning with the CPD and leave “the door open for additional custodians” depending on the results of that search. (Id. at 2, 4).[3]

Another Silly “Go Fish” Case

As further background, this is one of those negotiated keywords Go Fish cases where the attorneys involved all thought they had the magical powers to divine what words were used in relevant ESI. The list is not shared, but I bet it included wondrous words like “torture” and “off the books,” plus every plaintiff’s favorite “claim.”

The parties agreed that the defendants would only review for relevant evidence the ESI of the custodians that happened to have one or more of the keyword incantations they dreamed up. Under this still all to common practice the attorneys involved, none of whom appear to have any e-discovery search expertise, the majority of documents in the custody of the defense custodians would never be reviewed. They would not be reviewed because they did not happen to have a “magic word” in them. This kind of untested, keyword filtering agreement is irrational, archaic and not a best practice in any but small cases, but that is what the attorneys for both sides agreed to. They were convinced they could guess that words were used by police, city administrators and politicians in any relevant document. It is a common delusion facilitated by Google’s search of websites.

When will the legal profession grow up and stop playing Go Fish when it comes to a search for relevant legal evidence? I have been writing about this for years. Losey, R., Adventures in Electronic Discovery (West 2011); Child’s Game of ‘Go Fish’ is a Poor Model for e-Discovery Search. Guessing keywords does not work. It almost always fails in both precision and recall. The keyword hits docs are usually filled with junk and relevant docs often used unexpected language, not to mention abbreviations and spelling errors. If you do not at least test proposed keywords on a sample custodian, then your error rate will multiply. I saw a review recently where the precision rate on keywords was only six percent, and that is with superficial feedback, i.w. – unskilled testing. You never want to waste so much attorney time, even if you are reviewing at low rates. The ninety-four irrelevant docs to find six is an inefficient expensive approach. We try to improve precision without a significant loss of recall.

When I first wrote about Go Fish and keywords back in 2010 most everyone agreed with me, even if they disagreed on the significance, the meaning and what you should do about it. That started the proportionality debate in legal search. E-Discovery search expert Judges Peck and Scheindlin joined in the chorus of criticism of negotiated keywords. National Day Laborer Organizing Network v. US Immigration and Customs Enforcement Agency, 877 F.Supp.2d 87 (SDNY, 2012) (J. Scheindlin) (“As Judge Andrew Peck — one of this Court’s experts in e-discovery — recently put it: “In too many cases, however, the way lawyers choose keywords is the equivalent of the child’s game of `Go Fish’ … keyword searches usually are not very effective.” FN 113“); Losey, R., Poor Plaintiff’s Counsel, Can’t Even Find a CAR, Much Less Drive One (9/1/13). Don’t you love the quote within a quote. A rare gem in legal writing.

Judge Rowland’s Ruling

I have previously written about the author of the Mann v. City of Chicago opinion, Judge Mary Rowland. Spoliated Schmalz: New Sanctions Case in Chicago That Passes-Over a Mandatory Adverse Inference. She is a rising star in the e-discovery world. Judge Rowland found that the information sought from the additional custodians would be relevant. This disposed of the defendants first and weakest argument. Judge Rowland then held that Defendants did not meet the burden of proof “—failing to provide even an estimate—” and for that reason granted, in part, Plaintiffs’ motion to compel, including their request to add the Mayor. Judge Rowland reviewed all six of the proportionality factors under Rule 26(b)(1), including the importance of the issues at stake and the plaintiffs’ lack of access to the requested information.

On the relevance issue Judge Rowland held that, in addition to the agreed-upon staff liaisons, the Mayor and his “upper level staff” might also have relevant information in their email. As to the burden argument, Judge Rowland held that the City did not “offer any specifics or even a rough estimate about the burden.” Judge Rowland correctly rejected the City’s argument that they could not provide any such information because “it is impossible to determine how many emails there may be ‘unless the City actually runs the searches and collects the material.’” Instead, the court held that the defendants should have at least provided “an estimate of the burden.” Smart Judge. Here are her words:

The City argues that it will be “burdened with the time and expense of searching the email boxes of nine (9) additional custodians.” (Dkt. 74 at 5). The City does not offer any specifics or even a rough estimate about the burden. See Kleen Prods. LLC 2012 U.S. Dist. LEXIS 139632, at *48 (“[A] party must articulate and provide evidence of its burden. While a discovery request can be denied if the `burden or expense of the proposed discovery outweighs its likely benefit,’ Fed. R. Civ. P. 26(b)(2)(C)(iii), a party objecting to discovery must specifically demonstrate how the request is burdensome.”) (internal citations and quotations omitted).

As the Seventh Circuit stated in Heraeus Kulzer, GmbH, v. Biomet, Inc., 633 F.3d 591, 598 (7th Cir. 2011):

[The party] could have given the district court an estimate of the number of documents that it would be required to provide Heraeus in order to comply with the request, the number of hours of work by lawyers and paralegals required, and the expense. A specific showing of burden is commonly required by district judges faced with objections to the scope of discovery . . . Rough estimates would have sufficed; none, rough or polished, was offered.

The City argues in its sur-reply that it is impossible to determine how many emails there may be “unless the City actually runs the searches and collects the material.” (Dkt. 78-1 at 4). Still, the City should have provided an estimate of the burden. The Court is not convinced by the City’s argument about the burden.

Judge Rowland also held that the City should have addressed the “other Rule 26 factors—the importance of the issues and of the discovery in resolving the issues, and the parties’ relative access to information and their resources.” She noted that these other factors: “weigh[ed] in favor of allowing discovery of more than just the two custodians proposed by the City.”  However, the court declined to compel the search of four proposed custodians based on their “short tenure” or the “time during which the person held the position,” concluding the requested searches were “not proportional to the needs of the case.”

Judge Rowland’s opinion notes with seeming surprise the failure of the City of Chicago to provide any argument at all on the five non-economic factors in Rule 26(b)(1). I do not fault them for that. Their arguments on these points were necessarily weak in this type of case, but a conciliatory gesture, a polite acknowledgement showing awareness, might have helped sweeten the vinegar. As it is, they came across as oblivious to the full requirements of the Rule.

What Chicago Should Have Done

What additional information should the defendants have provided to oppose the search and review of the additional nine custodians, including the Mayor’s email? Let’s start with the obvious. They should have shared the total document count and GB size of the nine custodians, and they should have broken that information down on a per-custodian basis. Then they should have estimated the costs to review that many emails and attachments.

The file count information should have been easy to ascertain from the City’s IT department. They know the PST sizes and can also determine, or at least provide a good estimate of the total document count. The problem they had with this obvious approach is that they wanted a keyword filter. They did not want to search all documents of the custodians, only the ones with keyword hits. Still, that just made the process slightly more difficult, not impossible.

Yes, it is true, as defendant’s alleged, that to ascertain this supporting information, they would have to run the searches and collect the material. So what? Their vendor or Chicago IT department should have helped them with that. It is not that difficult or expensive to do. No expensive lawyer time is required. It is just a computer process. Any computer technician could do it. Certainly any e-discovery vendor. The City could easily have gone ahead and done the silly keyword filtering and provide an actual file count. This would have provided the City some hard facts to support their burden argument. It should not be that expensive to do. Almost certainly the expense would have been less than this motion practice.

Alternatively, the City could have at least estimated the file count and other burden metrics. They could have made reasonable estimated based on their document review experience in the case so far. They had already reviewed uncontested custodians under their Go Fish structure, so they could have made projections based on past results. Estimates made by projections like this would probably have been sufficient in this case and was certainly better than the track they chose, not providing any information at all.

Another alternative, the one that would have produced the most persuasive evidence, would be to load the filtered ESI of at least a sample of the nine custodians, including the Mayor. Then begin the review, say for a couple of days, and see what that costs. Then project those costs for the rest of the review and rest of the custodians. By this gold standard approach you would not only have the metrics from the data itself — the file counts, page counts, GB size — but also metrics of the document review, what it costs.

You would need to do this on the Mayor’s email separately and argue this burden separately. The Mayor’s email would likely be much more expensive to review than any of the other custodians. It would take attorneys longer to review his documents. There would be more privileged materials to find and log and there would be more redactions. It is like reviewing a CEO’s email. If the attorneys for the City had at least begun some review of Emanuel’s email, they would have been able to provide extensive evidence on the cost and time burden to complete the review.

I suspect the Mayor was the real target here and the other eight custodians were of much less importance. The defense should have gauged their response accordingly. Instead, they did little or nothing to support their burdensome argument, even with the Mayor’s sensitive government email account.

We have a chance to learn from Chicago’s mistake. Always, at the very least, provide some kind of an estimate of the burden. The estimate should include as much information as possible, including time and costs. These estimates can, with time and knowledge, be quite accurate and should be used to set budgets, along with general historical knowledge of costs and expenses. The biggest problem now is a shortage of experts on how to properly estimate document review projects, specifically large ESI-only projects. I suggest you consult with such an cost-expert anytime you are faced with a disproportionate ESI review demands. You should do so before you make final decisions or reply in writing.

 Conclusion

Mann v. City of Chicago is one of those cases where we can learn from the mistakes of others. At least provide an estimate of costs in every dispute under Rule 26(b)(1). Learn to estimate the costs of document reviews. Either that or hire an expert who can do that for you, one that can provide testimony. Start with file counts and go from there. Always have some metrics to back-up your argument. Learn about your data. Learn what it will likely cost to review that data. Learn how to estimate the costs of document reviews. It will probably be a range. The best way to do that is by sampling. With sampling you at least start the document review and estimate total costs by projection of what it has actually cost to date. There are fewer speculative factors that way.

If you agree to part of the review requested, for instance to three out of ten custodians requested, then do that review and measure its costs. That creates the gold standard for metrics of burden under Rule 26(b)(1) and is, after all, required in any objections under Rule 34(b)(2)(B)&(C). See: Judge Peck Orders All Lawyers in NY to Follow the Rules when Objecting to Requests for Production, or Else ….

For more on cost burden estimation listen to my Ed-Talk on the subject, Proportional Document Review under the New Rules and the Art of Cost Estimation.

 


Evidence Code Revisions and the Grimm/Brady Evidence Admissibility Chart

April 22, 2018

Great fanfare was provided for  the changes to the Federal Rules of Civil Procedure in December 2015. But not much attention has been given to the December 2017 changes to the Federal Rules of Evidence. Maybe that has to do with the disappearing trial, the fact that less than one percent of federal cases actually go to trial. Still, you need to know the rules of evidence admissibility, even if you are preparing for a trial that will never come. You need to collect and discover evidence in a way that it can be used, even if it is just in a motion for summary judgment.

Two New Subsections to Rule 902 on Self-Authenticating Evidence

In December 2017 two new subsections were added to Evidence Rule 902, subsections (13) and (14). They are designed to streamline authentication of electronically stored information (ESI). The goal is to eliminate the need to call a witness at trial to authenticate evidence, at least in most instances. Here are the two new provisions:

Rule 902. Evidence That Is Self-Authenticating

The following items of evidence are self-authenticating; they require no extrinsic evidence of authenticity in order to be admitted: . . .

(13) Certified Records Generated by an Electronic Process or System. A record generated by an electronic process or system that produces an accurate result, as shown by a certification of a qualified person that complies with the certification requirements of Rule 902(11) or (12). The proponent must also meet the notice requirements of Rule 902(11).

(14) Certified Data Copied from an Electronic Device, Storage Medium, or File. Data copied from an electronic device, storage medium, or file, if authenticated by a process of digital identification, as shown by a certification of a qualified person that complies with the certification requirements of Rule (902(11) or (12). The proponent also must meet the notice requirements of Rule 902 (11).

The Evidence Rules Committee Notes explain the background of these two new subsections.

Committee Notes on Rules—2017 Amendment

Paragraph (14). The amendment sets forth a procedure by which parties can authenticate data copied from an electronic device, storage medium, or an electronic file, other than through the testimony of a foundation witness. As with the provisions on business records in Rules 902(11) and (12), the Committee has found that the expense and inconvenience of producing an authenticating witness for this evidence is often unnecessary. It is often the case that a party goes to the expense of producing an authentication witness, and then the adversary either stipulates authenticity before the witness is called or fails to challenge the authentication testimony once it is presented. The amendment provides a procedure in which the parties can determine in advance of trial whether a real challenge to authenticity will be made, and can then plan accordingly.

Today, data copied from electronic devices, storage media, and electronic files are ordinarily authenticated by “hash value”. A hash value is a number that is often represented as a sequence of characters and is produced by an algorithm based upon the digital contents of a drive, medium, or file. If the hash values for the original and copy are different, then the copy is not identical to the original. If the hash values for the original and copy are the same, it is highly improbable that the original and copy are not identical. Thus, identical hash values for the original and copy reliably attest to the fact that they are exact duplicates. This amendment allows self-authentication by a certification of a qualified person that she checked the hash value of the proffered item and that it was identical to the original. The rule is flexible enough to allow certifications through processes other than comparison of hash value, including by other reliable means of identification provided by future technology.

Nothing in the amendment is intended to limit a party from establishing authenticity of electronic evidence on any ground provided in these Rules, including through judicial notice where appropriate.

A proponent establishing authenticity under this Rule must present a certification containing information that would be sufficient to establish authenticity were that information provided by a witness at trial. If the certification provides information that would be insufficient to authenticate the record of the certifying person testified, then authenticity is not established under this Rule.

The reference to the “certification requirements of Rule 902(11) or (12)” is only to the procedural requirements for a valid certification. There is no intent to require, or permit, a certification under this Rule to prove the requirements of Rule 803(6). Rule 902(14) is solely limited to authentication, and any attempt to satisfy a hearsay exception must be made independently.

A certification under this Rule can only establish that the proffered item is authentic. The opponent remains free to object to admissibility of the proffered item on other grounds—including hearsay, relevance, or in criminal cases the right to confrontation. For example, in a criminal case in which data copied from a hard drive is proffered, the defendant can still challenge hearsay found in the hard drive, and can still challenge whether the information on the hard drive was placed there by the defendant.

A challenge to the authenticity of electronic evidence may require technical information about the system or process at issue, including possibly retaining a forensic technical expert; such factors will affect whether the opponent has a fair opportunity to challenge the evidence given the notice provided.

The reference to Rule 902(12) is intended to cover certifications that are made in a foreign country.

Also see: Paul Grimm, Gregory Joseph, Daniel Capra, Manual on Best Practices for Authenticating Digital Evidence; Authenticating Digital Evidence, 69 BAYLOR L. REV. 1 (2017).

Grimm/Brady Evidence Admissibility Chart

The rule change is a helpful addition to the litigator’s toolkit, but many challenges remain for attorneys handling electronic evidence. I agree with Kevin Brady, who is a top expert in the field of ESI evidence, who says that “the challenge for lawyers trying to authenticate digital evidence using the traditional rules of evidence can be confusing.” This may be an understatement! Kevin thinks that part of the challenge for attorneys arises from the rapidly-evolving landscape of data sources. He gives examples such as bitcoin, blockchain, smart contracts, social media, IoT, mobile devices, and cloud computing services. Moreover, the use of social media like Facebook, LinkedIn, Instagram and others continues to increase at an unbelievable rate and adds to the problem. Moreover, according to Business Insider, there are more people using the top four social messaging apps (WhatsApp, Messenger, WeChat, and Viber) than the top four social media apps (Facebook, Instagram, Twitter, and LinkedIn). According to Tech Crunch, Facebook’s Messenger alone has more than 1.3 billion monthly active users, and Instagram is officially testing a standalone messaging app, Direct.

Recognizing the problem Kevin Brady teamed up with U.S. District Court Judge Paul Grimm, the leading judicial expert in the field, to create the Grimm/Brady Evidence Admissibility Chart shown below.

The detailed reference chart provides discovery lawyers and trial attorneys with a quick reference guide for handling many different sources of ESI evidence. It covers Rule 104 to Rule 803(6) to Rule 901 and 902. The chart provides a step by step approach for authenticating digital information and successfully getting that information admitted into evidence.

The e-Discovery Team highly recommends that you carefully study this chart. Click on the photos and they will open in a larger size. Also suggest you download your own copy here: Grimm Brady Evidence Admission Chart 2018. Many thanks to Kevin Brady for helping me with this blog.



 


%d bloggers like this: