The Good, the Bad, and the Ugly: “Mt. Hawley Ins. Co. v. Felman Production, Inc.”

The Good The Bad and The Ugly - movie posterThis essay is about a new case on keyword search and sampling that I recommend you read. It is from West Virginia of all places, which shows that subtle e-discovery arguments and important rulings can now pop up in any jurisdiction, not just N.Y. and D.C.. Mt. Hawley Ins. Co. v. Felman Production, Inc., 2010 WL 1990555 (S.D. W. Va. May 18, 2010). The Felman opinion has many interesting qualities. (I use Felman as the case name, not Mt. Hawley Insurance, because Felman Production, Inc. is the plaintiff, and the insurance company is an intervenor for another insurance company defendant, who has thereby taken over the lead name, a case style practice I don’t like.) Felman addresses search reasonability in the scary context of privilege and applies Victor Stanley.

Unfortunately, my analysis of Felman brings to mind the title of the famous Western movie: The Good, the Bad, and the Ugly. I would be surprised if the plaintiff’s law firm, Venable, did not agree. The judge concluded that their client, Felman Productions, Inc., waived it’s attorney-client privilege by their negligent search and review. I am not so sure. It looks more like a software malfunction to me.

The Facts

United States Magistrate Judge Mary E. Stanley did a good job overall in this case, especially considering what I suspect are the low number of e-discovery issues heard in her district. The parts of the opinion I don’t like can be explained by the extreme facts (and poor math), so that is where we need to begin.

This is a case for insurance proceeds where the plaintiff responded to defendants’ requests for production by producing over 346 gigabytes of data. Id. at *12. Defendants were not happy about receiving so much information. Instead they complained and called it a “classic document dump.” Id. at *1. But then they searched though the data and found a “smoking gun.” The plaintiff had produced an email to their attorney dated May 14, 2008, actually two versions of the email. After finding the emails, which the plaintiff had listed on its privilege log, the defense did not tell plaintiff’s counsel about it and ask if they wanted it back. This, by the way, is what plaintiff’s counsel later argued, unsuccessfully, they should have done, and were required to do by ABA Model Rule 4.4(b) which states:

A lawyer who receives a document relating to the representation of the lawyer’s client and knows or reasonably should know that the document was inadvertently sent shall promptly notify the sender.

Defense counsel did not do that. Instead, they disclosed the email to the judge and made it a public record. They apparently did this by filing a copy of the email with the court as an exhibit to their motion for leave to amend their pleadings to add a claim for fraud. They argued that there was no privilege to the attorney email because of the crime-fraud exception, or alternatively by disclosure to a third-party. Also, they claimed waiver of any privilege by negligent review before production. The court agreed with the last point and that is what puts this decision on the e-discovery-world map.

This is very close to the scenario that Judge Grimm considered in Victor Stanley, and so of course that case was heavily argued by both sides. Unfortunately for plaintiff’s counsel, the Venable law firm, they were also on the losing side in the Victor Stanley case.  Judge Mary Stanley (no relation I assume to Victor Stanley) took pains to point that out in Felman. Id. at 12. See: Victor Stanley, Inc. v. Creative Pipe, Inc., 250 F.R.D. 251 250 F.R.D. 251, 70 Fed.R.Serv.3d 1052 (D. Md. 2008). These attorneys have had some very bad facts against them in both cases, whereas their opposing counsel have had smoking guns. The well-respected Venable firm has had to dig deep to try to defend their clients, but so far, it has been a grave, not a stash for bags of gold.

I am reminded of the famous line from the movie by the Good cowboy, Blondie, played by Clint Eastwood. After the famous three-way stand-off at the end of the movie, where Angel Eyes (the Bad) is killed and Tuco (the Ugly) finds out that there’s no bullets in his gun. Blondie tells Tuco to dig a hole (where buried gold is found) and says:

You see in this world there’s two kinds of people, my friend. Those with loaded guns, and those who dig. You dig.

The facts of this case also remind me of another e-discovery rich insurance claim case, Bray & Gillispie v. Lexington Insurance. You know the one, where bungling plaintiff’s counsel got themselves, their law firm, and their client sanctioned not once, but three times. Bray & Gillespie Mgmt. LLC v. Lexington Ins. Co., 2009 WL 546429 (M.D. Fla. Mar. 4, 2009) (Bray I); 2009 WL 2407754 (M.D.Fla. August 3, 2009) (Bray II); 2010 WL 55595 (M.D. Fla. Jan. 5, 2010) (Bray III). The attorneys in Bray dug a deep hole for themselves, but unlike Felman, the attorneys were sanctioned for not producing ESI, and for stripping it of key metadata when they did. Still, the underlying insurance claim facts of these two cases have interesting parallels.

Bray & Gillespie concerned an insurance claim for damages and business interruption from three hurricanes that hit Florida one summer and closed down plaintiff’s hotel. The plaintiff claimed all three hurricanes caused a loss of business and sought damages for each. The insurance company doubted that the last two hurricanes caused damages, claiming that the damages all came from the first event, Hurricane Charlie. The insurance company suspected that the claims for the second and third hurricanes were bogus and sought the hotel’s records to prove it. They did not get the ESI with smoking gun they wanted, but they did get case-dispositive sanctions. Dig?

Felman also concerns an insurance claim for property damage and business interruption. But here it was caused by a transformer failure in one of the three massive silicomanganese furnaces they owned. Only one furnace was destroyed, but they claimed damages for a complete loss of all of their business. Plaintiff claimed that it needed all three furnaces to run their business, and therefore its business operations were completely shut down by one furnace failure. Here again, the pesky insurance company doubted that claim, and demanded proof before they would pay.

The Smoking Gun

Here is where the smoking gun email comes in, the one that prompted defense counsel to cry foul. The plaintiff sought advice from its local attorney (not Venable) on how they might make the one furnace failure support a claim for loss of all business.  Here is the explanation of the contents of the letter by Judge Stanley:

In the email, the writer confirms his understanding that Felman “need[s] to show some kind of sales contract on the amount of production equal to production of all three furnaces.” (# 283-4, at 2.) He then comments that Felman does not have sales contracts for production from three furnaces, just two. Id. He reports that he discussed with customers the possibility of signing back-dated sales contracts, but the customers’ lawyers “didn’t like this option.” Id. Denys indicates that the customers were

ready to ma[k]e official letter to Felman saying that because of our problem with transformer they will not receive from us necessary amount of metal to cover their contracts and that they [are] losing money and market because we cannot supply them with metal (something like this). What do you think– what … will be b[e]tter for insurance company–to have this letter or contract?”

Id. Attorney Burd responded that the “insurance company may request any document relating to damages issue. Therefore all relevant documents need to be reviewed.” Id. After inquiring about the cause of the loss, Mr. Burd suggested “that we have a conference call with you[,] Katerina and others to make sure we are all on the same page.” Id

Id. at *14.

First, in defense of lawyers, may I point out that the customers’ lawyers did not like this idea! Of course, who listens to their lawyer? Most lawyers are lucky to get paid these days, much less have their advice followed. Now for the plaintiff’s no-fraud counter-argument after this inadvertent email disclosure outed their scheme:

Felman states that “the May 14 e-mail exchange evidences, at best, an inquiry regarding a contemplated fraud–not an attempted fraud. … It then argues that its evidence shows that Felman and its customers did not backdate sales contracts. Id. Felman contends that an inquiry concerning the legal effect of possible conduct is insufficient to meet the standard of being a crime or fraud, even if the client had bad intentions. … it argues that it is equally plausible that the question was innocent and not nefarious, id. at 16. That is, Felman suggests that the May 14 email was “a good faith inquiry regarding the appropriateness of contemplated conduct.”

The plaintiff is in effect arguing that the lawyers talked them out of it, that they may have thought bad thoughts, but then their lawyer set them straight and they did not do anything wrong. That is what lawyers are supposed to do, right? But wouldn’t this put the rest of the lawyer-client communications at issue? Wouldn’t the judge then want to see if this is in fact what happened, rather than just take their word for it? Of course, the plaintiff does not want that. They want to undo the privilege waiver, not expand upon it. I can understand the judge’s curiosity. In fact, in my view, this entire case is one of bad facts making bad law, which we will get to in The Bad and The Ugly part of this essay. But first, we have to go deeper into the facts.

Metrics of the Gun

After plaintiff found out that the insurance company had this email, they demanded it back under Federal Rule of Procedure 26(b)(5)(B), and Evidence Rule 502(b). The defense refused claiming the fraud exception to privilege. That part alone in interesting, but what makes the case really blog-worthy is the secondary argument that even if the email was privileged, the privilege had been waived. It was waived by the negligent review of plaintiff and its attorneys to exclude privileged files before production. It was waived by the negligent production.

By the way, although the May 14, 2008 email was the hottest smoking gun, defense counsel found more attorney client communications had been accidently produced. Plaintiff claimed privilege and sought return of 377 such communications, which I presume were all emails and attachments. Here is Judge Stanley’s explanation:

*3 The May 14 email is the proverbial tip of the iceberg. Defendants, arguing that Felman failed to take reasonable steps to protect privileged communications, identified nearly 980 attorney-client communications which were produced, of which Felman has now recalled 377. (# 325, at 4 and Ex. D.) Some of the communications are listed on the privilege log, but Felman has not recalled them. Id. at 5.

Under Venable’s direction, the plaintiff says it collected 1,638 gigabytes of data and produced 346 gigabytes. Defendants argued that 30% of the ESI produced were irrelevant, which it said was 14.3 gigabytes of “junk documents” equivalent to 1 Million pages of documents. Id. at *2. These gigabyte numbers obviously don’t add up: 30% of 346 gigabytes is 103.8 gigabytes, not 14.3. In fact, 14.3 gigs equals just a little over 4% of the total produced, not 30%. If it was in fact 30% irrelevant, or 103.8 gigs, that would mean 7,800,000 irrelevant pages, not 1,000,000. Perhaps there is an explanation for this, but not in the opinion. Sound metrics remains a weak point for most trial lawyers (and judges). More on that later in The Bad.

I must also note that the total numbers of files collected and produced are not stated, instead the gigabyte and bogus paper printout out numbers are used. I am opposed to this type of paper-based protodigital thinking and fuzzy metrics. I think you should speak in terms of files of ESI and forget about so called page equivalents. Also, I like the numbers to all add up and make sense.

The Good

In spite of these shaky facts, much of what Judge Stanley says on the subjects of privilege and search are good. There are also several interesting discussions on side-isses, but I’m skipping those to cut right to the core search issue. Judge Stanley explains at *8:

The next issue is whether Felman waived the privilege when it produced the May 14 email. Felman argues that its efforts to claw-back the May 14 email satisfy the reasonableness requirement of Rule 502(b)(3) (as set forth in Section H of the ESI Stipulation), that its production of the May 14 email was inadvertent and that Felman took reasonable steps to prevent its inadvertent disclosure. (# 323 at 11-12.) Felman points to its “carefully selected privilege search terms,” document-by-document review of potentially privileged documents, and a second electronic search of remaining documents, as evidence of its reasonable steps. Id. at 12. It notes that the May 14 email was listed on its privilege log. Id. After learning of the production of apparently privileged materials, Felman investigated and determined that certain documents were not tagged for attorney review due to an undetermined software error. Id. at 12-13.

The insurance company successfully argued to the contrary that these efforts were insufficient:

*9 Defendants assert that Felman did not take reasonable precautions to avoid disclosure of the May 14 email and other allegedly privileged communications and thereby waived the privilege. (# 325, at 3-7.) They compiled a list of the defects in Felman’s e-discovery and contend that Felman’s production fails the five-factor test set forth in Victor Stanley, Inc. v. Creative Pipe, Inc., 250 F.R.D. 251 (D.Md.2008). Id.

Here is Judge Stanley’s analysis:

Victor Stanley was decided shortly before the enactment of Rule 502(b), which states as follows:

The following provisions apply, in the circumstances set out, to disclosure of a communication or information covered by the attorney-client privilege or work-product protection.

* * *

(b) Inadvertent disclosure. When made in a Federal proceeding …, the disclosure does not operate as a waiver in a Federal … proceeding if

(1) the disclosure is inadvertent;

(2) the holder of the privilege or protection took reasonable steps to prevent disclosure; and

(3) the holder promptly took reasonable steps to rectify the error, including (if applicable) following Federal Rule of Civil Procedure 26(b)(5)(B).

The court will apply Rule 502(b), considering Victor Stanley and similar cases as to reasonableness. There is no dispute that Felman’s disclosure of the May 14 email was inadvertent, and the court so finds. Therefore Rule 502(b)(1) is satisfied.

The court has found that Felman’s notice of recall complied with Section H of the ESI Stipulation; thus pursuant to the last paragraph of Section H, and “notwithstanding any argument to the contrary,” the court finds that Rule 502(b)(3) is satisfied.

The remaining issue is whether Felman took reasonable steps to prevent disclosure in the first place.

Id. at *9. So far, so good. Judge Stanley has found the production was inadvertent and  the producing party took reasonable steps to rectify the error in accord with the rules. The judge then goes on to consider the key issue of 502(b)(2): whether Felman “took reasonable steps to prevent disclosure.”

The court begins by making findings of facts as to the efforts made by Felman and it attorneys, Venable. They seems pretty impressive to me, but obviously, this was not good enough to meet a reasonability standard in Judge Stanley’s court. In reading this laundry list of diligence, see if you can guess what the judge thought was so substandard as to warrant a finding of negligence and loss of privilege.

[T]he court finds that Felman and its counsel took the following steps to produce relevant records yet prevent disclosure of privileged materials:

• Negotiated the ESI Stipulation with Defendants

• Hired an ESI collection vendor, Innovative Discovery

• Discussed with Felman’s Information Technology Department the computer network structure at Felman and identified potential sources of relevant ESI

• Visited Felman’s West Virginia plant to coordinate and oversee ESI collection

• Decided to collect data using forensic imaging

• Directed the vendor to collect ESI from the current server and the backup server

• Collected 1,638 gigabytes of data

• Downloaded emails from 29 custodians for processing by Venable

• Hired a new vendor to process Felman’s Oracle and Soloman databases

• Identified the first six workstations to be processed and learned that each contained more data than anticipated

• Examined methods to cull non-relevant materials

• Selected search terms to retrieve documents responsive to Defendants’ document requests

• Tested the search terms against the Felman emails and added additional search terms

• Tested the search terms, including the additional terms, against the Felman emails, tagged responsive documents, and set them aside for privilege review

• Produced 17,064 Excel spreadsheets in November, 2009

• Selected “Privilege Search Terms” to identify materials which are potentially privileged and relevant

• Set aside potentially privileged materials for individualized document-by-document review for relevancy and privilege

• Tested Privilege Search Terms against Felman’s emails

• Retrieved native files of all images and examined thumbnails

• Conducted “eyes-on” review of all documents identified both as relevant and potentially privileged

*12 • Decided to use a vendor to complete the processing of Felman’s emails

• Produced ESI in native or .TIF format, with 36 fields of metadata

• Produced more than 346 gigabytes of data without sampling for relevancy, over-inclusiveness or under-inclusiveness

• Marked all 346 gigabytes of data as “CONFIDENTIAL.” (# 314, Ex. I.)  (Losey’s note: Judge Stanley really did not like this move, as she noted elsewhere in the opinion, but it is not why she found negligence. In reality, it is a common practice.)

• After Defendants complained that Felman had produced a large quantity of irrelevant material, Felman began a document-by-document review of all materials produced. Id.

[T]he court further finds as follows:

• The vendor Innovative Discovery processed Felman’s ESI, applied the relevance search terms and then posted the responsive data as 13 Concordance database files to a secure website for Venable to download and apply Privilege Search Terms

• The fourth Concordance database file was processed in a manner similar, if not identical, to other database files, and Privilege Search Terms were applied to it

• The May 14 email originated from the fourth Concordance database file

• After learning of the inadvertent disclosure, investigation revealed that the fourth Concordance database file inexplicably built an incomplete index of potentially privileged materials

• The manufacturer of the Concordance software, Lexis-Nexis, has not been able to explain why the index was incomplete

• When Defendants contended that 966 documents produced by Felman were potentially privileged, Felman and Venable determined that 377 of those documents were privileged and that 328 came from the fourth Concordance database file.

So what does this mean? It means the Concordance software failed for reasons unknown, that one out of thirteen Concordance database files was corrupted, and so Venable and Felman have an out, right? Isn’t it reasonable to rely on software? Especially from a big-time vendor like Concordance and Lexis? Wrong! Here is why (at least the stated reasons).

The Bad

Judge Stanley went on to make more findings of fact, which is where the opinion, in my opinion, begins to get bad, really bad.

[T]he court finds as follows:

• Felman over-produced documents to Defendants by more than 30% (Losey comment: really, or was it 4% as I noted before, which is pretty good?)

• Venable undertook to process and review Felman’s emails until December 19, 2009 (Losey comment: that was just five months after the request was served, again, considering a production of 346 gigs, is that really so bad to take five months to do it?)

• Felman produced 377 documents which it now claims are privileged (Losey comment: using industry averages of 16,000 files per gig, a production of 346 gigabytes probably consisted of 5,536,000 files. So 377 files is an error rate of .0006% Is that really so bad?)

• The production of 377 documents which are now claimed to be privileged is not solely attributable to the problem with the fourth Concordance database file (Losey comment: “solely” – what does that mean? Is there a causal relationship or not? The software did not index this batch, so it was not searchable. Period. That covered 328 out of 377 of the files to which a privilege is claimed.)

• Some documents on Felman’s privilege log were produced and not clawed-back (Losey comment: So? Amending logs and other discovery responses is encouraged. This is very common.)

• Some documents claimed by Felman to be privileged are not on the original privilege log (Losey Comment: Again, so? 5,536,000 files produced and you don’t think errors will be made? Mistakes are always made in big projects. The question is whether reasonable efforts were made.)

• Felman and Venable failed to perform critical quality control sampling to determine whether their production was appropriate and neither over-inclusive nor under-inclusive, even though Venable was counsel in the Victor Stanley case. (Losey comments: Uh oh! Judge Stanley appears at first glance to have them on this one. I’ve been advocating for sampling and quality control for years, trying to tell lawyers and vendors that it might come back to bite them if they don’t. Well, this case proves that my fears had at least some basis in fact. And in West Virginia of all places. Think you are safe in your jurisdiction? Think maybe you better start sampling too? Vendors – where is the random button I’ve been asking for? Still, let’s be honest. Would random sample searches have detected the accidental production of a few statistically insignificant privileged emails? Again, the math says no. Sampling would not have made any difference here. It would only make a difference if there was a 30% error rate on relevancy, and even then as to the relevancy problem, not privilege. But as we have already shown, that looks like a math error. The real error rate is 4% (14.3/346). Would sampling have caught a 4% error rate? Probably not, and even of it did, what’s wrong with only 4% error. Isn’t that a pretty good success rate for relevancy determinations. Do you really think judges will agree on relevance more than 96% of the time? I don’t. Anyway, this is all irrelevant to the issue at hand, the reasonability of the privilege review.)

• Felman and Venable apparently failed to perform simple keyword searches to locate copies of the May 14 email which were produced and to identify documents which comprise attorney communications (Losey comment: Apparently? Not so sure about that. It does say searches were run and privilege search terms tests were run. Do you run a search to check your prior searches? When is enough, enough?)

*13 • Felman’s post-production claw-backs appear to be based on notifications by Defendants and not on its own review. (Losey comment: Again, so?  The Commentary to Rule 502 expressly states that post production review is not required, unless there are obvious indications that a mistake was made. There were no findings of such obvious indications. There is no suggestion that anyone should have known that the one Concordance database out of thirteen was defective.)

The Ugly

Judge Stanley then takes these findings and applies them to the law. Here it turns ugly, as rationalizations to keep the smoking gun public take over. You be the judge. Here are her words:

The Court will apply the five-factor test discussed above in the context of these findings and the commentary to the Rules. First, the precautions taken to prevent inadvertent disclosure were not reasonable. As warned in Victor Stanley, 250 F.R.D. at 257, the failure to test the reliability of keyword searches by appropriate sampling is imprudent. Second, the number of inadvertent disclosures is large, more than double the number discussed in Victor Stanley, a number which underscores the lack of care taken in the review process.

Id. at *13.

This is bad and ugly. I agree that the failure to test the reliability of keyword searches by appropriate sampling is imprudent. But that is not what caused the disclosure of the privileged emails. It was caused by failures in the indexing of the Concordance software! Judge Stanley’s own findings say that. The failure to sample may have been negligent here, but it did not cause the damages. You cannot search ESI that has not been indexed.

Judge Stanley’s reliance on numbers of inadvertent disclosures and comparisons with Victor Stanley is also misplaced. The inadvertent production of 377 files out of 5,536,000 is actually a far better error rate than in Victor Stanley. The actual number of mistakes is irrelevant. What counts is the percentage. One mistake is a ridiculously high error rate suggesting negligence where there are only three documents to review. Conversely, a million mistakes out of a hundred billion is pretty good. In Victor Stanley 165 privileged documents were mistakenly produced. Yes, that’s more than the 377 mistakenly produced in Felman. But there were only 9,008 PDF files produced in Victor Stanley, compared to an estimated 5,536,000 produced in Felman. Judges and litigants have got to start understanding metrics and the technological realities of high volume ESI search.

There seems to be a basic disconnect here in what is going on.  There is a disconnect in understanding math and metrics and a disconnect in understanding sampling’s irrelevance when the software indexing is defective. These misunderstandings were caused, at least in part, by a rush to justify the disclosure of the May 14 email. Judge Stanley’s next words show that to be true:

The May 14 email resonates throughout this case–a bell which cannot be unrung. Its content has had great influence on Defendants’ discovery requests and deposition questions. Confidentiality cannot be restored to that document.

To state the obvious, it also had great influence on the court’s ruling. Judge Stanley goes on:

Third, the extent of the disclosures is not known to the Court because the 377 documents have not been submitted in camera. Fourth, there has been delay in measures taken to rectify the disclosure of the documents. It is an important fact that identification of privileged documents which were disclosed to Defendants was made by the Defendants, not Felman or its counsel. Finally, as in Victor Stanley, id. at 263, Felman has “pointed to no overriding interests in justice that would excuse them from the consequences of producing privileged/protected materials.”

The court finds that Felman and Venable did not take reasonable steps to prevent disclosure of the May 14 email, that Felman and Venable have not satisfied all three subsections of Rule 502(b), and that they have waived protection for the May 14 email.


At the end of the opinion Judge Stanley further discusses the crime fraud exception and decides to defer ruling on that issue until she has a chance to review all 377 emails. She should have delayed ruling on the whole case until after the in camera review of these emails. The suspicion of crime and fraud permeates and distorts the analysis. If this opinion is not reversed, it is likley to be cited again and again to attack search efforts as unreasonable. If this happens, it is my hope that the courts considering this opinion will see that it is a product of bad facts and distinguish it from the matter they are considering. They should also insist that all of the numbers argued to them add up. If they do that, and if they understand the true nature of proportionality, then this opinion will not do too much damage. It has its good parts, and if nothing else, should encourage more sampling.

It is possible that after Judge Stanley reviews all 377 emails, she might well agree that the crime fraud exception does not apply. She could be persuaded that the client, Felman Productions, was just asking if they could do it. The still unseen emails may clearly said no. They may clearly indicate that the client followed the advice and no fraud was committed. It’s possible. Anyway, it could happen in another case. It could happen to you. How do you know if your software is perfect (it isn’t)? How do you know if it has been corrupted? How do you know if all of the words were correctly indexed so that a search will now catch-all of your privileged emails? Do you do everything twice? How much will that cost? What about Rule 1? Yes, this case raises many troubling questions.

So what is a practitioner to do? First of all, if you are in a situation like this and the rest of the emails really are exculpatory, go ahead and show them to the judge, and only the judge in camera. Get all of the other rulings delayed until that happens. Sometimes the bark is worse than the bite and the Magistrate’s imagination of what’s in the emails may be far worse than the reality.

Second, get an agreement with opposing counsel at the start of the case not to raise the reasonability argument, or, alternatively, to stipulate that the procedure agreed to is reasonable. It could be a quick-peek procedure and both sides agree that under the circumstances the provisions of 502(b)(1) and (2) are met. The disclosure, if it later happens, is automatically deemed inadvertent and the steps taken to prevent disclosure, like a quick-peek agreement, or whatever, are automatically deemed reasonable. Then get this agreement included in a 502(e) Court Order. That last step is critical. Then you should be protected.

Of course, all of this requires talking about e-discovery with opposing counsel at the beginning of the case. It also requires transparent disclosure of search processes. Many lawyers are still reluctant to go there. Perhaps the lessons of Felman will convince them of the necessity to cooperate and disclose to properly serve and  protect. In any event, it shows the wisdom of a 502(e) agreement and court order. This agreement and order is a friend to all lawyers dealing with search. As Blondie said: “I’ll sleep better knowing my good friend is by my side to protect me.”

I’ll leave you with this clip from the great movie:

22 Responses to The Good, the Bad, and the Ugly: “Mt. Hawley Ins. Co. v. Felman Production, Inc.”

  1. You quoted above that “The production of 377 documents … is not solely attributable to the problem with the fourth Concordance database file” and questioned what “solely” meant in this context.

    When I read that section, I immediately jumped to the prior comment that “Venable determined that 377 of these documents were privileged and that 328 came from the fourth Concordance database file.” To me, that comment implied that 49 of the documents came from one of the other 12 Concordance files. So either the indexing failed more than once or something else happened to cause the leak.

    Your comments about ratios and reasonable standards given the extraordinary volumes are well-taken. I share your frustration with the confusion between absolute and relative magnitudes. But I was not as concerned by the comment about sole cause.

  2. Andrew says:

    Maybe I am missing something- but if the Plaintiff ran search terms to determine which documents were responsive and there was an indexing failure then the documents that were not indexed should never have been produced in the first place (they would not have hit on a search term). How did they produce documents that were not searchable?

  3. Brian Conrad says:

    It sounds as if the relevancy searches were run and the responsive docs were imported to Concordance DBs. The Concordance DBs were then indexed and Priv searches were run in Concordance to identify Priv docs. If the indexing was incomplete…..

  4. Joe Howie says:

    It appears that one of the real “bads” in this case was the plaintiff’s failure to consolidate duplicates prior to review. How else was there more than one copy of the email to be treated inconsistently? In fact the opinion states, “Apparently Felman produced two versions of the May 14 email, with different Bates numbers.” With emails from 29 custodians, one would expect duplicate emails across custodians. A survey by the eDiscovery Institute showed that parties that did not consolidate duplicates across custodians reviewed 27% more records on average than those who consolidated duplicates across custodians with sometimes far higher percentages in individual cases.

    The court noted that of the 377 records now claimed to be privileged, some were on the privilege log lending further support to the notion that duplicate consolidation across custodians was probably not done.

    So what we appear to have here is a case where the plaintiff’s attorneys billed for reviewing duplicate records and the client not only got overbilled for the review but ended up having privilege waived when, guess what, duplicate copies were treated inconsistently. Plaintiff would have been in a better position without having been faced with having made inconsistent privilege decisions on different copies of the same record – a risk that was discussed in “Ethics and Ediscovery Review,” by Patrick Oot, Anne Kershaw and me, published in the Jan/Feb 2010 issue of the ACC Docket.

    The court should have pointed to the failure to consolidate duplicates as a factor weighing against a finding of plaintiff having made reasonable efforts to identify privileged records. Carefully designed hash value calculation procedures would most likely have enabled plaintiff to find all records having hash values equal to records for which privilege had been claimed, independent of whether text indexing failed or didn’t fail. At the very least it would have been another prudent measure they could have pointed to in support of their argument of having taken reasonable screening measures.

    As a final comment, note that defendants were complaining about the volume of data produced to them, but there’s no record of whether they asked for across-custodian duplicate consolidation which could have reduced the volume of data, perhaps significantly. The old argument that you need a copy from each custodian who had a copy in order to know who they were doesn’t hold water – the producing party can identify all custodians who had copies or each record and could even provide the path information for each copy.

  5. Craig Ball says:

    Hi Ralph:

    You’ve outdone yourself with this ithoughtful and spirited analysis. May I take issue with your statement–or at least inquire as to your intent when you recite–“You cannot search ESI that has not been indexed.”

    Certainly you can,if you have the source ESI (not soley the indices) available to you. For good or ill, indexed search is the standard for the e-discovery industry, but linear search tends to be the standard in computer forensics. With linear search, you actually search the data. With indexed search, you’re arguably only searching metadata; that is, if you define a rules-based detection of a word as a form of metadata.

    I’ve long believed that privilege is a term well-suited to describing a narrow dispensation from the duty to produce relevant evidence. It’s a “privilege” to be granted this exception, and so it’s not unreasonable to impose a substantial burden on those who claim such privilege. Perhaps, there should be an obligation for counsel to segregate privileged communications found in counsel’s file (including counsel’s e-mail about the case) and run targeted searches on materials slated for production seeking known, core privileged communications.

    One need not be a Mensa member to figure out that the e-mail with counsel geared to whether or not to fabricate backdated evidence (if, indeed,that’s the tenor of the May 14 e-mail exchange) should perhaps be deemed so critical–so inflammatory–that it and like known privileged material warrants particularized linear search through the material slated for production as a prudent QA/QC mechanism.

    My thinking is that, for all the many challenges there are to isolating privileged material in voluminous ESI, finding the privileged items well known to counsel and appearing in their own files need not be one of them.

  6. Jeane Thomas says:

    Hi Ralph,
    Really good analysis of this opinion. Also encourages me to watch the movie again.

    Your points about proportionality in terms of number of inadvertently privileged documents produced relative to the total produced, and the defect of the software in spite of arguably “reasonable” efforts are right on point (IMHO).

    What I really appreciate is your notion of “quick peek” in a context I hadn’t considered. We tend to think of “quick peek” as isolating a set of ESI and turning it over without any review or analysis, and my experience along with recent (informal) surveys show that clients/counsel are very reluctant to take that approach — largely because of privilege issues. The concern is that the bell cannot be unrung. However, I think you are suggesting that the parties agree upon a process, including automated searches or protocols, to isolate a subset of ESI unlikely (but not guaranteed) to be free of privileged material and then turn that over in a “quick peek” type of situation. To the extent the process did not filter out all privileged information, which is certainly the case, the producing party is protected. It also helps with the “data dump” argument.

    Anyway, thanks for posting. And also congrats on your new position. Looking forward to catching up soon.


  7. Sonya Sigler says:

    I like that this opinion is drawing attention to how search and retrieval is being done, what tools are used, and if any quality control or sampling was done, but I think the more important point is that clients, their attorneys, and their vendors need to know what is being done to the review data. How is it being indexed and what, exactly, is being indexed – because you won’t be able to find what isn’t indexed when you are using keyword search.

  8. Ed Fiducia says:

    This is exactly why I have been preaching the “concept search/near duplicate clustering” gospel for years. Even if database 4 had a failed Concordance index, any of the tools readily available in the marketplace (Synthetix, Equivio, Stratify, etc.) would have at least pointed reviewers to the existence of other similar emails… even those that had failed to “Hash”.

    I have long been a proponent of a “Best Practice” that dictates the final step review and QC of the priv log using some type of Near Duplicate technology before documents are produced. Best to spend the time and money to do everything reasonably possible to ensure that privilege is properly asserted on all iterations of a document than to rely on simple keyword searches and hope for the best.

    Ralph makes some great comments with regard to the incredibly small percentage of missed documents when compared to the entirety of the corpus, but it only takes one to kill a case.

  9. Patrick Oot says:


    Interesting analysis. You’re always at the front of the pack.

    Craig’s point on linear search is well taken. I would be interested to know how much extra time and effort a litigant would need to conduct linear searching over the traditional (extracted text) indexed search.

    I imagine the time required to search a large set of documents with hundreds (if not thousands) of search terms and queries might seem daunting at best.

    A well thought out privilege screen would at the very least contain the domain names of outside law firms, attorney e-mail addresses, and attorney names, perhaps even hash values of documents logged as privileged in prior cases. Can linear search accommodate this request in a timely fashion?

    Alternatively, maybe a linear search of the production set alone might be advisable.

    I went and pulled the joint stipulation

    I find it troubling that the parties were so specific about many aspects of the stipulation (production format, metadata, etc.) but didn’t seek a protective order with language defining:

    How a party will be deemed to have taken reasonable steps to prevent disclosure; and

    How a party will be deemed to have taken reasonable steps to rectify the error.

    Specifically, the parties should have agreed to process, timing, and logging methodology.

    Two years ago, several of us from The Sedona Conference crafted a few protective order provisions that would have avoided the judge’s analysis. They are attached to my article at page 15.

    Had the parties followed a specific protocol identified in a protective order, this war might have been avoided.

    Patrick Oot

    • Ralph Losey says:

      Thanks for those good practice tips. This kind of agreement and order should become a common practice, especially after this case. Yet, confusion is still widespread on 502 orders. I still run into attorneys who think it only applies to quick peek agreements.

  10. Craig Ball says:


    I was advocating linear search only against the materials slated for production, not the entire collection for the very reasons you suppose: time.

    As to your good questions, yes, linear search can handily traverse the proposed production items for attorney names and addresses. A hash comparison for known privileged items could also be undertaken–and it’s very fast and valuable, assuming you’ve reconciled the peculiar challenges attendant to hashing e-mail messages so as to facilitate useful comparisons. That’s a whole other post in itself.

    I’ve talked further about linear search as a QA/QC backstop for privileged material at the EDDUpdate blog:

  11. JMW says:

    Dear Mr. Losey,
    Thank you for sharing your insights and analysis on this blog.
    I do have a comment regarding 502(b) and (d). Instead of having the parties agree on what is reasonable, why not try to avoid rule 502(b) altogether. The advisory committee notes state, “the court order may provide for return of documents without waiver irrespective of the care taken by the disclosing party.” I believe this would be in keeping with the recent analysis of 502(d) in Rajala v. McGuire Woods, LLP.
    Thank you for your consideration of this idea.

  12. […] a much more detailed discussion of this case, see Ralph Losey’s post, The Good, the Bad, and the Ugly: ‘Mt. Hawley Ins. Co. v. Felman Production, Inc.’ The case is also discussed at the K&L Gates blog, Electronic Discovery Law. Share | […]

  13. […] paraphrase Ralph Losey, who wrote an excellent post about this case, “So what?” It may properly have taken five months to collect and review over a million […]

  14. Antonio Lucchiani says:

    Honestly, I am a bit offended that you are making West Virginia sound like some type of third world country with no technology. As you probably do not know, WV is home to many companies and government installations such as the FBI document center, the Dept. of Veterans Affairs network and security operations center amond others. Not to mention that West Virginia University has a nationally acclaimed computer science and engineering department. Although i enjoyed reading your blog on the Felman case, it would have been much better had you not taken the point of view that WV is backwards where everyone is barefooted and do not even have running water, let alone an internet connection.

  15. […] recently commented on a long, thoughtful post of Ralph Losey’s discussing Mt. Hawley Ins. Co. v. Felman Production, Inc., 2010 WL 1990555 (S.D. W. Va. May 18, […]

  16. […] paraphrase Ralph Losey, who wrote an excellent post about this case, “So what?” It may properly have taken five months to collect and review over a million […]

  17. […] a much more detailed discussion of this case, see Ralph Losey’s post, The Good, the Bad, and the Ugly: ‘Mt. Hawley Ins. Co. v. Felman Production, Inc.’ The case is also discussed at the K&L Gates blog, Electronic Discovery […]

Leave a Reply