Responding Party’s Complaints of Financial Burden of Document Review Were Unsupported by the Evidence, Any Evidence

August 5, 2018

One of the largest cases in the U.S. today is a consolidated group of price-fixing cases in District Court in Chicago. In Re Broiler Chicken Antitrust Litigation, 290 F. Supp. 3d 772 (N.D. Ill. 2017) (order denying motions to dismiss and discussing the case). The consolidated antitrust cases involve allegations of a wide spread chicken price-fixing. Big Food Versus Big Chicken: Lawsuits Allege Processors Conspired To Fix Bird Prices (NPR 2/6/18).

The level of sales and potential damages are high. For instance, in 2014 the sales of broiler chickens in the U.S. was $32.7 Billion. That’s sales for one year. The classes have not been certified yet, but discovery is underway in the consolidated cases.

The Broiler Chicken case is not only big money, but big e-discovery. A Special Master (Maura Grossman) was appointed months ago and she developed a unique e-discovery validation protocol order for the case. See: TAR for Smart Chickens, by John Tredennick and Jeremy Pickens that analyzes the validation protocol.

Maura was not involved in the latest discovery dispute where, Agri Stats, one of many defendants, claimed a request for production was too burdensome as to it. The latest problem went straight to the presiding Magistrate Judge Jeffrey T. Gilbert who issued his order on July 26, 2018. In re Broiler Chicken Antitrust Litig., 2018 WL 3586183 (N.D. Ill. 7/26/18).

Agri Stats had moved for a protective order to limit an email production request. Agri Stats claimed that the burden imposed was not proportional because it would be too expensive. Its lawyers told Judge Gilbert that it would cost between $1,200,000 and $1,700,00 to review the email using the keywords negotiated.

Fantasy Hearing

I assume that there were hearings and attorney conferences before the hearings. But I do not know that for sure. I have not seen a transcript of the hearings with Judge Gilbert. All we know is that defense counsel told the judge that under the keywords selected the document review would cost between $1,200,000 and $1,700,000, and that they had no explanation on how the cost estimate was prepared, nor any specifics as to what it covered. Although I was not there, after four decades of doing this sort of work, I have a pretty good idea of what was or might have been said at the hearing.

This representation of million dollar costs by defense counsel would have gotten the attention of the judge. He would naturally have wanted to know how the cost range was calculated. I can almost hear the judge say from the bench: “$1.7 Million Dollars to do a doc review. Yeah, ok. That is a lot of money. Why so much counsel? Anyone?” To which the defense attorneys said in response, much like the students in Ferris Beuller’s class:

“. . . . . .”


Yes. That’s right. They had Nothing. Just Voodoo Economics

Well, Judge Gilbert’s short opinion makes it seem that way. In re Broiler Chicken Antitrust Litig., 2018 WL 3586183 (N.D. Ill. 7/26/18).

If a Q&A interchange like this happened, either in a phone hearing, or in person, then the lawyers must have said something. You do not just ignore a question by a federal judge. The defense attorneys probably did a little hemming and hawing, conferred among themselves, and then said something to the judge like: “We are not sure how those numbers were derived, $1.2M to $1.5M, and will have to get back to you on that question, Your Honor.” And then, they never did. I have seen this kind of thing a few times before. We all try to avoid it. But it is even worse to make up a false story, or even present an unverified story to the judge. Better to say nothing and get back to the judge with accurate information.

Discovery Order of July 26, 2018

Here is a quote from Judge Gilbert’s Order so you can read for yourself the many questions the moving party left unanswered (detailed citations to record removed; graphics added):

Agri Stats represents that the estimated cost to run the custodial searches EUCPs propose and to review and produce the ESI is approximately $1.2 to $1.7 million. This estimated cost, however, is not itemized nor broken down for the Court to understand how it was calculated. For example, is it $1.2 to $1.7 million to review all the custodial documents from 2007 through 2016? Or does this estimate isolate only the pre-October 2012 custodial searches that Agri Stats does not want to have to redo, in its words? More importantly, Agri Stats also admits that this estimate is based on EUCPs’ original proposed list of search terms. But EUCPs represent (and Agri Stats does not disagree) that during their apparently ongoing discussions, EUCPs have proposed to relieve Agri Stats of the obligation to produce various categories of documents and data, and to revise the search terms to be applied to data that is subject to search. Agri Stats does not appear to have provided a revised cost estimate since EUCPs agreed to exclude certain categories of documents and information and revised their search terms. Rather, Agri Stats takes the position that custodial searches before October 3, 2012 are not proportional to the needs of the case — full stop — so it apparently has not fully analyzed the cost impact of EUCPs’ revised search terms or narrowed document and data categories.

The Court wonders what the cost estimate is now after EUCPs have proposed to narrow the scope of what they are asking Agri Stats to do. (emphasis added) EUCPs say they already have agreed, or are working towards agreement, that 2.5 million documents might be excluded from Agri Stats’s review. That leaves approximately 520,000 documents that remain to be reviewed. In addition, EUCPs say they have provided to Agri Stats revised search terms, but Agri Stats has not responded. Agri Stats says nothing about this in its reply memorandum.

EUCPs contend that Agri Stats’s claims of burden and cost are vastly overstated. The Court tends to agree with EUCPs on this record. It is not clear what it would cost in either time or money to review and produce the custodial ESI now being sought by EUCPs for the entire discovery period set forth in the ESI Protocol or even for the pre-October 3, 2102 period. It seems that Agri Stats itself also does not know for sure what it would have to do and how much it would cost because the parties have not finished that discussion. Because EUCPs say they are continuing to work with Agri Stats to reduce what it must do to comply with their discovery requests, the incremental burden on what Agri Stats now is being asked to do is not clear.

For all these reasons, Agri Stats falls woefully short of satisfying its obligation to show that the information [*10] EUCPs are seeking is not reasonably accessible because of undue burden or cost.

Estimations for Fun and Profit

In order to obtain a protective order you need to estimate the costs that will likely be involved in the discovery from which you seek protection. Simple. Moreover, it obviously has to be a reasonable estimate, a good faith estimate, supported by the facts. The Brolier Chicken defendant, Agri Stats, came up with an estimate. They got that part right. But then they stopped. You never do that. You do not just throw up a number and hope for the best. You have to explain how it was derived. Blushing at any price higher than that is not a reasonable explanation, but is often honest.

Be ready to explain how you came up with the cost estimate. To break down the total into its component parts and allow the “Court to understand how it was calculated.” Agri Stats did not do that. Instead, they just used a cost estimate of between $1.2 to $1.7 million. So of course Agri Stats’ motion for protective order was denied. The judge had no choice because no evidence to support the motion was presented, neither factual or expert evidence. There was no need for Judge Gilbert to go into the secondary questions of whether expert testimony was also needed and whether it should be under Rule 702. He got nothing remember. No explanation for the $1.7 Million.

The lesson of the latest discovery order in Broiler Chicken is pretty simple. In re Broiler Chicken Antitrust Litig., 2018 WL 3586183 (N.D. Ill. 7/26/18). Get a real cost estimate from an expert. The expert needs to know and understand document review, search and costs of review. They need to know how to make reasonable search and retrieval efforts. They also need to know how to make reliable estimates. You may need two experts for this, as not all have expertise in both fields, but they are readily available. Many can even talk pretty well too, but not all! Seriously, everybody knows we are the most fun and interesting lawyer subgroup.

The last thing to do is skimp on an expert and just pull out a number from your hat (or your vendor’s hat) and hope for the best.

This is federal court, not a political rally. You do not make bald assertions and leave the court wondering. Facts matter. Back of the envelope type guesses are not sufficient, especially in a big case like Broiler Chicken. Neither are guesstimates by people who do not know what they are doing. Make disclosure and cooperate with the requesting party to reach agreement. Do not just rush to the courthouse hoping to  dazzle with smoke and mirrors. Bring in the experts. They may not dazzle, but they can get you beyond the magic mirrors.

Case Law Background

Judge Paul S. Grewal, who is now Deputy G.C. of Facebook, said quoting The Sedona Conference in Vasudevan: There is no magic to the science of search and retrieval: only mathematics, linguistics, and hard work.Vasudevan Software, Inc. v. Microstrategy Inc., No. 11-cv-06637-RS-PSG, 2012 US Dist LEXIS 163654 (ND Cal Nov 15, 2012) (quoting The Sedona Conference, Best Practices Commentary on the Use of Search and Information and Retrieval Methods in E-Discovery, 8 Sedona Conf. J. 189, 208 (2007). There is also no magic to the art of estimation, no magic to calculating the likely range of cost to search and retrieve the documents requested. Judge Grewal refused to make any decision in Vasudevan without expert assistance, recognizing that this area is “fraught with traps for the unwary” and should not be decided on mere arguments of counsel.

Judge Grewal did not address the procedural issue of whether Rule 702 should govern. But he did cite to Judge Facciola’s case on the subject, United States v. O’Keefe, 537 F. Supp. 2d 14 (D.D.C. 2008). Here Judge Facciola first raised the discovery expert evidence issue. He not only opined that experts should be used, but that the parties should follow the formalities of Evidence Rule 702. That governs things such as whether you should qualify and swear in an expert and follow otherwise follow Rule 702 on their testimony. I discussed this somewhat in my earlier article this year, Judge Goes Where Angels Fear To Tread: Tells the Parties What Keyword Searches to Use.

Judge Facciola in O’Keffe held that document review issues require expert input and that this input should be provided with all of the protections provided by Evidence Rule 702.

Given this complexity, for lawyers and judges to dare opine that a certain search term or terms would be more likely to produce information than the terms that were used is truly to go where angels fear to tread. This topic is clearly beyond the ken of a layman and requires that any such conclusion be based on evidence that, for example, meets the criteria of Rule 702 of the Federal Rules of Evidence. Accordingly, if defendants are going to contend that the search terms used by the government were insufficient, they will have to specifically so contend in a motion to compel and their contention must be based on evidence that meets the requirements of Rule 702 of the Federal Rules of Evidence.


In the Boiler Chicken Antitrust Order of July 27, 2018, a motion for protective order was denied because of inadequate evidence of burden. All the responding party did was quote a price-range, a number presumably provided by an expert, but there was no explanation. More evidence was needed, both expert and fact. I agree that generally document review cost estimation requires opinions of experts. The experts need to be proficient in two fields. They need to know and understand the science of document search and retrieval and the likely costs for these services for a particular set of data.

Although all of the formalities and expense of compliance with Evidence Rule 702 may be needed in some cases, it is probably not necessary in most. Just bring your expert to the attorney conference or hearing. Yes, two experts may well disagree on some things, probably will, but the areas of agreement are usually far more important. That in turn makes compromise and negotiation far easier. Better leave the technical details to the experts to sort out. That follows the Rule 1 prime directive of “just, speedy and inexpensive.” Keep the trial lawyers out of it. They should instead focus and argue on what the documents mean.




Judge Goes Where Angels Fear To Tread: Tells the Parties What Keyword Searches to Use

June 24, 2018

John Facciola was one of the first e-discovery expert judges to consider the adequacy of a producing parties keyword search efforts in United States v. O’Keefe, 537 F. Supp. 2d 14 (D.D.C. 2008). He first observed that keyword search and other computer assisted legal search techniques required special expertise to do properly. Everyone agrees with that. He then reached an interesting, but still somewhat controversial conclusion: because he lacked such special legal search expertise, and knew full well that most of the lawyers appearing before him did too, that he could not properly analyze and compel the use of specific keywords without the help of expert testimony. To help make his point he paraphrased Alexander Pope‘s famous line from An Essay on Criticism: “For fools rush in where angels fear to tread.

Here are the well-known words of Judge Facciola in O’Keffe (emphasis added):

As noted above, defendants protest the search terms the government used.[6]  Whether search terms or “keywords” will yield the information sought is a complicated question involving the interplay, at least, of the sciences of computer technology, statistics and linguistics. See George L. Paul & Jason R. Baron, Information Inflation: Can the Legal System Adapt?; 13 Ricn. J.L. & TECH. 10 (2007). Indeed, a special project team of the Working Group on Electronic Discovery of the Sedona Conference is studying that subject and their work indicates how difficult this question is. See The Sedona Conference, Best Practices Commentary on the Use of Search and Information Retrieval, 8 THE SEDONA CONF. J. 189 (2008).

Given this complexity, for lawyers and judges to dare opine that a certain search term or terms would be more likely to produce information than the terms that were used is truly to go where angels fear to tread. This topic is clearly beyond the ken of a layman and requires that any such conclusion be based on evidence that, for example, meets the criteria of Rule 702 of the Federal Rules of Evidence. Accordingly, if defendants are going to contend that the search terms used by the government were insufficient, they will have to specifically so contend in a motion to compel and their contention must be based on evidence that meets the requirements of Rule 702 of the Federal Rules of Evidence.

Many courts have followed O’Keffe, even though it is a criminal case, and declined to step in and order specific searches without expert input. See eg. the well-known patent case, Vasudevan Software, Inc. v. Microstrategy Inc., No. 11-cv-06637-RS-PSG, 2012 US Dist LEXIS 163654 (ND Cal Nov 15, 2012). The opinion was by U.S. Magistrate Judge Paul S. Grewal, who later became the V.P. and Deputy General Counsel of Facebook. Judge Grewal wrote:

But as this case makes clear, making those determinations often is no easy task. “There is no magic to the science of search and retrieval: only mathematics, linguistics, and hard work.”[9]

Unfortunately, despite being a topic fraught with traps for the unwary, the parties invite the court to enter this morass of search terms and discovery requests with little more than their arguments.

More recently, e-discovery expert Judge James Francis addressed this issue in Greater New York Taxi Association v. City of New York, No. 13 Civ. 3089 (VSB) (JCF) (S.D.N.Y. Sept. 11, 2017) and held:

The defendants have not provided the necessary expert opinions for me to assess their motion to compel search terms. The application is therefore denied. This leaves the defendants with three options: “They can cooperate [with the plaintiffs] (along with their technical consultants) and attempt to agree on an appropriate set of search criteria. They can refile a motion to compel, supported by expert testimony. Or, they can request the appointment of a neutral consultant who will design a search strategy.”[10] Assured Guaranty Municipal Corp. v. UBS Real Estate Securities Inc., No. 12 Civ. 1579, 2012 WL 5927379, at *4 (S.D.N.Y. Nov. 21, 2012).

I am inclined to agree with Judge Francis. I know from daily experience that legal search, even keyword search, can be very tricky, depends on many factors, including the documents searched. I have spent over a decade working hard to develop expertise in this area. I know that the appropriate searches to be run depends on experience and scientific, technical knowledge on information retrieval and statistics. It also depends on tests of proposed keywords; it depends on sampling and document reviews; it depends on getting your hands dirty in the digital mud of the actual ESI. It cannot be done effectively in the blind, no matter what your level of expertise. It is an iterative process of trial and errors, false positives and negatives alike.

Enter a Judge Braver Than Angels

Recently appointed U.S. Magistrate Judge Laura Fashing in Albuquerque, New Mexico, heard a case involving a dispute over keywords. United States v. New Mexico State University, No. 1:16-cv-00911-JAP-LF, 2017 WL 4386358 (D.N.M. Sept. 29, 2017). It looks like the attorneys in the case neglected to inform Judge Fashing of United States v. O’Keefe. It is a landmark case in this field, yet was not cited in Judge Fashing’s order. More importantly, Judge Fashing did not take the advice of O’Keefe, nor the many cases that follow it. Unlike Judge Facciola and his angels, she told the parties what keywords to use, even without input from experts.

The New Mexico State University opinion did, however, cite to two other landmark cases in legal search, William A. Gross Const. Assocs., Inc. v. Am. Mfrs. Mut. Ins. Co., 256 F.R.D. 134, 135 (S.D.N.Y. 2009) by Judge Andrew Peck and Victor Stanley, Inc. v. Creative Pipe, Inc., 250 F.R.D. 251, 260, 262 (D. Md. May 29, 2008) by Judge Paul Grimm. Judge Fashing held in New Mexico State University:

This case presents the question of how parties should search and produce electronically stored information (“ESI”) in response to discovery requests. “[T]he best solution in the entire area of electronic discovery is cooperation among counsel.” William A. Gross Const. Assocs., Inc. v. Am. Mfrs. Mut. Ins. Co., 256 F.R.D. 134, 135 (S.D.N.Y. 2009). Cooperation prevents lawyers designing keyword searches “in the dark, by the seat of the pants,” without adequate discussion with each other to determine which words would yield the most responsive results. Id.

While keyword searches have long been recognized as appropriate and helpful for ESI search and retrieval, there are well-known limitations and risks associated with them, and proper selection and implementation obviously involves technical, if not scientific knowledge.

* * *

Selection of the appropriate search and information retrieval technique requires careful advance planning by persons qualified to design effective search methodology. The implementation of the methodology selected should be tested for quality assurance; and the party selecting the methodology must be prepared to explain the rationale for the method chosen to the court, demonstrate that it is appropriate for the task, and show that it was properly implemented.

Id. (quoting Victor Stanley, Inc. v. Creative Pipe, Inc., 250 F.R.D. 251, 260, 262 (D. Md. May 29, 2008)).

Although NMSU has performed several searches and produced thousands of documents, counsel for NMSU did not adequately confer with the United States before performing the searches, which resulted in searches that were inadequate to reveal all responsive documents. As the government points out, “NMSU alone is responsible for its illogical choices in constructing searches.” Doc. 117-1 at 8. Consequently, which searches will be conducted is left to the Court.

Judges Francis, Peck and Facciola

Judge Laura Fashing had me in the quote above until the final sentence. Up till then she had been wisely following the four great judges in this area, Facciola, Peck, Francis and Grimm. Then in the next several paragraphs she rushes in to specify what search terms should be used for what categories of ESI requested. Why should the Court go ahead and do that without expert advice? Why not wait? Especially since Judge Fashing starts her opinion by recognizing the difficulty of the task, that “there are well-known limitations and risks associated with them [keyword searches], and proper selection and implementation obviously involves technical, if not scientific knowledge.” Knowing that, why was she fearless? Why did she ignore Judge Facciola’s advice? Why did she make multiple detailed, technical decisions on legal search, including specific keywords to be used, without the benefit of expert testimony? Was that foolish as several judges have suggested, or was she just doing her job by making the decisions that the parties asked her to make?

Judge Fashing recognized that she did have enough facts to make a decision, much less expert opinions based on technical, scientific knowledge, but she went ahead and ruled anyway.

Although NMSU argues that the search terms proposed by the government will return a greater number of non-responsive documents than responsive documents, this is not a particular and specific demonstration of fact, but is, instead, a conclusory argument by counsel. See Velasquez, 229 F.R.D. at 200. NMSU’s motion for a protective order with regard to RFP No. 8 is DENIED.

NMSU will perform a search of the email addresses of all individuals involved in salary-setting for Ms. Harkins and her comparators, including Kathy Agnew and Dorothy Anderson, to include the search terms “Meaghan,” “Harkins,” “Gregory,” or “Fister” for the time period of 2007-2012. If this search results in voluminous documents that are non-responsive, NMSU may further search the results by including terms such as “cross-country,” “track,” “coach,” “salary,” “pay,” “contract,” or “applicants,” or other appropriate terms such as “compensation,” which may reduce the results to those communications most likely relevant to this case, and which would not encompass every “Meaghan” or “Gregory” in the system. However, the Court will require NMSU to work with the USA to design an appropriate search if it seeks to narrow the search beyond the four search terms requested by the United States.

Judge Fashing goes on to make several specific orders on what to do to make a reasonable effort to find relevant evidence:

NMSU will conduct searches of the OIE databases, OIE employee’s email accounts, and the email accounts of all head coaches, sport administrators, HR liaisons working within the Athletics Department, assistant or associate Athletic Directors, and/or Athletic Directors employed by NMSU between 2007 and the present. The USA suggests that NMSU conduct a search for terms that are functionally equivalent to a search for (pay or compensate! or salary) and (discriminat! or fair! or unfair!). Doc. 117-1 at 13. If NMSU cannot search with “Boolean” connectors as suggested, it must search for the terms “pay” or “compensate” or “salary” and “discriminate” or “fair” or “unfair” and the various derivatives of these terms (for example the search would include “compensate” and “compensation”). The parties are to work together to determine what terms will be used to search these databases and email accounts.

Judge Laura Fashing hangs her hat on cooperation, but not on experts. She concludes her order with the following admonishment:

The parties are reminded that:

Electronic discovery requires cooperation between opposing counsel and transparency in all aspects of preservation and production of ESI. Moreover, where counsel are using keyword searches for retrieval of ESI, they at a minimum must carefully craft the appropriate keywords, with input from the ESI’s custodians as to the words and abbreviations they use, and the proposed methodology must be quality control tested to assure accuracy in retrieval and elimination of “false positives.” It is time that the Bar—even those lawyers who did not come of age in the computer era—understand this.

William A. Gross Const. Assocs., Inc., 256 F.R.D. at 136.


Of course I agree with Judge Fashing’s concluding reminder to the parties. Cooperation is key, but so is expertise. There is a good reason for the fear felt by Facciola’s angels. They wisely  knew that they lacked the necessary technical, scientific knowledge for the proper selection and implementation of keyword searches. I only wish that Judge Fashing’s order had reminded the parties of this need for experts too. It would have made her job much easier and also helped the parties. Sometimes the wisest thing to do is nothing, at least not until you have more information.

There is widespread agreement among legal search experts on such simplistic methods as keyword search. They would have helped. The same holds true on advanced search methods, such as active machine learning (predictive coding), at least among the elite. See There is still some disagreement on TAR methods, especially when you include the many pseudo experts out there. But even they can usually agree on keyword search methods.

I urge the judges and litigants faced with a situation like Judge Fashing had to deal with in New Mexico State University, to consider the three choices set out by Judge Francis in Greater New York Taxi Association:

  1. Cooperation with the other side and their technical consultants to attempt to agree on an appropriate set of search criteria.
  2. Motions supported by expert testimony and facts regarding the search.
  3. Appointment of a neutral consultant who will design a search strategy.

Going it alone with legal search in a complex case is a fool’s errand. Bring in an expert. Spend a little to save a lot. It is not only the smart thing to do, it is also required by ethics. Rule 1.1: Competence, Model Rules of Professional Conduct. The ABA Comment two to Rule 1.1 states that “Competent representation can also be provided through the association of a lawyer of established competence in the field in question.” Yet, in my experience, this is seldom done and is not something that clients are clamoring for. That should change, and quickly, if we are ever to stop wasting so much time and money on simplistic e-discovery arguments. I am again reminded of the great Alexander Pope (1688–1744) and another of his famous lines from An Essay on Criticism.



After I wrote this blog I did a webinar for ACEDS about this topic. Here is a one-hour talk to add to your personal Pierian spring.






Good, Better, Best: a Tale of Three Proportionality Cases – Part Two

April 15, 2012

Continuation of Part One of Good, Better, Best: a Tale of Three Proportionality Cases.

The Best Case: DCG Systems

Compared to I-Med Pharma and U.S. ex rel McBride, DCG Systems is the best of the lot. DCG Sys., Inc. v. Checkpoint Techs, LLC, 2011 WL 5244356 (N.D. Cal. Nov. 2, 2011) It is better than the rest because of timing. The issue of proportionality of discovery was raised in DCG Systems at the beginning of the case. It was raised at the 26(f) conference and 16(b) hearing as part of discovery plan discussions. That is what the rules intended. Proportionality protection requires prompt, diligent action.

In I-Med Pharma the party responding to discovery waited to take action until after a stipulation and order to review 64,382,929 hits covering 95 Million pages. In U.S. ex rel McBride the party responding to discovery waited until after the email of 230 custodians had been produced, and, in the words of Judge Facciola, a king’s ransom had already been paid.

The lesson is clear. Be a good little lawyer hacker. Be fast, be bold, and be open to impact discovery in a proportional way.  Impactful, Fast, Bold, Open, Values: Guidance of the “Hacker Way.”  Timing is everything, in law and in life. Are we not all trapped in an hour-glass? There is no getting out!

Timing and Rule 26(g)

A key lesson of these three cases is that timing is everything. Consider proportionality from the get go, and remember that it is not only based on the protective order rule, 26(b)(2)(C), it is based on the rule governing a requesting party’s signing a discovery request. I am talking about the Rule 11 of discovery, Rule 26(g)(1)(B)(iii):

(g) Signing Disclosures and Discovery Requests, Responses, and Objections.

(1) Signature Required; Effect of Signature. Every disclosure under Rule 26(a)(1) or (a)(3) and every discovery request, response, or objection must be signed by at least one attorney of record in the attorney’s own name—or by the party personally, if unrepresented—and must state the signer’s address, e-mail address, and telephone number. By signing, an attorney or party certifies that to the best of the person’s knowledge, information, and belief formed after a reasonable inquiry: …

(B) with respect to a discovery request, response, or objection, it is: …

(iii) neither unreasonable nor unduly burdensome or expensive, considering the needs of the case, prior discovery in the case, the amount in controversy, and the importance of the issues at stake in the action.

Judge Paul Grimm calls Rule 26(g) the most overlooked and misunderstood of all of the rules of civil procedure. That is the fault of us lawyers, and, it is also the fault of our judges. Rule 26(g) in subsection (3) requires a court, “on its own,” to sanction anyone who signs a discovery request in violation of the rule. This means that judges must impose sanctions, on their own initiative, whenever they see a disproportionate discovery request. There is no discretion given to judges about this. The rule does not say “may” impose sanctions. It says the court “on motion or on its own, must impose an appropriate sanction on the signer.” Yet, in my thirty-two years of legal practice in federal court, I have never once seen this done by a judge. Have you?

Here is the language of subsection (3) of Rule 26(g). It should be in all of your discovery briefs.

(3) Sanction for Improper Certification. If a certification violates this rule without substantial justification, the court, on motion or on its own, must impose an appropriate sanction on the signer, the party on whose behalf the signer was acting, or both. The sanction may include an order to pay the reasonable expenses, including attorney’s fees, caused by the violation.

Lawyers need to start including this rule in their initial analysis of any discovery request. If one side refuses in engage in cooperative discussions to narrow discovery requests, if, for instance, they refuse to limit discovery to the actual factual issues in the case, then Rule 26(g) must be squarely brought to the attention of the supervising judge. There is no time to wait.  We are all trapped in an hour-glass, and a billable one at that!

As Judge Waxse has pointed out, there is a clear path in the rules to deal with non-cooperators, and Rule 26(g) is one of the road signs on that path. See: Judge David Waxse on Cooperation and Lawyers Who Act Like Spoiled Children. But you have to time your motions. You have to seek protection before you pay the piper, but after you make a good faith effort to cooperate. Timing is everything.

Model Patent Order

The Patent Bar is trying an experiment to try to control run away e-discovery costs in patent litigation. They have a committee composed of a handful of patent lawyers and a few key judges who are well-known in patent law, Chief Judge James Ware (ND Cal), Judge Virginia Kendall (ND Ill), Magistrate Judge Chad Everingham (ED Tex), and Chief Judge Randall Rader (Fed. Cir.). They have come up with what they call a Model Order Limiting E-Discovery in Patent Cases. They explain that the Model Order:

… is intended to be a helpful starting point for district courts to use in requiring the responsible, targeted use of e-discovery in patent cases. The goal of this Model Order is to promote economic and judicial efficiency by streamlining ediscovery, particularly email production, and requiring litigants to focus on the proper purpose of discovery—the gathering of material information—rather than permitting unlimited fishing expeditions. It is further intended to encourage discussion and public commentary by judges, litigants, and other interested parties regarding e-discovery problems and potential solutions.

The Model Order is inspired by Rule 30 that presumptively limits cases to ten depositions and seven hours per deposition. The Committee notes that since email is the biggest time-waster in patent litigation (well, except for Qualcomm of course), and so it uses this same limiting approach to email discovery. It limits initial e-discovery to email from five custodians and five keywords per custodian. The Committee is careful to note that “the parties may jointly agree to modify these limits or request court modification for good cause.” Even if they do not agree, or there is no order permitting more email discovery, a requesting party is still entitled to more if they pay for it. This is their approach to proportionality:

This is not to say a discovering party should be precluded from obtaining more e-discovery than agreed upon by the parties or allowed by the court. Rather, the discovering party shall bear all reasonable costs of discovery that exceeds these limits. This will help ensure that discovery requests are being made with a true eye on the balance between the value of the discovery and its cost.

The Model Order also addresses concerns regarding waiver of attorney-client privilege and work product protection in order to minimize human pre-production review. It does so by including Rule 502(d) non-waiver language into the standard order. The Order itself is pretty short and simple, which is one of its virtues, so I reproduce it here in its entirety:



The Court ORDERS as follows:

1. This Order supplements all other discovery rules and orders. It streamlines Electronically Stored Information (“ESI”) production to promote a “just, speedy, and inexpensive determination” of this action, as required by Federal Rule of Civil Procedure 1.

2. This Order may be modified for good cause. The parties shall jointly submit any proposed modifications within 30 days after the Federal Rule of Civil Procedure 16 conference. If the parties cannot resolve their disagreements regarding these modifications, the parties shall submit their competing proposals and a summary of their dispute.

3. Costs will be shifted for disproportionate ESI production requests pursuant to Federal Rule of Civil Procedure 26. Likewise, a party’s nonresponsive or dilatory discovery tactics will be cost-shifting considerations.

4. A party’s meaningful compliance with this Order and efforts to promote efficiency and reduce costs will be considered in cost-shifting determinations.

5. General ESI production requests under Federal Rules of Civil Procedure 34 and 45 shall not include metadata absent a showing of good cause. However, fields showing the date and time that the document was sent and received, as well as the complete distribution list, shall generally be included in the production.

6. General ESI production requests under Federal Rules of Civil Procedure 34 and 45 shall not include email or other forms of electronic correspondence (collectively “email”). To obtain email parties must propound specific email production requests.

7. Email production requests shall only be propounded for specific issues, rather than general discovery of a product or business.

8. Email production requests shall be phased to occur after the parties have exchanged initial disclosures and basic documentation about the patents, the prior art, the accused instrumentalities, and the relevant finances. While this provision does not require the production of such information, the Court encourages prompt and early production of this information to promote efficient and economical streamlining of the case.

9. Email production requests shall identify the custodian, search terms, and time frame. The parties shall cooperate to identify the proper custodians, proper search terms and proper timeframe.

10. Each requesting party shall limit its email production requests to a total of five custodians per producing party for all such requests. The parties may jointly agree to modify this limit without the Court’s leave. The Court shall consider contested requests for up to five additional custodians per producing party, upon showing a distinct need based on the size, complexity, and issues of this specific case. Should a party serve email production requests for additional custodians beyond the limits agreed to by the parties or granted by the Court pursuant to this paragraph, the requesting party shall bear all reasonable costs caused by such additional discovery.

11. Each requesting party shall limit its email production requests to a total of five search terms per custodian per party. The parties may jointly agree to modify this limit without the Court’s leave. The Court shall consider contested requests for up to five additional search terms per custodian, upon showing a distinct need based on the size, complexity, and issues of this specific case. The search terms shall be narrowly tailored to particular issues. Indiscriminate terms, such as the producing company’s name or its product name, are inappropriate unless combined with narrowing search criteria that sufficiently reduce the risk of overproduction. A conjunctive combination of multiple words or phrases (e.g., “computer” and “system”) narrows the search and shall count as a single search term. A disjunctive combination of multiple words or phrases (e.g., “computer” or “system”) broadens the search, and thus each word or phrase shall count as a separate search term unless they are variants of the same word. Use of narrowing search criteria (e.g., “and,” “but not,” “w/x”) is encouraged to limit the production and shall be considered when determining whether to shift costs for disproportionate discovery. Should a party serve email production requests with search terms beyond the limits agreed to by the parties or granted by the Court pursuant to this paragraph, the requesting party shall bear all reasonable costs caused by such additional discovery.

12. The receiving party shall not use ESI that the producing party asserts is attorney-client privileged or work product protected to challenge the privilege or protection.

13. Pursuant to Federal Rule of Evidence 502(d), the inadvertent production of a privileged or work product protected ESI is not a waiver in the pending case or in any other federal or state proceeding.

14. The mere production of ESI in a litigation as part of a mass production shall not itself constitute a waiver for any purpose.

This Model Order is a terrific first experiment to try to reign in disproportionate e-discovery expenses and stop wasting everybody’s time. Still, the plaintiff in DCG Systems did not like it and tried to avoid its application in its case. I have my own criticisms of the Model Order, including the obvious one of reliance on five blind keywords, and that puzzling para five on metadata, but I will save that for the conclusion.

DCG Sys., Inc. v. Checkpoint Techs, LLC

DCG Systems is a garden variety patent case between two companies with competing patent rights. It is not another very common type of patent case where a small patent troll with only a little ESI sues a big company with lots of ESI. They call those NPE cases. This means that in the DCG Systems case both companies could find e-discovery equally troubling. The plaintiff, DCH Systems Inc., argued that the Model Order should not be applied to their case because the Order was primarily designed for the David and Goliath, troll versus big company NPE type patent case.

United States Magistrate Judge Paul S. Grewal did not agree:

The court is not persuaded by DCG’s argument for at least two reasons. First, although the undersigned will not presume to know whether Chief Judge Rader or any of the esteemed members of the subcommittee were focused exclusively on reducing discovery costs in so-called “NPE” cases, there is nothing in the language of the Chief Judge’s speech or the text of the model order so limiting its application. Second, and more fundamentally, there is no reason to believe that competitor cases present less compelling circumstances in which to impose reasonable restrictions on the timing and scope of email discovery. To the extent DCG faces unique or particularly undue constraints as a result of the limitations, it remains free, under the Model Order, to seek relief from the court. But in general copying and the availability of an injunction are issues that are impacted by such restrictions no more than the myriad of other issues (e.g., inducement, state of the art, willfulness) that are present in just about all patent cases. And if competitor cases such as this lack the asymmetrical production burden often found in NPE cases, so that two parties might benefit from production restrictions, the Model Order would seem more appropriate, not less.

I know nothing about patent cases, but I do know e-discovery, and Judge Grewal’s argument sounds compelling. Judge Grewal ends his opinion with the following cautionary comment, words that I again completely agree with:

Perhaps the restrictions of the Model Order will prove undue. In that case, the court is more than willing to entertain a request to modify the limits. But only through experimentation of at least the modest sort urged by the Chief Judge will courts and parties come to better understand what steps might be taken to address what has to date been a largely unchecked problem.

We have to take new steps to control e-discovery costs, to make them proportionate. That is why I came up with my Bottom Line Driven Proportional Review approach. But the Patent Committee approach has the advantage of far greater simplicity. Moreover, little or no skill in e-discovery is required to implement this proportionality reform. Still, I am troubled by the reliance on Go Fish keyword search methods. See Child’s Game of “Go Fish” is a Poor Model for e-Discovery Search. The lack of precision and recall in blind keyword search makes this method both expensive and ineffective.

Methods aside, the Model Order Limiting E-Discovery in Patent Cases makes an important first step in litigation reform. The DCG Systems case shows timely application of the Model Order. The opinion also includes good language explaining the order and why courts should try using it to attain proportionality. (It’s use is at the discretion of the presiding judge.) You may want to use Judge Grewal’s language in DCG Systems in your case memos, patent or otherwise, to show the need to control e-discovery:

Critically, the email production requests must focus on particular issues for which that type of discovery is warranted. The requesting party must further limit each request to a total of five search terms and the responsive documents must come from only a defined set of five custodians. These restrictions are designed to address the imbalance of benefit and burden resulting from email production in most cases. As Chief Judge Rader noted in his recent address in Texas on the “The State of Patent Litigation” in which he unveiled the Model Order, “[g]enerally, the production burden of expansive e-requests outweighs their benefits. I saw one analysis that concluded that .0074% of the documents produced actually made their way onto the trial exhibit list-less than one document in ten thousand. And for all the thousands of appeals I’ve evaluated, email appears more rarely as relevant evidence.”

Remember that statistic and use it. Only .0074% of e-docs discovered ever make it onto a trial exhibit list, much less ever get used to make a difference in a case. That is why in my Secrets of Search article, Part Three, I say Relevant Is Irrelevant and point out the old trial psychology rule of 7±2, to argue for higher culling rates in e-discovery search.

More Authorities on Proportionality

Want to learn more about proportionality? Don’t rely on a keyword search to find the cases. As seen, they often do not even use the word proportionality. Try these additional articles, cases, and Mr. Shepherd instead. Mr. Google will help you find still more.

  • The Sedona Conference® Commentary on Proportionality in Electronic Discovery.
  • Bottom Line Driven Proportional Review.
  • Discovery As Abuse.
  • An Old Case With a New Opinion Demonstrating Perfect Proportionality.
  • Rimkus Consulting Group v. Cammarata, 688 F.Supp. 2d 598, 613 (S.D. Tx. 2010) (the Rules require that the parties engage in “reasonable efforts” and what is reasonable “depends on whether what was done – or not done – was proportional to that case…”)
  • Moody v. Turner Corp. Case No. 1:07-cv-692. (S.D. OH, 2010) (“…the mere availability of such vast amounts of electronic information can lead to a situation of the ESI-discovery-tail wagging the poor old merits-of-the-dispute dog.”)
  • Dilley v. Metro. Life Ins. Co., 256 F.R.D. 643, 644 (N.D. Cal. 2009) (“The court must limit discovery if it determines that ‘the burden or expense of the proposed discovery outweighs its likely benefit,’ considering certain factors including ‘the importance of the issues at stake in the action, and the importance of the discovery in resolving the issues.’” ) (quoting FED. R. CIV. P. 26(b)(2)(C)(iii))
  • Averett v. Honda of Am. Mfg., Inc., No. 2:07-cv-1167, 2009 WL 799638, at *2 (“the court always has a duty to limit discovery under Rule 26(b)(2)(C)(i)-(iii)”)
  • Wood v. Capital One Services, LLC, No. 5:09-CV-1445 (NPM/DEP), 2011 WL 2154279, at *1-3, *7 (N.D.N.Y, 2011) (the “rule of proportionality” dictated that the plaintiff’s motion be denied “without prejudice to his right to renew the motion to compel in the event he is willing to underwrite the expense associated with any such search.”)
  • Thermal Design, Inc. v. Guardian Building Products, Inc., No. 08-C-828 (E.D. Wis., 2011), (Judge refused to approve plaintiff’s electronic fishing expedition simply because the defendant had the financial resources to pay for the searches. Th financial resources of the defendant are not tantamount to good cause under FRCP 26(b)(2)(C))
  • General Steel Domestic Sales, LLC v. Chumley, No. 10-cv-01398 (D. Colo., 2011) (Judge rejected defendant’s request for the production of every recorded sales call on plaintiff’s database for a two-year period because it would take four years to listen to the calls to identify potentially responsive information.)
  • Daugherty v. Murphy, No. 1:06-cv-0878-SEB-DML, 2010 WL 4877720, at *5 (S.D. Ind., 2010) (The cost and burden of the additional production outweighed the benefit. The defendant’s sworn testimony on burden and cost was credible.)
  • Willnerd v. Sybase, 2010 U.S. Dist. LEXIS 121658 (SD Id., 2010)(“… a search of the employees’ e-mails would amount to the proverbial fishing expedition — an exploration of a sea of information with scarcely more than a hope that it will yield evidence to support a plausible claim of defamation. … In employing the proportionality standard of Rule 26(b)(2)(C), as suggested by Willnerd, the Court balances Willnerd’s interest in the documents requested, against the not-inconsequential burden of searching for and producing documents.”)
  • Rodriguez-Torres v. Gov. Dev. Bank of P.R., 265 F.R.D. 40 (D. P.R., 2010) (“… the Court determines that the ESI requested is not reasonably accessible because of the undue burden and cost. The Court finds that $35,000 is too high of a cost for the production of the requested ESI in this type of action. Moreover, the Court is very concerned over the increase in costs that will result from the privilege and confidentiality review that Defendant GDB will have to undertake on what could turn out to be hundreds or thousands of documents.”
  • Madere v, Compass Bank, 2011 U.S. Dist. LEXIS 124758, (WD Tx. 2011) (“As the cost to restore Compass Bank’s backup tapes “outweighs its likely benefit,” especially in light of the amount in controversy, the Court DENIES Madere’s request for production.”)
  • Convolve, Inc. v. Compaq Comp. Corp, 223 F.R.D. 162 (SDNY 2004) (The production request “would require an expenditure of time and resources far out of proportion to the marginal value of the materials to this litigation.”)
  • United Central Bank v. Kanan Fashions, Inc., 2010 U.S. Dist. LEXIS 83700 (DN Ill, 2010) (Restrictive date range required, but further protection from excessive burden denied due to failure to support the contentions of high cost to comply with specific facts.)
  • High Voltage Beverages, LLC v. Coca-Cola Co., 2009 U.S.Dist. LEXIS 88259 (WD NC, 2009) (“Under Rule 26(b)(2)(C)(i), the court finds that requiring defendant to sift sand for documents it has already produced would be unreasonably duplicative of earlier efforts and that the material contained therein is likely available from other sources, to wit, an earlier production of documents. … Under Rule 26(b)(2)(C)(iii), defendant has made an unrebutted showing that the man-hours and expense of reviewing the collection would be extraordinary, and it appears to the court that the burden or expense of the proposed discovery outweighs its likely benefit. Thus, the court find that it would be disproportional to require defendant to review such information prior to producing it to plaintiff and deny plaintiff’s request.”)
  • Bassi Bellotti S.p.A. v. Transcon. Granite, Inc., 2010 U.S. Dist. LEXIS 93055 (D. Md., 2010) (“… Federal Rules do impose an obligation upon courts to limit the frequency or extent of discovery sought in certain circumstances, such as when the discovery requested is unreasonably duplicative or cumulative, or the burden or expense of the proposed discovery outweighs the likely benefit, considering the needs of the case, the importance of the issues at stake in the action, and the importance of the discovery in resolving those issues. “)
  • Call of the Wild Movie, LLC v. Does 1-1062, No. 10-455 (BAH), — F. Supp. 2d —-, 2011 WL 996786, at *18-20 (D.D.C., 2011) (granting motion to compel because the request was narrow and the ESI requested was important, compared with an insufficient showing of undue burden.)
  • Hock Foods, Inc. v. William Blair & Co., LLC, No. 09-2588-KHV, 2011 WL 884446, at *9 (D. Kan. 2011) (Sebelius, Maj. J.) (denying in part a motion to compel in light of costs estimated between $1.2 and $3.6 million to search 12,000 gigabytes of data in order to answer an overbroad interrogatory.)
  • Diesel Mach., Inc. v. Manitowoc Crane, Inc., No. CIV 09-cv-4087-RAL, 2011 WL 677458, at *2-3 (D.S.D., 2011) (motion to compel the production of documents in native format was denied because no explanation provided on why information contained in native format was necessary to facts of case when those same documents had already been produced as PDFs).
  • Tucker v. American Intern. Group, Inc., 2012 WL 902930 (D. Conn. Mar. 15, 2012) (Plaintiff’s non-party Rule 45 subpoena to inspect hard drives asked the Court “to allow plaintiff “essentially carte blanche access to rummage through Marsh’s electronically stored information, purportedly in the hope that the needle she is looking for lurks somewhere in that haystack. … [T]he burdens of plaintiff’s proposed inspection upon Marsh outweigh the benefits plaintiff might obtain were she to obtain the emails through a Datatrack inspection. Plaintiff seeks to search, inter alia, the mirror images of eighty-three laptops — in effect, to dredge an ocean of Marsh’s electronically stored information and records in an effort to capture a few elusive, perhaps non-existent, fish. … Courts are obliged to recognize that non-parties should be protected with respect to significant expense and burden of compelled inspections under Fed. R. Civ. P. 45(c)(2)(B)(ii). … Moreover, courts have focused on the importance of the Rule 26(b)(2)(C) proportionality limit to implement fair and efficient operation of discovery. … Balancing the prospective burden to Marsh against the likely benefit to plaintiff from the proposed inspection, the Court concludes that the circumstances do not warrant compelling Marsh to endure inspection of its computer records by Datatrack.”)


DCG Systems, Inc. v. Checkpoint Techs, LLC is, by far, the best of the three cases, but it is still far from perfect. It embraces proportionality, and will no doubt save the parties lots of money in e-discovery, but at what cost? Litigation is about finding justice. If you lose that. You lose everything.

Rule 1 says, among other things, that litigation should be speedy and inexpensive. Limiting discovery to five keywords and five custodians will get you that. But Rule 1 also says litigation should be just. That is, after all, the whole point of litigation. In America, like most of the civilized world, we don’t just go through the motions of legal process in a fast and cursory manner. Court systems are not just an empty charade. The heart of law as we know it is due process. We decide cases on the merits, on the facts, on the evidence; not just on the whim of judges or juries. That is what justice means to us. I am concerned about arbitrary limits on e-discovery to save money, and speed things along, that do so at the price of justice.

Judge Paul S. Grewal, who decided DCG Systems, shares these concerns, I am sure. So too does the Patent Bar who adopted this Model Order, and Chief Judge Randall Rader who promotes it. They are, like all bona fide professionals in the Law, trying hard to find a proportional balance between benefit and burden, to know when enough is enough in the search for evidence. They don’t want too much, like some unscrupulous attorneys for whom e-discovery is little more than a legal tool of extortion. They don’t want too little, like some equally unscrupulous attorneys who play hide the ball. Good attorneys are like Goldilocks; they are looking for the just-right amount of e-discovery. They are looking for proportionality.

The patent judges show this concern in the pains they take to say that the five/five rule is just a starting point. They make clear that more e-discovery outside of these limits may be appropriate, that parties can always move the court for additional discovery. For instance, Judge Grewal in DCG Systems says: “Perhaps the restrictions of the Model Order will prove undue. In that case, the court is more than willing to entertain a request to modify the limits.” The Model Order shows the same concern that justice not be sacrificed at the altar of efficiency: “The Court shall consider contested requests for up to five additional custodians per producing party, upon showing a distinct need based on the size, complexity, and issues of this specific case.

My main criticism of the case and Model Order, aside again from the bizarre comment in paragraph five against metadata, pertains to the reliance on Go Fish type keyword search. It is not so much the arbitrary limit to five keywords that bothers me, much less the limit to five custodians, which I think is fine. What bothers me about the Model Order, and bothers every other expert I have talked to, is the reliance on keyword search alone, and blind-pick keyword search at that. It should bother anyone who has read the scientific studies. The Model Order is promoting the worst kind of search: the blind keyword guessing kind. That is inadvertent I’m sure. The lawyers and judges behind the model order were not aware of the limits of blind-guessing-based-keywords. When they do, I assume they will consider appropriate revisions to the Model Order.

The Model Order should be reformed to require that basic metrics be shared on proposed keywords. It should require enough disclosure so that the keyword picks are not blind. Some keyword testing should be permitted a requesting party before five terms are settled upon. The Order is a good start, but it needs tweaking so that the keyword searches can be more effective. I am sure there are many search experts who would help the Committee if asked. I hope they do, because the Patent Bar’s heart is in the right place, a proportionality place.

Now please, would someone get me out of this damn time bottle?







Thank You!

Good, Better, Best: a Tale of Three Proportionality Cases – Part One

April 8, 2012

Three cases came out recently on proportionality, the key legal doctrine to discovery based on Federal Rules 26(b)(2)(C), 26(b)(2)(B)(iii), and 26(g)(1)(B). I-Med Pharma Inc. v. Biomatrix, 2011 WL 6140658 (D.N.J. Dec. 9, 2011) (GOOD); U.S. ex rel McBride v. Halliburton Co.,, 272 F.R.D. 235 (D.D.C. 2011) (BETTER); DCG Sys., Inc. v. Checkpoint Techs, LLC, 2011 WL 5244356 (N.D. Cal. Nov. 2, 2011) (BEST).  Since all three cases embody proportionality, they are all good. But some are better looking than others.

The quality of the application of the doctrine in these cases is directly tied to the parties timing. In the best case the issue was raised fast, even before discovery. It was raised in the 26(f) conference and 16(b) hearing. DCG Sys., Inc. v. Checkpoint Techs, LLC, 2011 WL 5244356 (N.D. Cal. Nov. 2, 2011). Counsel followed the Facebook good-hacker credo, as I explained in Impactful, Fast, Bold, Open, Values: Guidance of the “Hacker Way.”  They were fast, and their bold use of proportionality will likely have a big impact on the case. With the guidance of the wise judge supervising the parties’ discovery, U.S. Magistrate Judge Paule S. Grewal, the application of the proportionality doctrine should lead to open, yet cost-controlled discovery. This should allow both parties in the patent case to focus on  the merits. It should help avoid wasteful email reading endurance contests. Is that not a positive value we should all endorse? Answer: Yes! See Rule 1 and Judge Waxse’s article, Cooperation—What Is It and Why Do It?, as discussed in last week’s blog, Judge David Waxse on Cooperation and Lawyers Who Act Like Spoiled Children.

The Good Case: I-Med Pharma

Before I get into the California case that shows the right way to raise proportionality – at the beginning of the case – let’s look at the other two cases. The responding parties in both cases raised proportionality, but they came to the party a tad late. The worst of these good cases illustrates the saying better late than never. I-Med Pharma Inc. v. Biomatrix, 2011 WL 6140658 (D.N.J. Dec. 9, 2011). The party responding to discovery, the plaintiffs, finally raised the doctrine when their backs were against the wall fighting sanctions for failing to comply with a discovery order they had stipulated to.

Yup. You read that right. The plaintiff, I-Med Pharma Inc., was seeking relief on the basis of disproportionate burden from a discovery order they had agreed to. Kind of makes you wonder. But no explanation is provided in this opinion as to why plaintiff’s counsel in this case agreed to review and produce all non-privileged files that matched a ridiculously long list of keywords from opposing counsel. This was a contract dispute and the keyword list dreamed up by defense counsel, who apparently engaged in a rousing game of “Go Fish,” included such zingers as: contract*, loss, profit*, credit, refund, revenue, CL, HS*, return, claim, FDA, HA. I could go on, but you get the picture. (Side note: when I say keyword search sucks, as I did in Secrets of Search: Part One, this is the kind of search I am referring to: the blind guessing, Go Fish, linear kind with no quality controls. I am certainly not referring to the kind of iterative keyword search on steroids that we see in Kleen Products, which appears to be almost as good as predictive coding based hybrid multimodal methods. But I digress.)

In I-Med Pharma the attorneys not only used Go Fish keyword search, the kind that sucks, they agreed to have the search run by an outside forensic expert with no limits placed on target custodians. It was a search of the plaintiff’s entire corporate computer system. Not only that, there were no time limits placed on the search. To make matters worse, they not only agreed to search the active files with word matches, they agreed to search the slack space too. That is the so-called “unallocated space files” recovered by a  forensic exam of plaintiff’s computer system.

Yup. You read all that right again. No wonder the wise judge presented with this conundrum, Senior U.S. District Court Judge Dickinson R. Debevoise, began his opinion with these words:

This case highlights the dangers of carelessness and inattention in e-discovery.

Boy did Judge Debevoise get that right!

Plaintiff’s counsel finally woke up and discovered proportionality (here is where we get to the better late than never part), when the forensic expert searched the unallocated space of their client’s computer system and found 64,382,929 hits covering the equivalent of 95 Million pages of documents! Based on the complete failure to limit the search to custodians, or date, and the Go Fish type list of keywords, this result, in Judge Debevoise’s own words, should come as no surprise. Id. at pg. 5.

The opinion does not say how many pages of documents with hits were found in the allocated spaces of the system, the active files, but it was probably millions more. I-Med Pharma, Inc., apparently did not oppose the privilege review and production of these documents. No doubt they paid millions in vendor costs and attorney fees to comply with this portion of the stipulation.

Since plaintiff’s counsel by now probably had a pretty good idea of what a privilege review of another 95 Million pages of mostly gibberish from slack space might cost, and since at this point the client probably did not want to pay for more, plaintiff’s counsel said no. They asked defense counsel to give them a break on their prior agreement. Defense counsel said no to that, a deals a deal, and  perhaps sensing victory, refused to modify the prior stipulation. Then plaintiff moved for a protective order asking relief from the prior stipulation on discovery that had, as a matter of course, been converted to an order.

Here is where plaintiff raised the doctrine of proportionality and suggested that the costs and burdens to review 64,382,929 hits covering the equivalent of 95 Million pages of documents from slack space would exceed any possible benefit from that exercise. The Magistrate assigned to hear the dispute, Judge Shipp, quickly agreed, and the defendant, having little to lose (except perhaps credibility), appealed the decision to Judge Debevoise.

Judge Debevoise, of course, affirms his magistrate, which is why, after all, I call this comedy of errors a good decision. Judge Debevoise is a master of understatement and notes that:

A privilege review of 65 million documents is no small undertaking. Even if junior attorneys are engaged, heavily discounted rates are negotiated, and all parties work diligently and efficiently, even a cursory review of that many documents will consume large amounts of attorney time and cost millions of dollars.

Id. at pg. 10.

Judge Debevoise granted a hearing on plaintiff’s appeal, which I imagine would have been interesting to observe, since this judge really seems to have a good understanding of e-discovery. At the hearing defense counsel argued that plaintiff’s obligation to review 95 Million pages was not really that burdensome. As footnote six of the opinion explains, Judge Debevoise responded by asking defense counsel how they would do a privilege review of that many documents. That’s where it gets fun, as defendants’ counsel said they would simply run a search for the word privilege and only review the documents with that word. Uh, huh. Sure. Judge Debevoise observed in footnote six:

In spite of the answer given, it is difficult to believe that lawyers from xxxxxxx and xxxxxxxx regularly disclose large quantities of information from their client’s files without examining it.

Id. at pg. 10-11, FN 6. (self censored, hey it could happen to any otherwise good firm.)

So, Judge Debevoise let’s plaintiff’s counsel off the hook and relieves them of their prior e-discovery agreement. But he has some choice words for them too (and again I will not name plaintiff’s counsel), and provides good advice for all on a better way to do keyword search, a way that goes far beyond the simple guessing game the attorneys in this case had apparently been playing:

While the precise number of hits produced was not known in advance and Plaintiff argues that it could not have predicted the volume of material that the search would uncover, it should have exercised more diligence before stipulating to such broad search terms, particularly given the scope of the search. In evaluating whether a set of search terms are reasonable, a party should consider a variety of factors, including: (1) the scope of documents searched and whether the search is restricted to specific computers, file systems, or document custodians; (2) any date restrictions imposed on the search; (3) whether the search terms contain proper names, uncommon abbreviations, or other terms unlikely to occur in irrelevant documents; (4) whether operators such as “and”, “not”, or “near” are used to restrict the universe of possible results; (5) whether the number of results obtained could be practically reviewed given the economics of the case and the amount of money at issue.

While Plaintiff should have known better than to agree to the search terms used here, the interests of justice and basic fairness are little served by forcing Plaintiff to undertake an enormously expensive privilege review of material that is unlikely to contain non-duplicative evidence.

Id. at pgs. 11-12.

I-Med Pharma Inc. v. Biomatrix, is a good case, not only for proportionality, but also for search. It is very telling to note that even though the case embodies the doctrine of proportionality, the keyword itself is never used – even Rule 26(b)(2)(C) is never referred to. This once again demonstrates the limits of keyword search. I would not have found this case but for the genius of a non-linear, protodigital, artificial intelligence agent I know, a mobile Siri of e-discovery, Kenneth J. Withers. Thank you Ken. (Actually, truth be known, he found all three of these cases.)

The Better Case: U.S. ex rel McBride

The next case demonstrating proportionality is U.S. ex rel McBride v. Halliburton Co. , 272 F.R.D. 235 (D.D.C. 2011). It is a better case than I-Med-Pharma because proportionality was applied in a slightly more timely fashion. True, discovery had already closed, but at least protection was sought before stipulation to an order. It is also a good case because it was written by one of my favorite judges, a master wordsmith and  e-discovery guru, John M. Facciola. He’s got a proportional look too.

Judge Facciola is a strong advocate for proportionality. By the way, that does not mean the judge has prejudged anything, or favors responding parties over requesting parties, or anything like that. It is the same thing as saying a judge favors reasonability, or motherhood, or apple pie, or, dare I say it, modern efficient search and review methods.

Judge Facciola was one of my keynote speakers in a CLE event that I co-chaired in 2010 with Maura Grossman. The sole topic of that day-long event was proportionality. It was one of the best CLEs I have ever done for that reason. There was a thorough examination of one issue from some of the top minds in e-discovery, including especially the top judges who specialize in the subject. Judge Facciola started us off by speaking of proportionality in the law and art and music where the principle is called the golden mean or golden ratio. He even played a few lines of Bach that demonstrated the golden ratio in music. I followed his presentation with my own where I spoke and showed slides on proportionality in the law and art, paintings and design. I showed famous paintings, buildings (Parthenon) and other golden ratio based designs (iPods) and examined how this ratio would apply to e-discovery costs. It seemed like part two of what Judge Facciola had started.

Judge Facciola and I were both blown away by the eerie coincidence of our art based approaches to explaining proportionality in the law. I am sure everyone there thought this was a well orchestrated presentation. But the truth is, I had no idea he was going to use the same art based approach to proportionality. Neither of us had ever talked to the other about what we were going to say. (You don’t need to have prep talks with a pro like him.) The truth is, there was no need for us to talk and prepare in order to have harmonious presentations. There is a high degree of consensus and similar thinking among judges and lawyers who specialize in this area. We have all read each others writings and heard each other speak a number of times. We think about this stuff, a lot. We tend to reach the same conclusions. There is a general consensus on most issues. What’s wrong with that? But I digress.

Back to U.S. ex rel McBride v. Halliburton Co.. Although Judge Facciola is an expert and strong proponent of proportionality, my keyword search of the opinion shows that he never once uses the word in this opinion. He cites Rule 26(b)(2)(C) several times to be sure, but never says proportionality, proving once again the limits of that old keyword technology. For more on new search alternatives to 1940s era software methods, new technologies that favor the truth, not one side of the Bar or another, see (in reverse chronological order) my following recent articles (yes, I’m obsessed with this):

So Judge Facciola implements proportionality without ever saying the word, and grants the defendant’s motion for protective order. Here Halliburton in this qui tam action had already reviewed and produced relevant emails of 230 custodians. Yup, you read that right, 230! Discovery closed and what does the plaintiff want? They want Halliburton to search and produce still more email from an additional 35 custodians. These additional custodians were now targeted by the plaintiff, McBride, because they were shown as CCs on emails transmitting relevant documents that were already produced as part of the 230. No other reason provided. Let’s listen to how the Sage from D.C., who loves the mathematical proportionality of Johann Sebastian Bach (not to mention Bruce Springsteen), deals with such a dissonant request:

In excruciating, but highly educational and useful, detail, she (one of Halliburton’s e-discovery compliance managers) explains how the process and collection of data from a current or former employee takes place, from what may be many different sources … Once the data is found, it must be copied in a forensically appropriate manner to preserve its metadata and prevent its alteration. … The data must then be processed to be rendered searchable by the review tool being used, a process that can overwhelm the computer’s capacity and require that the data be processed by batch, as opposed to all at once. …

All discovery, even if otherwise permitted by the Federal Rules of Civil Procedure because it is likely to yield relevant evidence, is subject to the court’s obligation to balance its utility against its cost. Fed.R.Civ.P. 26(b)(2)(C). More specifically, the court is obliged to consider whether (1) the discovery sought is unreasonably cumulative or duplicative, or obtainable from a cheaper and more convenient source; (2) the party seeking the discovery has had ample opportunity to obtain the sought information by earlier discovery; or (3) the burden of the discovery *241 outweighs its utility. Id. The latter requires the court to consider (1) the needs of the case; (2) the amount in controversy; (3) the parties’ resources; (4) the importance of the issues at stake in the action; and (5) the importance of the discovery in resolving the issues. Fed.R.Civ.P. 26(b)(2)(C)(iii).

While the present record does not permit a precise conclusion, I can presume, given the numbers of hours for which the defendants billed and the period of time at issue, that the amount in controversy is great and that the defendants’ resources are greater than the relator’s. Claims of fraud in providing services to military personnel raise important, vital issues of governmental supervision and public trust. Thus, these factors might weigh in favor of the discovery sought.

On the other hand, the defendants protest, and relator does not deny, that they have already spent a king’s ransom on discovery in this case–$650,000–without the addition of attorneys’ fees.  They have produced more than two million paper documents, thousands of spreadsheets, and over a half a million e-mails.

Id. at pgs. 240-241 (record citations omitted).

Judge Facciola goes on, and note once again that he never says proportion* or any variation thereof:

Given the discovery that relator has had, what defendants have already spent, and the detailed showing made of how much more time and money will likely have to spent to search an additional thirty-five custodians, surely relator has to make a showing that the e-mails not produced are crucial to her proof. She has not made such a showing, and they are not. First, she has the LOGREP reports, and no one is pretending that defendants are (or could) be asserting that, without the transmitting e-mails, she cannot establish the submission of a false claim. To the contrary, the defendants conceded that point. Thus, the transmitting e-mails seem to be hopelessly insignificant.

In this context, it is telling that relator does not show from the e-mails she has received that there is good reason to believe that the ones she claims are missing are highly probative of some fact. Indeed, there is no showing whatsoever from what has been produced that those e-mails not produced will make the existence of some crucial fact more likely than not. It is, after all, unlikely that a transmitting e-mail will do any more than transmit attached information and, by copy, alert others of that transmittal.

Without any showing of the significance of the non-produced e-mails, let alone the likelihood of finding the “smoking gun,” the search relator demands cannot possibly be justified when one balances its cost against its utility. The motion will be denied.

Id. at 241.

Well put Judge Facciolla. This is music to the ears of any e-discovery devotee. I am reminded of my favorite, the Goldberg Variations. There are additional melodic words in this opinion about a party not losing their right to proportionate discovery because of preservation failures, but I suggest you read the opinion yourself for that last variation. Id. at 241-242.

The Best Case: DCG Systems

End of Part One. Stay tuned for the exciting conclusion, where you will hear the happy ending to this tale of three cases with the story of the best case, DCG Systems, more case-law on proportionality, and an opinion-riddled conclusion that will leave your jaws gaping.

As a sneak preview on the ending I will say this much. Although I agree with Siri’s explanation of DCG Systems, err, I mean, Ken Withers’ explanation: DCG Systems is a case that “proactively cuts the Gordian Knot by adopting Judge Rader’s Model Patent Discovery Order,” (yes, he actually talks like that, and with links too), I nevertheless have some criticisms of this case and the Model Order on which it is based. Its reliance on five keywords is flawed. Still, given the cost of most vendor’s predictive coding software these days, and the weak understanding most lawyers have of legal search, this reliance on outdated technology and search methods is to be expected.

Now for something completely different, I leave you with a few lyrics from Judge Facciola’s favorite musician-poet, Bruce Springsteen:

That you know flag flying over the courthouse
Means certain things are set in stone
Who we are, what we’ll do and what we won’t.

“Long Walk Home”

God have mercy on the man who doubts what he’s sure of.

“Brilliant Disguise”

But it’s a sad man my friend who’s livin’ in his own skin
And can’t stand the company.
Every fool’s got a reason to feelin’ sorry for himself
And turn his heart to stone.
Tonight this fool’s halfway to heaven and just a mile outta hell
And I feel like I’m comin’ home.

“Better Days”

So let’s take the good times as they go and I’ll meet you further on up the road…

“Further On (Up the Road)”

%d bloggers like this: