Proportionality Φ and Making It Easy To Play “e-Discovery: Small, Medium or Large?” in Your Own Group or Class

November 26, 2017

Every judge who has ever struggled with discovery issues wishes that the lawyers involved had a better understanding of proportionality, that they had spent more time really thinking about how it applies to the requisites of their case. So too does every lawyer who, like me, specializes in electronic discovery. As Chief Justice Roberts explained in his 2015 Year-End Report on the Federal Judiciary on the new rules on proportionality:

The amended rule states, as a fundamental principle, that lawyers must size and shape their discovery requests to the requisites of a case. Specifically, the pretrial process must provide parties with efficient access to what is needed to prove a claim or defense, but eliminate unnecessary or wasteful discovery. The key here is careful and realistic assessment of actual need.

Proportionality and reasonableness arise from conscious efforts to realistically assess actual need. What is the right balance in a particular situation? What are the actual benefits and burdens involved? How can you size and shape your discovery requests to the requisites of a case?

There is more to proportionality than knowing the rules and case law, although they are a good place to start. Proportionality is a deep subject and deserves more than black letter law treatment. 2015 e-Discovery Rule Amendments: Dawning of the “Goldilocks Era” (, 11/11/15) (wherein I discuss proportionality, the Golden Ratio or perfect proportionality, aka Φ, which is shown in this graphic and much more, including the spooky “coincidence” at a CLE with Judge Facciola and the audience vote). Also see: Giulio Tononi, Phi Φ, a Voyage from the Brain to the Soul (Pantheon Books, 2012) (book I’m rereading now on consciousness and integrated information theory, another take on Phi Φ).

We want everyone in the field to think about proportionality. To be conscious of it, not just have information about it. What does proportionality really mean? How does it apply to the e-discovery tasks that you carry out every day? How much is enough? Too much? Too burdensome? Too little? Not enough? Why?

What is a reasonable effort? How do you know? Is there perfect proportionality? One that expresses itself in varying ways according to the facts and circumstances? Does Law follow Art? Is Law an Art? Or is it a Science? Is there Beauty in Law? In Reason? There is more to proportionality than meets the eye. Or is there?

Getting people to think about proportionality is one of the reasons I created the Hive Mind game that I announced in my blog last week: “e-Discovery: Small, Medium of Large?”

This week’s blog continues that intention of getting lawyers to think about proportionality and the requisites of their case. It concludes with a word document designed to make it easier to play along with your own group, class or CLE event. What discovery activities required in a Big Case are not necessary in a Small Case, or even a Medium Sized case? That is what requires thought and is the basis of the game.

Rules of Federal Procedure

Proportionality is key to all discovery, to knowing the appropriate size and shape of discovery requests in order to fit the requisites of a case. Reading the rules that embody the doctrine of proportionality is a good start, but just a start.  The primary rule to understand is how proportionality effects the scope of relevance as set forth in Rule 26(b)(1), FRCP:

Parties may obtain discovery regarding any nonprivileged matter that is relevant to any party’s claim or defense and proportional to the needs of the case, considering the importance of the issues at stake in the action, the amount in controversy, the parties’ relative access to relevant information, the parties’ resources, the importance of the discovery in resolving the issues, and whether the burden or expense of the proposed discovery outweighs its likely benefit.

But you also need to understand how it impacts a lawyer’s overall duty to supervise a discovery request and response as set forth in Rule 26(g). See Rule 26(g)(1)(B)(iii), FRCP:

neither unreasonable nor unduly burdensome or expensive, considering the needs of the case, prior discovery in the case, the amount in controversy, and the importance of the issues at stake in the action.

Many other rules have concepts of proportionality either expressly or implicitly built in, including Rule 26(b)(2)(B) (not reasonably accessible); Rule 26(b)(2)(C)(i) (cumulative); Rule 1 (just, speedy and inexpensive), Rule 34, Rule 37(e), Rule 45.

Case Law

Reading the key cases is also a help, indispensable really, but reading what the judges say is not enough either. Still you need to keep up with the fast growing case law on proportionality. See for instance the fine collection by K&L Gates at: and the must-read, The Sedona Conference Commentary on Proportionality_May 2017. Here a few of my favorites cases:

  • In re Bard IVC Filters Prods. Liab. Litig., D. Ariz., No. MDL 15-02641-PHX DGC, 2016 U.S. Dist. LEXIS 126448 (D. Ariz. Sept. 16, 2016). In this must-read opinion District Judge David G. Campbell, who was the chair of the Rules Committee when the 2015 amendments were passed, takes both lawyers and judges to task for not following the new rules on proportionality. He then lays it all out in a definitive manner.
  • In re Takata Airbag Prods. Liab. Litig., No. 15-02599-CIV-Moreno, MDL No. 5-2599 (S.D. Fla. Mar. 1, 2016). Judge Moreno quotes Chief Justice Roberts’ comments in the 2015 Year-End Report that the newly amended Fed.R.Civ.Pro. 26 “crystalizes the concept of reasonable limits in discovery through increased reliance on the common-sense concept of proportionality.” 2015 Year-End Report on the Federal Judiciary.
  • Hyles v. New York City, No. 10 Civ. 3119 (AT)(AJP), 2016 WL 4077114 (S.D.N.Y. Aug. 1, 2016) (Judge Peck: “While Hyles may well be correct that production using keywords may not be as complete as it would be if TAR were used, the standard is not perfection, or using the “best” tool, but whether the search results are reasonable and proportional. Cf. Fed. R. Civ. P. 26(g)(1)(B)”)
  • Johnson v Serenity TransportationCase No. 15-cv-02004-JSC (N.D. Cal. October 28, 2016) (“… a defendant does not have discretion to decide to withhold relevant, responsive documents absent some showing that producing the document is not proportional to the needs of the case.”)
  • Apple Inc. v. Samsung Elecs. Co., No. 12-CV-0630-LHK (PSG), 2013 WL 4426512, 2013 U.S. Dist. LEXIS 116493 (N.D. Cal. Aug. 14, 2013) (“But there is an additional, more persuasive reason to limit Apple’s production — the court is required to limit discovery if “the burden or expense of the proposed discovery outweighs its likely benefit.” This is the essence of proportionality — an all-to-often ignored discovery principle. Because the parties have already submitted their expert damages reports, the financial documents would be of limited value to Samsung at this point. Although counsel was not able to shed light on exactly what was done, Samsung’s experts were clearly somehow able to apportion the worldwide, product line inclusive data to estimate U.S. and product-specific damages. It seems, well, senseless to require Apple to go to great lengths to produce data that Samsung is able to do without. This the court will not do.)
  • PTSI, Inc. v. Haley, 2013 WL 2285109 (Pa. Super. Ct. May 24, 2013) (“… it is unreasonable to expect parties to take every conceivable step to preserve all potentially relevant data.”)
  •  Kleen Products, LLC, et al. v. Packaging Corp. of Amer., et al.Case: 1:10-cv-05711, Document #412 (ND, Ill., Sept. 28, 2012).

Also see: The Top Twenty-Two Most Interesting e-Discovery Opinions of 2016 (, 1/2/17) (the following top ranked cases concerned proportionality: 20, 18, 17, 15, 14, 11, 6, 4, 3, 2, 1); and, Good, Better, Best: a Tale of Three Proportionality Cases – Part Two ( 4/8/12) (includes collection of earlier case law).

Sedona Commentary

The Sedona Conference Commentary on Proportionality_May 2017 is more than a collection of case law. It includes commentary hashed out between competing camps over many years. The latest 2017 version includes Six Principles that are worthy of study. They can certainly help you in your own analysis of proportionality. The cited case law in the Commentary is structured around these six principles.


Principle 1: The burdens and costs of preserving relevant electronically stored information should be weighed against the potential value and uniqueness of the information when determining the appropriate scope of preservation.

Principle 2: Discovery should focus on the needs of the case and generally be obtained from the most convenient, least burdensome, and least expensive sources.

Principle 3: Undue burden, expense, or delay resulting from a party’s action or inaction should be weighed against that party.

Principle 4: The application of proportionality should be based on information rather than speculation.

Principle 5: Nonmonetary factors should be considered in the proportionality analysis.

Principle 6: Technologies to reduce cost and burden should be considered in the proportionality analysis.


Proportionality is one of those deep subjects where you should think for yourself, but also be open and listen to others. It is possible to do both, although not easy. It is one of those human tricks that will make us hard to replace by smart machines. The game I have created will help you with that. Try out the Small, Medium or Large? proportionality game by filling out the online polls I created.

But, you can do more. You can lead discussions at your law firm, company, class or CLE on the subject. You can become an e-discovery proportionality Game-Master. You can find out the consensus opinion of any group. You can observe and create statistics of how the initial opinions change when the other game players hear each others opinions. That kind of group interaction can create the so-called hive-effect. People often change their mind until a consensus emerges.

What is the small, medium or large proportionality consensus of your group? Even if you just determine majority opinion, and do not go through an interactive exercise, you are learning something of interest. Plus, and here is the key thing, you are giving game players a chance to exercise their analytical skills.

To help you to play this game on your own, and lead groups to play it, I created a Word Document that you are welcome to use.




Six Sets of Draft Principles Are Now Listed at

October 8, 2017

Arguably the most important information resource of is the page with the collection of Draft Principles underway by other AI Ethics groups around the world. We added a new one that came to our attention this week from an ABA article, A ‘principled’ artificial intelligence could improve justice (ABA Legal Rebels, October 3, 2017). They listed six proposed principles from the talented Nicolas Economou, the CEO of electronic discovery search company, H5.

Although Nicolas Economou is an e-discovery search pioneer and past Sedona participant, I do not know him. I was, of course, familiar with H5’s work as one of the early TREC Legal Track pioneers, but I had no idea Economou was also involved with AI ethics. Interestingly, I recently learned that another legal search expert, Maura Grossman, whom I do know quite well, is also interested in AI ethics. She is even teaching a course on AI ethics at Waterloo. All three of us seem to have independently heard the Siren’s song.

With the addition of Economou’s draft Principles we now have six different sets of AI Ethics principles listed. Economou’s new list is added at the end of the page and reproduced below. It presents a decidedly e-discovery view with which all readers here are familiar.

Nicolas Economou, like many of us, is an alumni of The Sedona Conference. His sixth principle is based on what he calls thoughtful, inclusive dialogue with civil society. Sedona was the first legal group to try to incorporate the principles of dialogue into continuing legal education programs. That is what first attracted me to The Sedona Conference. intends to incorporate dialogue principles in conferences that it will sponsor in the future. This is explained in the Mission Statement page of

The mission of is threefold:

  1. Foster dialogue between the conflicting camps in the current AI ethics debate.
  2. Help articulate basic regulatory principles for government and industry groups.
  3. Inspire and educate everyone on the importance of artificial intelligence.

First Mission: Foster Dialogue Between Opposing Camps

The first, threshold mission of is to go beyond argumentative debates, formal and informal, and move to dialogue between the competing camps. See eg. Bohm Dialogue, Martin Buber and The Sedona Conference. Then, once this conflict is resolved, we will all be in a much better position to attain the other two goals. We need experienced mediators, dialogue specialists and judges to help us with that first goal. Although we already have many lined up, we could always use more.

We hope to use skills in both dialogue and mediation to transcend the polarized bickering that now tends to dominate AI ethics discussions. See eg. AI Ethics Debate. We need to move from debate to dialogue, and we need to do so fast.


Here is the new segment we added to the Draft Principles page.

6. Nicolas Economou

The latest attempt at articulating AI Ethics principles comes from Nicolas Economou, the CEO of electronic discovery search company, H5. Nicolas has a lot of experience with legal search using AI, as do several of us at In addition to his work with legal search and H5, Nicholas is involved in several AI ethics groups, including the AI Initiative of the Future Society at Harvard Kennedy School and the Law Committee of the IEEE’s Global Initiative for Ethical Considerations in AI.

Nicolas Economou has obviously been thinking about AI ethics for some time. He provides a solid scientific, legal perspective based on his many years of supporting lawyers and law firms with advanced legal search. Economou has developed six principles as reported in an ABA Legal Rebels article dated October 3, 2017, A ‘principled’ artificial intelligence could improve justice. (Some of the explanations have been edited out as indicated below. Readers are encouraged to consult the full article.) As you can see the explanations given here were written for consumption by lawyers and pertain to e-discovery. They show the application of the principles in legal search. See eg The principles have obvious applications in all aspects of society, not just the Law and predictive coding, so their value goes beyond the legal applications here mentioned.

Principle 1: AI should advance the well-being of humanity, its societies, and its natural environment. The pursuit of well-being may seem a self-evident aspiration, but it is a foundational principle of particular importance given the growing prevalence, power and risks of misuse of AI and hybrid intelligence systems. In rendering the central fact-finding mission of the legal process more effective and efficient, expertly designed and executed hybrid intelligence processes can reduce errors in the determination of guilt or innocence, accelerate the resolution of disputes, and provide access to justice to parties who would otherwise lack the financial wherewithal.

Principle 2: AI should be transparent. Transparency is the ability to trace cause and effect in the decision-making pathways of algorithms and, in hybrid intelligence systems, of their operators. In discovery, for example, this may extend to the choices made in the selection of data used to train predictive coding software, of the choice of experts retained to design and execute the automated review process, or of the quality-assurance protocols utilized to affirm accuracy. …

Principle 3: Manufacturers and operators of AI should be accountable. Accountability means the ability to assign responsibility for the effects caused by AI or its operators. Courts have the ability to take corrective action or to sanction parties that deliberately use AI in a way that defeats, or places at risk, the fact-finding mission it is supposed to serve.

Principle 4: AI’s effectiveness should be measurable in the real-world applications for which it is intended. Measurability means the ability for both expert users and the ordinary citizen to gauge concretely whether AI or hybrid intelligence systems are meeting their objectives. …

Principle 5: Operators of AI systems should have appropriate competencies. None of us will get hurt if Netflix’s algorithm recommends the wrong dramedy on a Saturday evening. But when our health, our rights, our lives or our liberty depend on hybrid intelligence, such systems should be designed, executed and measured by professionals with the requisite expertise. …

Principle 6: The norms of delegation of decisions to AI systems should be codified through thoughtful, inclusive dialogue with civil society. …  The societal dialogue relating to the use of AI in electronic discovery would benefit from being even more inclusive, with more forums seeking the active participation of political scientists, sociologists, philosophers and representative groups of ordinary citizens. Even so, the realm of electronic discovery sets a hopeful example of how an inclusive dialogue can lead to broad consensus in ensuring the beneficial use of AI systems in a vital societal function.

Nicolas Economou believes, as we do, that an interdisciplinary approach, which has been employed successfully in e-discovery, is also the way to go for AI ethics. Note his use of the word “dialogue” and mention in the article of The Sedona Conference, which pioneered the use of this technique in legal education. We also believe in the power of dialogue and have seen it in action in multiple fields. See eg. the work of physicist, David Bohm and philosopher, Martin Buber. That is one reason that we propose the use of dialogue in future conferences on AI ethics. See the Mission Statement.





%d bloggers like this: