New systems of e-discovery are emerging that are designed for today’s digital world. Unlike most existing e-discovery systems, they are not mere adaptations of old paper discovery ways. The new methods use an entirely new collaborative approach and technologies, exemplified by predictive coding software. Although this paradigm shift in discovery is just starting, many of the contours of the new methods are already apparent.
In general terms, we are coming to realize that our current linear, confrontative, one-dimensional, largely manual, Bates stamp approach to discovery must be replaced with a multidimensional, cooperative, iterative, largely automated, hash value approach.
Computers and advanced technologies and communication systems created this problem for the law. They can now provide the solution. As the influential Bar group The Sedona Conference puts it:
A consensus is forming in the legal community that human review of documents in discovery is expensive, time consuming, and error-prone. There is growing consensus that the application of linguistic and mathematic-based content analysis, search and retrieval technologies, and tools, techniques and process in support of the review function can effectively reduce the cost, time, and error rates.
The Sedona Conference Best Practices Commentary on the Use of Search & Information Retrieval Methods in E-Discovery, 8 Sedona Conf. J. 189, 215 (2007).
Attorneys are beginning to embrace new technologies as indispensable tools around which new legal processes and procedures are built. They understand that expensive human review must be significantly curtailed and re-directed to the use of new software tools, not just keyword searching. The new tools include context and conceptual search software, advanced multidimensional indexing, and other new types of artificial intelligence based predictive coding search and review programs. See Eg.: Kevin D. Ashley, Will Bridewell, Emerging AI+Law Approaches to Automating Analysis and Retrieval of ESI in Discovery Proceedings, ICAIL 2009 Global E-Discovery/E-Disclosure Workshop.
Technology should not be mistaken as an end in itself, or a magic panacea. New software and other technology that is just plugged into the old paper search and review methods and processes, and old attitudes, is not the answer. A more fundamental change in legal practice is required. Quoting again from another important Sedona Conference commentary:
The legal profession is at a crossroads: the choice is between continuing to conduct discovery as it has “always been practiced” in a paper world — before the advent of computers, the Internet, and the exponential growth of electronically stored information (ESI) — or, alternatively, embracing new ways of thinking in today’s digital world. Cost-conscious clients and over-burdened judges are demanding that parties now undertake new approaches to solving litigation problems.
Baron and Burke, The Sedona Conference® Commentary on Achieving Quality in the E-Discovery Process; (May 2009) in the Executive Summary.
Many of our leading jurists, information scientists, academics, scholars, writers, and legal practitioners recognize that the old methods and attitudes that worked for paper no longer work for ESI. Victor Stanley, Inc. v. Creative Pipe, Inc., 250 F.R.D. 251 (D. Md. 2008) (Judge Grimm); Securities and Exchange Commission v. Collins & Aikman Corp., 2009 WL 94311 (S.D.N.Y., 2009) (Judge Scheindlin); Disability Rights Council of Greater Wash. v. Wash. Metro. Area Transit Auth., 2007 WL 1585452 (D.D.C. 2007) (Judge Facciola); United States v. O’Keefe, 2008 WL 449729 (D.D.C. 2008) (Judge Facciola); William A. Gross Const. Associates, Inc. v. American Mfrs. Mut. Ins. Co., _F.R.D._, 2009 WL 724954 (S.D.N.Y. 2009) (Judge Peck); Digicel (St. Lucia) Ltd & Ors v. Cable & Wireless & Ors, [2008] EWHC 2522 (Ch) (Justice Morgan) (UK decision). Moreover, scientific research has shown that keyword search alone is ineffective and multi-modal approaches that use keyword and other methods work far better. See: Child’s Game of “Go Fish” is a Poor Model for e-Discovery Search; The Multi-Modal “Where’s Waldo?” Approach to Search and My Mock Debate with Jason Baron; Secrets of Search: Parts One, Two, and Three.
The new legal methods for search and review take advantage of the computational properties of ESI. These new methods are not only tolerated by judges, they are encouraged, because judges understand that they allow a way out of the unnecessary disputes and expenses that the old methods perpetuate. The new methods and technologies are also encouraged by a recent Congressional action amending the Federal Rules of Evidence to include new Rule 502 on the protection of privileged information from inadvertent disclosure, a significant problem now because of the high volumes of ESI to review. The rule sometimes requires a producing party to take reasonable steps to protect privilege. The Advisory Committee Notes To Federal Rule Of Evidence 502 state that:
… a party that uses advanced analytical software applications and linguistic tools in screening for privilege and work product may be found to have taken ‘reasonable steps’ to prevent inadvertent disclosure.
Also see: The Sedona Principles (2007) Principle 11:
A responding party may satisfy its good faith obligation to preserve and produce relevant electronically stored information by using electronic tools and processes, such as data sampling, searching, or the use of selection criteria, to identify data reasonably likely to contain relevant information.
Software has advanced to the point that computer review alone is at least as accurate as manual review, the so-called gold standard, and is getting better all of the time. Maura R. Grossman & Gordon v. Cormack, Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review, Rich. J.L. & Tech., Spring 2011, at 48; Herbert L. Roitblat, Anne Kershaw, Patrick Oot, Document Categorization in Legal Electronic Discovery: Computer Classification vs. Manual Review (Electronic Discovery Institute 2009); Also see my prior blogs: Secrets of Search: Parts One, Two, and Three; and, The Legal Implications of What Science Says About Recall.
Furthermore, highly focused, multiple-phase production is becoming an accepted best practice norm, replacing the old paper ways of trying to discover all possible relevant writings in a single, broad, boilerplate production request and production. Covad Communications Co. v. Revonet Inc., 2008 WL 5377698 (D.D.C. 2008) (boilerplate paper forms still used to request ESI); Shannon Capone Kirk, Kristin Ali, The Need for Art and Surgery in Discovery, Chapter 16 of Losey, Electronic Discovery (West 2010).

You can't afford the truth!
The profession is beginning to understand that in today’s world of Too Much Information, the production of all relevant information is a practical impossibility. See eg. Rowe Entm’t, Inc. v. William Morris Agency, Inc., 205 F.R.D. 421, 423 (S.D.N.Y. 2002):
The more information there is to discover, the more expensive it is to discover all the relevant information until, in the end, “discovery is not just about uncovering the truth, but also about how much of the truth the parties can afford to disinter.”
Instead, the goal should be production of as many highly relevant documents as is proportionate to the value and significance of the case. Rule 26(b)(2)(c) Federal Rules of Civil Procedure; Sedona Conference Commentary on Proportionality (2010); The Sedona Principles (2007), 2nd Principle:
When balancing the cost, burden, and need for electronically stored information, courts and parties should apply the proportionality standard embodied in Fed. R. Civ. P. 26(b)(2)(C) and its state equivalents, which require consideration of the technological feasibility and realistic costs of preserving, retrieving, reviewing, and producing electronically stored information, as well as the nature of the litigation and the amount in controversy.
Also see Final Report on the Joint Project of the American College of Trial Lawyers and the Institute for the Advancement of the American Legal System (2009) (At page 7: “Proportionality should be the most important principle applied to all discovery.”) Also see my recent blog: Bottom Line Driven Proportional Review.
The new discovery methods rely heavily upon various types of sampling and other metric based quality control systems. Rule 34(a)(1), Federal Rules of Civil Procedure specifically allows sampling as a form of discovery and the official Rules Commentary confirms that this provision is intended to cover ESI as well as tangible objects. Also see In re Vioxx Products Liability Litigation, No. 06-30378, 06-30379 2006 W.L. 1726675, at *2 n.5 (5th Cir. May 26, 2006) (“By random sampling, we mean adhering to a statistically sound protocol for sampling documents.”); In Re Seroquel Products Liability Litigation, 244 F.R.D. 650 (M.D. Fla. 2007) (“Common sense dictates that sampling and other quality assurance techniques must be employed to meet requirements of completeness.”); Mt. Hawley Ins. Co. v. Felman Production, Inc., 2010 WL 1990555 (S.D. W. Va. May 18, 2010) (plaintiff’s attorney client privilege was lost because the plaintiff “failed to perform critical quality control sampling to determine whether their production was appropriate and neither over-inclusive nor under-inclusive.”); McPeek v. Ashcroft, 202 F.R.D. 31, 34 (D.D.C. 2001) (ordering initial limited search of backup tapes within dates determined to be most relevant, as a “rational starting point” for ESI search). Also consider the opinion of Kipperman v. Onex Corp., 260 F.R.D. 682 (N.D. Ga. 2009). The court required production of “sample” backup tapes to weigh the volume and importance of the information on the tapes against the costs of their restoration and production before deciding whether to permit additional discovery. After reviewing the results of the sample, the court held that the ESI contained on the tapes was highly relevant and so compelled further discovery saying: “I don’t . . . declare these to be smoking guns but they certainly are hot and they certainly do smell like they have been discharged lately.” Id. at 691.
New Cooperative Strategies to Litigation Discovery
We know that the new methods should be based upon open communications and sharing of information of preliminary findings with the requesting party. This is what George Paul and Jason Baron call “virtuous cycle iterative feedback loops.” Information Inflation: Can the Legal System Adapt? Supra at pgs. 32-36.
The parties need to exchange expert views and gain a common understanding of the relevance parameters. Scientific investigations on the effectiveness of search and review processes performed by the TREC Legal Track confirm this. Douglas W. Oard, Bruce Hedin, Stephen Tomlinson and Jason R. Baron, Overview of the TREC 2008 Legal Track. Studies also confirm that at least one meeting between opposing counsel to share initial review results, and consider alterations based on mutual disclosure and input, significantly increases the overall precision and recall of the project. Jason R. Baron, Douglas W. Oard, Feng C. Zhao, Improving Search Effectiveness in the Legal E-Discovery Process Using Relevance Feedback.
For this new system of open, iterative communication processes to work, lawyers must change their attitudes. The total adversarial approach to discovery, which is now prevalent in the United States, must be replaced by strategic cooperation with active judicial management where required. This is critical for the new iterative open process to work at maximum efficiency.
Cooperative discovery processes have long been required by the rules of professional ethics and procedure, but they were frequently misunderstood or ignored by attorneys succumbing to economic pressures to win at all costs. Losey, Lawyers Behaving Badly, 60 Mercer L. Rev. 983 (Spring 2009). They were also not well understood or enforced by the courts. Mancia v. Mayflower Textile Services. Co., 253 F.R.D. 354 (D.Md. Oct. 15, 2008). But this has changed with recent scholarship and experiences showing the importance of cooperation in e-discovery. The Case for Cooperation, 10 Sedona Conf. J. 339 (2009 Supp.) (Preface by Justice Breyer; lead article by Sedona Conference Working Group, edited by Bill Butterfield, Richard Braman, Ken Withers and others); The Bull’s-Eye View of Cooperation in Discovery, 10 Sedona Conf. J. 363 (2009 Supp.) (Professor Steven S. Gensler); Mancia v. Mayflower Begins a Pilgrimage to the New World of Cooperation, 10 Sedona Conf. J. 377 (2009 Supp.) (Ralph Losey).
Experience has shown that an over-contentious approach to discovery is a significant cause of many of the inflated costs of e-discovery. A strong movement is now underway in the U.S. to adopt a cooperative approach to discovery. It is led by The Sedona Conference and over 100 prominent judges and many more legal practitioners. See: Sedona Cooperation Proclamation, Sedona Conf. (2008). As Jason R. Baron, the past Co-Chair of The Sedona Conference® Working Group on Electronic Document Retention and Production, recently explained:
[T]he challenge is how best to reasonably (not perfectly) manage the exponentially growing amount of ESI caught in, and subject to, modern-day discovery practice. The answer lies principally in culture change (i.e., fostering cooperation strategies), combined with savvier exploitation of a range of sophisticated software and analytical techniques.”
Baron, J. Law In The Age of Exabytes, XVII RICH. J.L. & TECH. 9, at pg. 5 (2011).
Conclusion
Discovery of evidence and the legal analysis of relevancy and privilege determinations are at the heart of our legal system. They are essential to the common law evidence based system of justice. The methods and tools used in paper discovery did not work with the vast stores of digital information ubiquitous in the Twenty-First Century. That is why e-discovery became so expensive and riddled with mistakes. That is why completely new methods and tools emerged for digital discovery, which, in 2012, finally caught on. The last impediment of no judicial approval has been destroyed. The way is now clear.
The old linear methods of reading all papers are being replaced by iterative methods that include predictive coding. The new ways involve cooperative, multidimensional, cyclic approaches that focus on proportional, phased productions.
The stated goal of the new legal processes has also changed. The goal is now legally adequate recall, not complete recall. It is discovery of all highly relevant information, but not necessarily all relevant information. Adequacy is determined on a case-by-case basis as necessary to render justice and, at the same time, not unfairly burden the parties to litigation. It is based on reasonability and proportionality, not a delusional notion of perfection that, in fact, has never been required by the Law.
The leaders in e-discovery have already made this change and are embracing and refining these new methods. They are managers of complex technologies and iterative quality control processes. They lead an e-discovery team. This trend will continue because it works. It saves money and mitigates risks.
The old-school, linear, confrontative, one-dimensional, largely manual, Bates stamp approach to e-discovery has already been abandoned by all of the top experts in the field. The old ways have been replaced by a new nine-step process I described in Bottom Line Driven Proportional Review, or something close to that. The new methods are multidimensional, cooperative, iterative, and include predictive coding as part of a multimodal approach. The predictive coding software works, and, if it is properly used with the new legal methods, will be accepted by courts. See eg. Chris Dale’s Judge Peck’s Predictive Coding Opinion – reporting the reaction and the case and articles cited therein. The time to employ these new methods is now. You have nothing to lose but high expense and low recall.
As the famous Indian lawyer turned saint once said:
Be the change that you wish to see in the world.

Posted by Ralph Losey
For millennium writings were on paper. For centuries the legal profession depended upon writings, referred to in the law as documents, as the key evidence to resolve disputes in a fair and just manner. Losey, Mathematical Formula for Justice Proves the Importance of ESI in Civil Litigation, Chapter 4 of Electronic Discovery (West 2010). Paper documents were well-known and mastered by every lawyer and judge who swore an oath to uphold the law.
Many see this as a much more profound cultural revolution than that precipitated by Guttenberg, which took centuries to play out, not decades. George L. Paul and Jason R. Baron,
The legal profession has been severely stressed by the rapid, ever-accelerating advances in technology. The changes in writing and resulting information explosion have been the key stressors. ESI is not only changing and evolving new into forms every year, but is now multiplying at an exponential rate that is almost beyond comprehension. See Eg., Rowe Entm’t, Inc. v. William Morris Agency, Inc., 205 F.R.D. 421, 429 (S.D.N.Y. 2002) (explaining that electronic data is so voluminous because, unlike paper documents, “the costs of storage are virtually nil. Information is retained not because it is expected to be used, but because there is no compelling reason to discard it”), aff’d, 2002 WL 975713 (S.D.N.Y. May 9, 2002); Data, Data Everywhere (The Economist, March 2010); Baron and Losey 
These old paper-based legal search and review methods are one dimensional and linear in nature. They typically follow a sequential Bates stamp organizational model created in the 1890s. The simple paper evidence discovery processes worked pretty well for decades before computers. It should be noted, however, that even before technology moved away from paper typing machines to computers in the 1980s, the discovery processes were already severely taxed by the growing volumes of paper documents generated from the 1960s forward. The increase in paper volume was caused by another technological innovation, the photocopy machine and by ever more complex transactions. Still, the legal profession coped somehow for the rest of the Twentieth Century. Lawyers added more numbers to the Bate stamps and used larger teams of lawyers and paralegals to manage the additional papers. They were still on familiar ground.
The old linear review methods involved serial culling of documents down to a final production set. The process generally required multiple reviews of the same document for different purposes. It was inefficient. It was expensive. Moreover, the quality control of human eyes on paper did not work with high volumes of documents. This is shown by the latest scientific experiments where the agreement rate among professional legal reviewers was found to be just less than 50%. Cormack, Grossman, Hedin, Oard, Overview of TREC 2010 Legal Track (February 21, 2012).
I want to make predictive coding software an affordable, everyday item. Now that I’ve helped to open the door, I want as many people as possible to be able to walk through. With the right methods to use this tool, a new world of affordable e-discovery awaits.
What ethical considerations and rules or professional conduct come into play in this scenario? Let us analyze the facts of the hypothetical one rule at a time and consider the impact of all six of the key rules: Rule 1.1 – Competence; Rule 1.3 – Diligence; Rule 1.6 – Confidentiality; Rule 3.2 – Expediting Litigation; Rule 3.3 – Candor Toward the Tribunal; and, Rule 3.4 – Fairness to Opposing Party and Counsel.
They did not cooperate in e-discovery because they did not know how. They probably had never even seen the cooperative model in action before, and so when they saw it here, they did not recognize it. They may instead have mistaken it for weak or timid opposing counsel. It just provoked them to be more caveman like. They got more aggressive in the face of the unknown. They also ran away from any real discussion on technical issues. A classic case of the fight or flight instincts of a pre-computer-literate-lawyer faced with e-discovery.
Confidentiality. Plaintiffs’ counsel here thought they were being very ethical by refusing to disclose their work product. They would not give the defendant an idea on their thinking of the case, on what information they thought would be highly relevant. They would not disclose why they thought some custodians and issues were more important. That was their confidential thinking, and they thought they should keep it secret. They hoped to keep their analysis of the case (assume they had one, and this was not just a superficial form-driven lawsuit) to themselves. They wanted to surprise defendants as much as possible. Indeed, they were initially surprised by how much confidential information the defense counsel here provided, which, again, they mistakenly mistook as a sign of weakness and egged them on to keep demanding more and more. Then they were surprised again when defense counsel said no, and never budged from the initial 5/7. And finally were surprised again by the court ruling against them and went with the defense plan.
In the same professorial discussion group my esteemed LegalTech debating adversary, Professor Craig Ball, had this to say concerning the policy behind limited work product disclosures, starting with a comment on pre-digital paper productions:
Candor Toward the Tribunal. I think the conduct of plaintiffs’ counsel in this hypothetical violated this fundamental Rule of Professional Conduct too, but admit their violation might be seen as technical, and is certainly the kind of conduct that goes on every day and is tolerated by both Bench and Bar. Plaintiffs’ counsel here did not “make a false statement of fact or law to a tribunal … or, offer evidence that the lawyer knows to be false.” But they did not provide the whole truth either. They made a demand in their motion to compel for 30 custodians and 30 issues, and they continued in this demand at the hearing before the tribunal. Let us assume that they also made oral representations to the judge at a hearing that they thought this 30/30 demand was necessary and appropriate under the facts of the case and governing law. But that is at best a half-truth because they have been willing all along to accept 20/20.
Fairness to Opposing Party and Counsel. Some attorneys are surprised when they see the terms of this important Rule of Professional Conduct. Candor to the tribunal is one thing, but fairness to the opposing party and their attorneys? That is simply not part of the culture of many lawyers and law firms! They seem surprised when they are reminded that the requirement is built into our rules of ethics. It is not a mere professional courtesy, as some think, it is an ethical imperative. Under Rule 3.4 a lawyer shall not “unlawfully obstruct another party’s access to evidence.” A lawyer shall not “conceal a document or other material that the lawyer knows or reasonably should know is relevant to a pending or a reasonably foreseeable proceeding; nor counsel or assist another person to do any such act.” A lawyer shall not in “make a frivolous discovery request or fail to make reasonably diligent effort to comply with a legally proper discovery request by an opposing party.”
Some may say my hypothetical is far-fetched, that attorneys do not engage in this type of behavior. I say, get real. It is an everyday occurrence. The only thing far-fetched about it is the simplicity of the facts, which I necessary injected into the hypothetical, and, the relatively mild nature of the violations. In my position as national e-discovery counsel for a 700 attorney law firm with 48 offices around the country, I see equivalent or worse behavior by opposing counsel almost every week. It is not exactly a crazy hell-zone as I used to suspect, and many still believe. Krueger v. Pelican Products Corp., C/A No. 87-2385-A (W.D. Okla. 1989) (J. Alley) (“If there is a hell to which disputatious, uncivil, vituperative lawyers go, let it be one in which the damned are eternally locked in discovery disputes with other lawyers of equally repugnant attributes.”) But it is bad, and we need to work together as a profession to break out of this prison.
I have been interested in the ethical issues surrounding electronic discovery since 2006. At that time I phased out my general trial practice, went full-time e-discovery, joined The Sedona Conference®, and started this e-Discovery Team® blog. As part of my practice I read most of the opinions around the country written on e-discovery. I quickly noticed something I had not seen before in any other field of law. The case law is dominated by sanctions cases involving spoliation of evidence. Not only that, attorneys are often directly implicated in this spoliation and accused of many other types of intentional or negligent misconduct. I began to wonder if I had stepped into a crazy zone of the law where all attorneys acted like sharks.





Next assume that plaintiffs’ counsel are unpersuaded by defense counsel, that they respond with little or no substance, and instead demand 30/30, instead of 50/50, arguing that they have now made major concessions, and thus suggesting or signaling that they will accept 20 custodians and 20 issues. They assume, incorrectly, that defense counsel is like them, that the 5/7 proposal was just the opening offer in a negotiation dance. They did not really care about the reasons stated by defendants for the proposal, and, truth be told, they did not really understand most of the e-discovery technical talk surrounding the issues. They were hardball trial lawyers; tough advocates doing their job by pounding out as much information from the other side as they could. They thought it would help their clients to make the defendant’s case as expensive as possible. They knew e-discovery was a good way to do that, and knew from past experience that this oppose everything tactic was a good way to drive up the settlement value of the case. Discovery, especially e-discovery, was just another tool in the battle against the other side.
Of course, in the real world all lawyers come before judges with a history. They have a reputation. This can also help a judge to evaluate “who’s on first” and know who is a poser, and who is not. This is especially true where a judge has seen and heard from one or more of the lawyers several times before. That is where the intangible value of an attorney’s reputation comes in.