e-Discovery Team ®

Case Central is celebrating its Third Anniversary of creating e-discovery cartoons that they call Case In Point. They have guesstimated what they think are their ten most popular cartoons. They gave me permission at ACCA in Denver this week (they were in the booth next door to my law firm’s booth) to reprint the top ten cartoons. I agreed that you, dear readers, would take on the heavy responsibility of determining, once and for all, their best all-time cartoon (so far). There is a voting poll at the end for you to make the final call.

If you know of any other e-discovery cartoons that you think are funny, let me know. God knows we could all use some more humor in this stress-filled field of ours. Do you know where all your ESI is tonight? Personally I have no idea, and don’t even get me started about my clients’ ESI.

___________________

________________

The last cartoon is number ten, and the first cartoon at the top is, they think, number one in popularity. But they don’t really know because Case In Point does not have cool voting polls like e-Discovery Team. So this is your chance to set the record straight and determine their number one cartoon. If you are like me, you probably didn’t break out loud laughing at any of these. They are more like smile and think, clever type cartoons. So which one did you like the most? It is a hard choice. But as an old client of mine, Dr. Schuler, a computer pioneer and emergency room physician, used to say, relax, no one is going to die today if you make the wrong decision. Kind of puts our job in perspective, doesn’t it?

It is now sweeter than ever to be a victor in federal court. That’s because of the hot new trend to award winners their e-discovery costs under 28 U.S.C. §1920(4) and Rule 54(d)(1) FRCP. Courts are now beginning to consider the services of an e-discovery vendor as “the 21st Century equivalent of making copies.” CBT Flint Partners, LLC v. Return Path, Inc., 676 F.Supp.2d 1376, 1381 (N.D. Ga. 2009).

We are now talking about dozens of cases around the country where significant cost awards have been made to the prevailing party. This started off being a fairly evenly divided issue a few years ago and there were only a few cases. Now the tide seems to be turning and a flood of cases are coming out. More and more judges are siding with the winners and, for the first time, construing §1920 to include six-figure e-discovery costs. See Eg In re Aspartame Antitrust Litigation, No. 2:06-CV-1732, 2011 (E.D. Penn. Oct. 5, 2011) ($500,000 e-discovery costs award to defendants); Race Tires America v. Hoosier Racing Tire Corp., 2011 WL 1748620 (W.D. Pa. May 6, 2011) (now on appeal to 3^rd Cir) ($400,000 award to defendants for e-discovery costs); Tibble v. Edison International, 2011 Lexis 94995 (C.D. Cal., July 8, 2011) ($370,000 cost award to defendants who won nine out of ten counts and used the costs award to more than offset the plaintiffs’ judgment on one count).

Like it or not, this is America where everybody loves a winner and so this trend will probably continue and grow stronger. As Vince Lombardi said:

If winning isn’t everything, why do they keep score?

Vinny also said:

I firmly believe that any man’s finest hour, the greatest fulfillment of all that he holds dear, is that moment when he has worked his heart out in a good cause and lies exhausted on the field of battle – victorious.

You probably won’t have Gatorade poured over your head if you win in federal court, but now you might get showered with cash.

New Strategies

There is still wide-spread disagreement on this issue in the courts and many judges are still going the other way to disallow most e-discovery costs, but there seems to be a trend to interpret §1920(4) broadly, especially when hidden equitable factors are at play. But see Specht v. Google, Inc., 2011 U.S. Dist. Lexis 68968 (N.D. Ill. June 27, 2011) (cost application denied because the parties did not have an express agreement to produce discovery in electronic form). If you are a winner, you might as well go for it. You might as well try to have the losers reimburse you for all of your e-discovery vendor fees. As Vince Lombardi said:

We would accomplish many more things if we did not think of them as impossible.

At this early stage of the game it is hard to predict how a judge will rule, especially if they have never considered this issue before, but you have nothing to lose in trying. For that reason this will probably now come up in every case where there is a winner with any e-discovery costs. Winners will file a Bill of Costs with the Clerk and hope for the best. The Court Clerk and supervising District Court Judge, who makes the final decisions (it is de novo review), could well award all or part of the winner’s e-discovery costs. Or you could just receive a small award for your paper copy costs. At this early stage it is hard to predict how judges may rule, but as Lombardi said:

If you can accept losing, you can’t win.

The possibility of large e-discovery costs awards will also change the matrix in settlement discussions. Both sides of a dispute will have to take into consideration the other side’s e-discovery costs, because if they lose, they may have to pay the winner’s costs. This is one more thing to talk about and consider in every mediation. What are the parties respective exposures if they decide to roll the dice? Could the losing plaintiffs get taxed with hundreds of thousands of dollars of defendants’ e-discovery costs? Do the plaintiffs have any assets to satisfy such a cost judgment? Does a losing defendant have the ability to pay? Would a prevailing defendant after judgment be willing to waive their right to seek costs in exchange for a losing plaintiff giving up their right to appeal? You know these conversations are going on right now.

It will also enter the matrix in motions for protective orders. The moving side will use the new costs rule to try to persuade the court to order the producer to go to the extra expense. This is a tempting argument to judges. When in doubt they may order the producing party to do more, to spend more, with the consolation that if they win, they can always get it back in a cost award. Please judges, don’t fall for that. Stay with proportionality, with Rule One. Most cases settle, and that means there is no prevailing party to be entitled to an award under §1920. Further, in many cases one side is collection-proof. That cost judgment will never lead to cash. Further, inability to pay is grounds for a loser to defeat a cost motion. The possible cost award repayment is always going to be remote and speculative, whereas the vendors fees are immediate and certain (and high).

Vendors Rejoice

This is a real bonanza for e-discovery vendors, and something of a downer for lawyers and law firms with special technical skills and built-in vendor services. Perhaps law firms can take consultation in Lombardi’s words:

It’s not whether you get knocked down, it’s whether you get up.

It will be interesting to see how firms respond to this new trend. Could more spin-off companies be in the works? That would also solve the many ethical issues of running an e-discovery business in a law firm.

The award of costs under §1920 cannot be construed to include any of the winner’s attorneys fees. The statute is only for an award of costs, even though the statute says fees as we will see. It means outside vendor fees, not inside law firm fees. It remains to be seen whether it will include non-fee cost billings by law firm litigation support departments, as opposed to billings by outside vendors for the same services. I do not think this specific issue has come up yet, but if you hear anything about it, please let me know.

Either way, the wording on bills to describe e-discovery charges by law firms and vendors is bound to change to try to track language used in the flurry of cases now coming out interpreting §1920. You are likely to need a technical dictionary to understand these fees in the future. Vendor’s will want to make their services look as technical, expert, and non-legal as possible. See Eg. Race Tires America v. Hoosier Racing Tire Corp., supra at *29:

A careful review of the vendor’s invoices reveals that the services provided were not the type of services that attorneys or paralegals are trained for or are capable of providing. The services were highly technical.

Craig Ball addressed this topic in February 2010, and, as usual, concluded with this good advice:

Lawyers seeking to tax e-discovery expenses as court costs should segregate and document disbursements that most closely correspond to § 1920 categories of taxable costs. It‟s useful to establish that the producing party incurred the expenses in response to a request by or agreement with an opponent or an instruction from the court (e.g., “produce as TIFF images with load files”) and that the cost was necessarily incurred to fulfill that request. It’s key to show that the benefits reached the requesting party or the court. Be specific, and closely track the statutory language, likening to printing, copying, exemplifying or even translating as appropriate. Studiously exclude costs principally benefiting the producing party or merely for counsel’s convenience, and consider whether using a court-appointed neutral for e-discovery enhances your ability to tax such expenses as costs.

Ball, Craig, Musings on Electronic Discovery, at pgs. 157-159, Are We Just ―Makin’ Copies? (Originally published in Law Technology News, February 2010).

Right now this is still a wide open book yet to be written. Creativity in description of the bills may be important. As Craig points out, it is important for counsel to be able to show a court that the expenses were necessary and were not incurred just for the convenience of counsel. This can be a hard line to draw, but the winners seem to have the better argument. After all, the winning litigants paid the bills, and they did so with no guaranty they would win or ever receive a costs award. The same argument applies as to any challenges to the amount of the costs as unreasonable. As Judge Terrence F. McVerry held in Race Tires:

Although the amount of costs assessed in this action is significant, the assessment is reflective of the amounts incurred by the prevailing parties without respect to their anticipation of being prevailing parties and, in that regard, can hardly be considered as “puffed,” exorbitant, or contrived. Such costs were, in fact, incurred – and in significant respects at the discovery demands of Plaintiffs. Although a reasonable defense to the imposition and request of the prevailing party for costs can be the actual inability to pay said costs, such defense has not been raised by the non-prevailing party here.

Race Tires America v. Hoosier Racing Tire Corp., supra at *35-36.

Besides, on the issue of convenience I know of no client willing to pay more for a service just so their lawyers can be convenienced. Get real. How many clients will pay for first class airfare? Clients will only agree to pay for e-discovery services if they are necessary and designed to save them money and/or risk. The whole convenience argument seems misplaced in this area. But in the paper days courts held that Bates stamping was not allowed as a cost because it was only done for the convenience of counsel. And they now try to make analogous arguments in high volume ESI discovery and productions.

It is instructive to see what was allowed, and what wasn’t, in the big cost awards case of In re Aspartame Antitrust Litigation, supra. The vendor costs award included fees associated with:

1. data recovery,
2. tape restoration,
3. imaging hard drives,
4. storage of data,
5. deduplication,
6. data extraction and processing,
7. OCR’ing paper documents,
8. the creation of a litigation database,
9. keyword searches,
10. privilege screening (i.e., keywords for privileged documents),
11. data hosting,
12. technical support,
13. project management,  and
14. the production costs for the creation of load files allowing documents saved as TIFFs to be loaded into review platforms.

Many of these sound like legal services to me, but apparently they were all done by vendors, including lawyers working for vendors who claim not to provide legal advice.

In Aspartame the cost award excluded fees associated with the vendor’s Attenex Document Mapper software described as “a document review tool with visual clustering of a document collection based on concepts extracted from those documents.” This was rejected, along with “concept based review platform and document analytics” as unnecessary. They were considered to be costs incurred merely for the convenience of counsel. The court in Aspartame seems to have engaged in a kind of Cadillac versus Chevy analysis that missed the cost-savings made possible with the Cadillac features of concept search, clustering, and data analytics. Perhaps with better explanations by counsel these costs would have been awarded too. The key here is to convince the judge that the bells and whistles actually saved the litigants money.

Hidden Equities Driving This Trend

It appears to me as if judges who are now making these large e-discovery costs awards should have applied principles of proportionality to prevent the winner from having to incur these large e-discovery costs in the first place. In many of the cases it is obvious that they did not. For instance, in Race Tires America v. Hoosier Racing Tire Corp, supra, the losing plaintiff was allowed 442 search terms and 119 separate requests for production. Not only that, there were eleven e-discovery related motions. The defendants were forced to run up e-discovery costs bills of over $400,000. I’m guessing they also incurred e-discovery related attorneys fees in the millions. All that and they won on summary judgment. What a waste of time and resources! The plaintiff did not even have enough evidence to create a genuine issue of material fact. Obviously the defendants were forced by the court to pay a disproportionally large amount to defend a very weak case.

This kind of situation sets up the equities for a judge to try to do some sort of rough justice for the winners. Before final judgment the judges had believed the losers who had assured them that they had a meritorious case, and that they could prove it if only they were given more discovery, including especially expensive e-discovery. So the judges gave it to them, and forced the winners to pay e-discovery vendors huge fees to get the job done. Then when it came time to put the evidence down, to show the cards, at summary judgment or trial, the losers were shown to be all bluff and bluster. They lost.

Now the judges could clearly see that they had had the wool pulled over their eyes. The losers did not have a case. They were just trying to get a settlement or they were on a fishing expedition hoping to get lucky and find some evidence to support their case, any case (they could freely amend, often after trial to conform to the evidence). Now that the case is over, the judges know the truth, they know that the losers had a weak case. This is what that the winners had been telling them all along. Under these circumstances, judges are more than happy to do rough justice by making the fool-me-once losers pay for the winners’ e-discovery costs. That is part of the hidden equity going on here to broaden the interpretation of this statute to include all types of e-discovery costs. This is the hidden equity to do rough after-the-fact justice for the winners.

Notice I don’t say plaintiffs or defendants here. Cost awards are party neutral. If the plaintiff wins, and they often do, they can get their e-discovery costs. If the defendant wins, they can get theirs. This is not about plaintiffs and defendants. This is about winners and losers. As Lombardi said:

Some of us will do our jobs well and some will not, but we will be judged by only one thing-the result.

Change in the Law To Expand Beyond Paper Costs

This new trend in allowing six-figure e-discovery cost awards came about because of a little known technical amendment in 2008 to 28 U.S.C. §1920(4). Apparently losers don’t have lobbyists because no one seemed to notice or much care when the statute for court costs was amended to remove the limitation to paper, which has been in their from the start. The obscure little statute has wonderfully archaic language that gives parties and judges a lot of latitude to interpret. It allows for an award of costs for:

Fees for exemplification and the costs of making copies of any materials where the copies are necessarily obtained for use in the case;  (It used to say “making paper copies” instead of copies of “any materials.”)

Parties are now starting to argue successfully that this little change in the statute showed Congressional intent for broad construction of §1920 to include non-paper related costs incurred in 21st Century litigation. Big money is now being made by winner attorneys interpreting exemplification and making copies. Pre-2008 amendment cases that went the other way and disallowed e-discovery costs are now all distinguishable. Again quoting Judge McVerry in Race Tires 

Prior to 2008, district courts struggled with the issue of whether e-discovery costs were recoverable under Section 1920(4). In 2008, Congress changed the relevant language of Section 1920(4) from “[a] judge or clerk of any court of the United States may tax as costs the following: . . . fees for exemplifications and copies of papers” to “fees for exemplification and the costs of making copies of any materials.” Judicial Administration and Technical Amendments Act of 2008, Pub. L. No. 110-496, § 6, 122 Stat. 4291, S. 3596. After the amendment, no court has categorically excluded e-discovery costs from allowable costs.

However, even before the 2008 congressional modification to Section 1920, courts in many jurisdictions had come to [*19] recognize that “exemplification,” in the modern era, includes electronic copying. See, e.g., Cefalu v. Village of Elk Grove, 211 F.3d 416, 427-28 (7th Cir. 2000) (explaining that exemplification “signifies the act of illustration;”: so long as “the means of presentation furthers the illustrative purposes of an exhibit . . . it is potentially compensable”); BDT Prods. v. Lexmark Int’l, Inc., 405 F.3d 415, 420 (6th Cir. 2005) (finding that “electronic scanning and imaging” may constitute “exemplification,” within the meaning of Section 1920); Brown v. McGraw-Hill Co.’s Inc., 526 F. Supp.2d 950, 959 (N.D. Iowa 2007) (finding that “the electronic scanning of documents is the modern day equivalent” of ‘exemplification and copies of paper’ as set forth in Section 1920; El Dorado Irrigation Dist. v. Traylor Bros., Inc., No. CIV. S-03-949, 2007 U.S. Dist. LEXIS 14440, 2007 WL 512428, at *10 (E.D. Cal. 2007) (“[S]canning is akin to copying. The purposes of the two methods are largely the same, to reproduce a document so that it may also be utilized by multiple parties and individuals.”

Race Tires America v. Hoosier Racing Tire Corp., supra *17 at FN6:

So it seems clear that courts after 2008 are free to interpret anew the meaning of the ancient paper-based terms of exemplify and copy. What does this mean in the today’s digital world of ESI collection, processing, analysis and review? What do fees for digital exemplification and copying include? For more and more judges burned by losers who pushed through e-discovery, it means all, or substantially all, of the fees and costs the winners incurred with their e-discovery vendors. More and more judges in this position will likely conclude that “the steps the third-party vendor performed appeared to be the electronic equivalents of exemplification and copying.” Race Tires America, at *21.

Are You Wearing a Technology Consultant Hat or a Lawyer Hat?

Section 1920 clearly does not allow for an award of attorney fees. The Federal Rule of Civil Procedures, Rule 54, that triggers an award under §1920 has not changed. The U.S. does not follow the English Rule allowing for an award of fees to the prevailing party in almost every case. Instead, fee awards are only allowed in the U.S. when authorized by statute or by the parties agreement. Rule 54(d)(1) makes it clear that attorney fees are not part of the costs award. It provides that “unless a federal statute, these rules or a court order provides otherwise, costs–other than attorney’s fees–should be allowed to the prevailing party.” (emphasis added)

I don’t think this is fair to law firms that have invested small fortunes in litigation support departments and providing some of their lawyers special technical skills. This trend to allow e-discovery cost awards for services when rendered by vendors, but not when rendered by law firms, seems like an unfair boost to outside consultants and non-professional corporations. The truth is, the line between fees for legal services and technical services is grey and blurry. It is hard to say what the long-term consequences of this cost-award trend will be, except that it is quite likely to move the grey line away from practicing attorneys and law firms. Still, I am consoled by Lombardi’s words:

The greatest accomplishment is not in never falling, but in rising again after you fall.

Conclusion

If you win a case in federal court, asks for an award of all of your e-discovery costs. It’s that simple and does not cost much to do. Who knows, you might just get all or a substantial part of your costs paid back by the loser. In the meantime, be careful how you word the cost billings. Make sure it is obvious to the judge that you are not just doing e-discovery for the hell of it, that it is necessary and highly technical and damn hard. But don’t get too flowery in your descriptions for fear someone will accuse you of driving a sports car because it is hot and convenient for you to be flashy, not because it will get you there faster and thus save your client a ton of money. And in all settlement discussions, bring this up, let the other side know what your expenses are and that they might have to pay them. Also beware of the bogus argument in discovery that more discovery should be allowed and forget about proportionality because you could get it back at the end of the case anyway.

A big cost award is still a long shot, even if you are a winner. But your team worked hard to get there and you should go for it. Remember the inspirational words of coach Vince Lombardi:

Teamwork is what the Green Bay Packers were all about. They didn’t do it for individual glory. They did it because they loved one another.

Perfection is not attainable, but if we chase perfection we can catch excellence.

Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile.

The quality of a person’s life is in direct proportion to their commitment to excellence, regardless of their chosen field of endeavor.

We didn’t lose the game; we just ran out of time.

This blog will report on a new article by Judge Andrew Peck endorsing the use of artificial intelligence and a recent speech by Jason R. Baron warning of a coming digital dark age of information overload. The two events, much like 7 of 9, make a compelling argument for law to embrace technology, not run from it.

Judge Peck’s Article on Predictive Coding

Judge Peck’s article brings good news to all those stressed by the costs and difficulties of e-discovery. The article shows that artificial intelligence can help alleviate this stress when it is used judiciously. Predictive Coding: Reading the Judicial Tea Leaves, (Law Tech. News, Oct. 17, 2011).

Judge Peck reviews the old days of linear paper review and is glad to see them go. He cites to the study by Herb Roitblatt, Ann Kershaw and Patrick Oot, that suggests computer assisted review is at least as accurate as manual review. Document Categorization in Legal Electronic Discovery: Computer Classification vs. Manual Review, Journal of Am. Society for Information Science & Technology, 61(1):70-80 (2010). Then he cites the study by Maura Grossman and University of Waterloo professor Gordon Cormack, using data from the Text Retrieval Conference Legal Track, which concluded that “[T]he idea that exhaustive manual review is the most effective — and therefore the most defensible — approach to document review is strongly refuted. Technology-assisted review can (and does) yield more accurate results than exhaustive manual review, with much lower effort.” Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review, Richmond J. of Law & Tech., Vol. XVII, Issue 3, 1-48 (2011).

Next, Judge Peck addresses search based on keywords and shows how it is a rather primitive, last-century application of computer assistance, especially when applied without sampling, testing, and transparency.  He uses my Go Fish analogy to illustrate how even this simple technology is not being used correctly by many attorneys.

Because the volume of ESI has made full manual review virtually impossible, lawyers have turned to keywords to cull ESI (particularly e-mail) for further (manual) review. A basic problem is that absent cooperation, the way most lawyers engage in keyword searches is, as Ralph Losey suggests, the equivalent of “Go Fish.” The requesting party guesses which keywords might produce evidence to support its case without having much, if any, knowledge of the responding party’s “cards” (i.e., the terminology used by the responding party’s custodians). Indeed, the responding party’s counsel often does not know what is in its own client’s “cards.”

Judge Peck also reviews the case law in this area and his excellent “wake up call” contribution to this jurisprudence:

Judicial decisions have critiqued keyword searches. Important early decisions in this area came from magistrate judges John Facciola (District of Columbia) and Paul Grimm (Maryland). See United States v. O’Keefe, 37 F. Supp. 2d 14, 24 (D.D.C. 2008) (Facciola, M.J.); Equity Analytics, LLC v. Lundin, 248 F.R.D. 331, 333 (D.D.C. 2008) (Facciola, M.J.); and Victor Stanley, Inc. v. Creative Pipe, Inc ., 250 F.R.D. 251, 260, 262 (D. Md. 2008) (Grimm, M.J.).

I followed their lead with William A. Gross Construction Associates, Inc. v. American Manufacturers Mutual Insurance Co., 256 F.R.D. 134, 136 (S.D.N.Y. 2009) (Peck, M.J.).

“This Opinion should serve as a wake-up call to the Bar in this District about the need for careful thought, quality control, testing, and cooperation with opposing counsel in designing search terms or ‘keywords’ to be used to produce e-mails or other electronically stored information (‘ESI’),” I wrote.

After this good set-up, Judge Peck gets to the meat of his essay, predictive coding and why lawyers should embrace this new technology. First he makes a stab at explaining what this new technology is all about:

By computer-assisted coding, I mean tools (different vendors use different names) that use sophisticated algorithms to enable the computer to determine relevance, based on interaction with (i.e., training by) a human reviewer.

Unlike manual review, where the review is done by the most junior staff, computer-assisted coding involves a senior partner (or team) who review and code a “seed set” of documents. The computer identifies properties of those documents that it uses to code other documents. As the senior reviewer continues to code more sample documents, the computer predicts the reviewer’s coding. (Or, the computer codes some documents and asks the senior reviewer for feedback.)

When the system’s predictions and the reviewer’s coding sufficiently coincide, the system has learned enough to make confident predictions for the remaining documents. Typically, the senior lawyer (or team) needs to review only a few thousand documents to train the computer.

Then Judge Peck addresses one reason that many lawyers are reluctant to use computer assisted analysis and review, that there is no judicial opinion that expressly approves of computer assisted review. He first asserts that no opinion expressly approves of keyword search either, in fact many criticize it, but attorneys use it anyway. I have to disagree with him slightly on that. Many cases by implication endorse keywords, at least if properly done, and one opinion by Judge Thomas F. Hogan enthusiastically endorses it.

… the glory of electronic information is not merely that it saves space but that it permits the computer to search for words or ‘strings’ of text in seconds.

In re Lorazepam & Clorazepate, 300 F.Supp.2d 43, 46 (D.D.C. 2004). To this I would add that the glory of electronic information is not merely that it permits keyword searches, but that it allows us to harness the artificial intelligence of computers to boost our own.

Judge Peck then address the Daubert straw-man that Judge Facciola unintentionally interjected into the discussion concerning the use of computer intelligence:

In addition to reluctance to be the guinea pig for a decision on predictive coding, lawyers perhaps are concerned that they will have to go through a Daubert hearing as to the “admissibility” of the results of predictive coding. Perhaps this fear comes from O’Keefe, where Judge Facciola said that opining on what keyword is better “is truly to go where angels fear to tread,” and is a topic “beyond the ken of a layman and requires that any such conclusion be based on evidence that, for example, meets the criteria of Rule 702 of the Federal Rules of Evidence,” dealing with expert opinions.

Lawyers’ fears in this regard seem largely misplaced. First, Facciola’s comments were directed at keywords, but everyone is using keywords, and I know of no decision after O’Keefe requiring expert testimony as to the use of keywords.

Second, with due respect to Facciola, I do not think Daubert applies — it applies when an expert will testify at trial in order to admit into evidence opinions or results (e.g., the result of DNA testing reveals a match).

Here, the hundreds of thousands of e-mails produced are not being offered into evidence at trial as the result of a scientific process. Rather, whether the handful of e-mails offered as trial exhibits is admissible is dependent on the document itself (e.g., whether it is a party admission or a business record), not how it was found during discovery.

Very good points Judge Peck. He then goes on to articulate standards for the judicious use of predictive coding, including sampling, cooperation, and transparency. You should read the article in full and study this part. He concludes his article with these reassuring words:

Until there is a judicial opinion approving (or even critiquing) the use of predictive coding, counsel will just have to rely on this article as a sign of judicial approval. In my opinion, computer-assisted coding should be used in those cases where it will help “secure the just, speedy, and inexpensive” (Fed. R. Civ. P. 1) determination of cases in our e-discovery world.

Artificial Intelligence and the Law

There you have it. Now at least one judge has endorsed the use of artificial intelligence to assist in the discovery and production of relevant evidence. I predict there will be many more. It is only common sense. We use computers for everything already. To paraphrase Judge Hogan, the glory of electronic information is that it allows us to harness the artificial intelligence of computers to boost our own.

Don’t get me wrong. I don’t think that AI is a panacea. Well trained lawyers are still indispensable. Ever heard the expression garbage in, garbage out? There is no substitute for skilled human judgments, but now our limited human intelligence and processing speed can be enhanced with predictive coding and other forms of computer assisted document review and analysis. As Judge Peck mentioned, new software is now entering the market that can help us to do that. This software is usually part of early case assessment tools, or built into review platforms, or both (my preference). The programs allow you to harness the power of the computer to assist in your searches, to amplify and expand upon your determinations of relevance, privilege and other issues. It is commonly called predictive coding, or technology assisted reviews, but I prefer to call it by the more general terminology of artificial intelligence, for that is what it is (“the study and design of intelligent agents”).

Two Ways to Use Predictive Coding

There are really two different ways to use AI in e-discovery today: 1. to search and find information, i.w. to cull-out; and, 2. to speed up review prior to production, i.w. quality control. Many people, even experts, get confused between the two and Judge Peck’s article does not address the distinction. It is important to understand the differences. The first is now well accepted as a replacement for manual culling and as a supplement for keyword culling. The second is also accepted by most experts as a good way to speed up manual review, but is not accepted as a total replacement for human review. The technology is not there yet, and neither is legal protection for inadvertent productions of privilege and confidential ESI. Sorry, but the New York Times article is wrong: Armies of Expensive Lawyers, Replaced by Cheaper Software. See: NY Times Discovers e-Discovery, But Gets the Jobs Report Wrong where I point out that the advancement of AI and the law will lead to more and better skilled jobs for lawyers, not less.

The first way to use predictive coding is to use the AI of the computer to help you select the ESI that is probably relevant from an unsorted larger mass of data.  This is ESI that is usually gathered from one or more custodians – their email and attachments, and their loose files. In this first, intake stage you use predictive coding to carve out the ESI in bulk, to cull out the probable irrelevant and cull in the probable relevant. This is using AI for search. So, for instance, as Judge Peck describes, an expert, a human expert, reviews and analyzes a few thousand documents and marks them as either relevant or irrelevant. If marked relevant, the expert also determines whether they are highly relevant, or just marginally so, whether they are privileged or not, or whether they pertain to certain issues, or confidentiality rankings. The computer then studies the documents marked, and using its programmed AI the computer applies what it has learned about relevance, issues, privilege and confidentiality to all of the remaining data.

The computer acts as an intelligent agent and uses what you taught it by your analysis to find the documents that it thinks are likley to be relevant, the issues they are relevant to, privileged, and confidential. It does this over billions of files and it does this very, very fast. The lawyer expert then studies the results and fine tunes the search. He or she teaches the computer some more. The process repeats according to budgetary constraints, and perhaps also precision and recall criteria, until the lawyer is satisfied that a good collection of probably-relevant-documents has been sorted from the larger whole. That is stage one, the culling search and intake stage, but it certainly does not end there. You do not simply produce all of the documents that you and the computer have found through sampling and AI to be probably relevant.

The state our of legal intelligence using this new software, although very high because lawyers now have an intelligent computer agents to assist, is still limited. Mistakes are still made. Guaranteed. If you have attained a precision ratio of 75% you are doing very good. But that means that 25% of the documents you have identified in stage one as likely relevant are not in fact relevant. That number may be as high as 50%. Regardless of the specific ratio, no system is anywhere near perfect at this early stage of AI development.

Your set of  probably-relevant ESI will contain many, many documents that are not. Further, it will contain privileged and secret documents that you are under no obligation to produce. These documents may cause irreparable damages if disclosed. That is where the next stage of human review before production comes in, a review that is again, or should be, computer assisted by predictive coding, including clustering of like documents. The second review stage is essentially a quality control function to weed out irrelevant ESI, identify highly relevant, and withhold privileged and confidential ESI.

Some requesting parties, especially those with little or no ESI of their own, think that the producing party should just skip this second stage and produce without review. Or that the second stage can be expedited by sampling and only some of the documents reviewed. Wrong. We are not there yet. The technology is not advanced enough yet for that. Moreover, despite some limited success with clawbacks and Rule 502, the law does not yet provide strong enough protections to the producing party for the inevitable errors in stage one.

The requesting party is only entitled to see the other side’s documents if they are relevant. That protects the requesting party from a document dump (something the government is famous for) and the producing party from a fishing expedition. What if the attorney for the requesting party has many different law suits for different clients against the requesting party? A document irrelevant for one suit might be highly relevant for another. It might even give the requesting attorney an idea for future law suits. Some large plaintiffs firms are known to go out and recruit plaintiffs for potentially lucrative class action litigation.

Further, privileged documents may be missed by the thousands in stage one culling, which is relevancy driven. Privileged communications and work product are missed in distressingly large quantities, even after a thorough stage two review. They would be missed in far greater quantities without it. That is the real game stopper. Once the bell has been rung, it can’t be unrung. Forced lobotomies are illegal is this country. The attorneys and requesting parties on the other side will know your secrets, both attorney client and work product, even of they do return your documents in a clawback.. That provides them with an unfair advantage, which, depending on the secrets, can be very significant. Then there is the whole question of confidential information, from key trade-secrets to very personal data. If not properly labeled or redacted, the secrets may be lost forever.

No. AI has not progressed to the point where we can dispense with stage two review of documents, all of the documents. Stage two cannot yet be performed with confidence based on sampling alone. But the computer with predictive coding and clustering can significantly speed up the manual review. We can go from fifty files per hour, to several hundred files per hour, at least in some cases where the ESI is fairly simple and not too dense. But the state of technology and law today still requires eyeballs on all ESI before it goes out the door and into the hands of the enemy. The negative consequences of disclosure of secrets, especially attorney-client privilege and work product privilege secrets, is simply too high. This will likely change in the future as judges start to put some real teeth into Rule 502 and technology improves, but we are not there yet.

The Dark Vision of Jason Baron

Jason Baron is a pioneer in this area, especially in the stage one use of AI for search, as all readers of this blog well know. But did you also know he was the 2011 Emmett Leahy Winner for excellence in records and information management? It is a big deal in the world of archivists and records managers. Take a look at the YouTube video below of Jason’s acceptance speech. Suggest you also watch the excellent introduction by Judge John Facciola. The several other intros are not that interesting for us non-librarian e-discovery types, so skip right to Judge Facciola at 1523 – 3313. John captures Jason’s playful creativity as an essential component of his intelligence. After that, I suggest you skip again to Jason’s speech at 4212-5845. It is good, even if it does end with a dark vision of a post-info-apocalyptic world. Apparently getting a big international award brings out the best in a person.

In his speech, after thanking everyone imaginable, Jason has three messages:

1. He calls for the end of the end-user being expected to act as the de facto records manager. He calls tongue-in-cheek for the workers of the world to unite in opposition to tedious records keeping enslavement. We must instead automate this work. We must turn these tedious records-filing tasks over to artificial intelligence. Amen to that!
2. He calls for more creative interdisciplinary action, which I might point out is also the theme of this blog with its emphasis on the team approach. Specifically, Jason calls upon lawyers to work as a team with experts in the fields of information retrieval and artificial intelligence. He calls upon the legal profession to partner with academia and industry and adopt standards for best-practices in legal search.
3. Jason then concludes with a disturbing vision that keeps him up at night. He warns of a coming dark age of digital flooding, of hopeless unwieldiness, where we all drown in too-much-information. He warns of information overload followed by inability to access anything of importance. He call this a vision of information dystopia. He asks us all to be smarter to avoid this catastrophe. Jason ends by promising to do his part with education on the importance of information management to prevent this from coming to pass.

Computers got us into this mess and they can get us out

Computers got us into the mess of too-much-information, too much complexity in forms of information, and too many places to store information. We are in danger of having so much information that no one can find anything. We are in danger of a world of justice where only the rich can afford to litigate. A world so filled with information that no one really knows anything. A world with too much information, but not enough  knowledge, much less wisdom. This is the nightmare scenario that keeps Jason and other deep thinkers awake at night.

Not me. I have other dreams. I have visions of smart computers augmenting our own intelligence. The know-nothing nightmare scenario can be avoided with still more and better technology, with intelligent agents. We must  fight fire with fire by harnessing the new AI capacities of computers. If we boost our own intelligence and abilities with algorithmic agents we will be able to find the evidence we need in the trillions of documents implicated by even average disputes. We have to use AI agents to boost our abilities to read, review, and analyze all of this ESI. Without AI agents we simply cannot find the right stuff for a reasonable sum. With them, we can and will do wonders. We can boost our analytic, reading, and classification speed a hundredfold. We can survive the flood, maybe even surf on it to ever greater records of achievement.

This is not a utopian dream. It will happen. As mentioned, we are not there yet. But it is only a matter of time as the algorithms, the AI, improves, and as our human skills improve. Yes, AI agents alone will not get us there. Advanced software is worthless without advanced attorneys and consultants to use them, and advanced judges to enforce the law. Again I come back to my training mantra, yet the marketplace, so far at least, seems much more interested in certification than education. Still, I remain hopeful. The new day is just beginning.

It is a New Dawn, Not a Sunset

Jason ends his speech by asking us to be smarter to avoid the information dystopia, the information dark ages. But he does not say how are we to get smarter. Here are my two main suggestions and I think Jason would agree. More education and more artificial intelligence. The only way we will ever get smart enough is by having our computers help us. Without AI agents, we will never be smart enough to deal with the deluge and Jason’s dark age will come true.

But not to worry too much, there will be no dark age. There will be smart computer agents to help us think and act smarter. Have you seen the new iPhone 4S with SIRI, the automated personal assistant? Just think what SIRI will be like in a few more years. Just think what the latest software tools in e-discovery will be like. Siri, find me the top ten relevant documents from John Smith’s email.

Sleep well. It is dark now, but we are certainly not approaching a dark age of information overload. The Technology Age is just beginning. AI and the law is just getting started. Thousands of coders around the world are working on this problem right now. A great light is coming. This is the dawn, not the sunset.

The future will not only bring more information, it will bring more processing. The new AI technologies will make us smart and the light will shine on all. Some day stage one search will reach near total recall and near perfect precision. Some day stage two AI assisted review will be so good that sampling will suffice, and production errors will approach zero. Some day the courts will protect producing parties when some errors still take place.

There is already a plethora of evidence out there, and, as Jason often points out, you ain’t seen nothing yet. The exponential curve in data multiplication is just starting. In the near future writings in the general sense of ESI will be even more voluminous and complex than we can now imagine. They will be everywhere. ESI on everything will be ubiquitous, but so will information processing, so will AI agents. Not to worry, the processing will keep up. The intelligence of our computers will increase even more dramatically than the information we generate with them. AI agents will be just as ubiquitous as raw information itself. Siri, pass me the salt.

Conclusion

We we will soon be able to preserve and mine the flood of facts that technology has made possible. We will be able to find the truth much more often and efficiently than ever before.  Witnesses will be assisted, neigh sometimes even forced to tell the truth by these past writings. The flood of truth will evoke a flood of justice. An age of legal enlightenment will follow the advances in information processing and analysis. See: We Are at the Dawn of a Golden Age of Justice. The truth is out there, we just need to learn how to find it. And with the help of Siri and Watson, and human sages like Peck and Baron, we will my friends, we will.

The information explosion is about to trigger a justice explosion. It will happen as soon as law catches up with technology. It will happen as soon as law learns to harvest the vast new fields of digital evidence that technology has created. We will then enjoy a much higher level of justice than we do now. Our judges and juries will reach just decisions more often because they will know more of the truth, more of the time. All justice is dependent upon truth, upon objective facts. The technology revolution has created far, far more information than ever before. As the law learns to retrieve this information there will be far, far more justice than ever before.

In law we find justice by the light of truth. We find truth by evidence of facts, the objective facts of past and present events and mental states. We find this truth through writings and through the testimony of witnesses, and sometimes through physical evidence, such as guns or fingerprints. Testimony is the oral account of witnesses as to the facts – what they saw, heard, and did. Although testimony is based upon solemn oaths to tell the truth, all lawyers and judges know that it is inherently subjective and never completely reliable. Testimony is built on the frailties of memory and observation. The great advances in technology have done little, if anything, to improve the reliability of our memory or our testimony. Our memory of facts remain as vague and subjective as as ever. (Someday the polygraph may reveal intentional lies, but we are not there yet, and it will likely never reveal unintentional errors or subjective coloring.)

The law has learned that testimony only leads to a reasonably accurate account of objective facts when supplemented with writings (and physical objects, when applicable) made at the time of the events in question. Writings, traditionally called documents in the law, are understood in the broadest sense to include not only paper (the mere tip of the iceberg), but all electronic information, including videos and computer records. Writings are the guardians of truth, the refreshers of memory that keep people honest. They are the best, most accurate repository of objective facts, and have been for centuries.

This should give us great hope for the future of justice because the extent and complexity of writings is now exploding exponentially. As soon as law learns to search and retrieve this new explosion of electronic evidence, cases will be decided on more and better documentary evidence than ever before. Testimony will become more reliable because it will be checked and constrained by more and more contemporaneous writings. A new dawn of justice will become possible as intentional and unintentional lies are exposed. We will be able to decide cases based on more of the truth than ever before. The flood of new information, new writings, will generate a new age of justice that far exceeds anything we have seen to date.

It is a good time to be a lawyer. If you can ride and understand the tidal wave of new technologies and new writings, you can advance the cause of justice. You can find the writings needed for judges and juries to make their decisions based upon accurate evidence. The truth is out there in vastly greater quantities than ever before. Yes, it is intangible, just electronic pulses representing zeros and ones. Yes, it is sometimes hidden like a needle in a haystack. But it can be found. It will be found by the lawyers of today and tomorrow as we learn e-discovery.

We can all tap into the information age to fill courts with truth. The days of justice based on paper and candlepower are over. Lawyers should instead sing the body electric.

Hear this change of tune my fellow legal educators. Join with me in a more positive approach. For every story of doom, gloom, and sanctions, tell two of victory, justice, and joy. Spread the word of how ESI was used to win the day. Let us not base our teachings on fear of failure, but on love of justice. Love is a far more powerful motivator. Love what you do. Do what you love. As Steve Jobs said: The only way to do great work is to love what you do. As Joseph Campbell said: Follow your bliss.

We need great work now from many in the law, great work inspired by love, love of justice, love of truth, freedom, and liberty. When we realize what is at stake, when we can see clearly the herald of coming good, the golden age of justice on the horizon, it is easy to love what we do. Discover this inner truth and your love of e-discovery will follow, your great works will change the world.

After I published the blog this week on polls and confidentiality, a reader, whose name will go unmentioned, gave me a good tip on a recent news event directly on point to the corporate confidentiality issue. On September 27, 2010, Google filed a motion to intervene in the U.S. Justice Department’s antitrust enforcement action against AT&T. Google seeks to intervene to attain greater protection of its documents under a protective order already set up in the case. See: Google Wants Business Data Protected In AT&T Antitrust Case.  According to this article by Mike Scarcella in the Blog of LegalTimes:

The attorneys for Google said this evening that the company provided Justice confidential documents during its probe of AT&T’s planned $39 billion acquisition of rival T-Mobile. The company wants the ability to object in advance to any public disclosure of confidential information.   ….

The protective order entered in the antitrust case earlier this month fails, according to Google, to require the parties in the dispute always to provide advance notice to Google of potential disclosure of confidential information.

Google’s attorneys said they want advance notice from the lawyers in the case when certain documents may be put on the public record, mentioned in open court or given to experts.

Google provided this confidential information in response to a subpoena as part of the government’s investigation of AT&T’s planned $39 billion acquisition of T-Mobile. Google now claims that “Disclosure of such plans to competitors could not only harm Google, but likely also competition in mobile markets.” Id. For full disclosure, see the now public because filed with the court Motion and Memorandum in Support of Non-Party Google Inc.’s Motion to Intervene and supporting exhibits. It looks to be a very well done pleading as you would expect in a case of this importance. The signatories are Michael L. Keeley and John D. Harkrider of Axinn, Veltrop & Harkrider LLP.

Many will smile at the irony of Google now fighting for privacy. Google has been sued many times for invading the privacy of others, especially including in Europe. See Eg.: Google, FTC settle charges of privacy violations; Google Privacy Violation Conviction: Three Executives Found Guilty In Italy; France Fines Google $142000 for Privacy Violations. What do you think?