evidence | e-Discovery Team

Second Circuit Uses “Klipsch” to Speak Loud and Clear on e-Discovery Misconduct

September 23, 2018

Klipsch makes some of the best speakers in the world, especially their high-end Klipschorn speaker series shown here. The Second Court of Appeals used a Klipsch in a dispute recently to sound an alarm about e-discovery abuse and how it will not be tolerated. Klipsch Group, Inc. v. ePRO E-Commerce Ltd., 880 F. 3d 620 (2d Cir, 2018). The unanimous opinion written by Circuit Judge Gerard E. Lynch upheld the lower court’s sanctions against Klipch’s adversary in the suit, ePro. The adverse inference sanction entered means that Klipsch will now almost certainly win the case.

The Second Circuit also affirmed the full amount of monetary sanctions, $2,680,000. Klipsch was compensated for almost* all of the additional discovery efforts occasioned by ePRO’s misconduct. ePro was also required to immediately secure payment of the full $5 Million amount of any future judgment against them, which includes damage trebling and fees. I know that sounded great to Klipsch and their attorneys. Especially since they had a four day evidentiary hearing on their spoliation motion to get there.

* Somewhat sadly for Klipsch, however, as the third footnote explained, under the lower court’s opinion all of Klipsch’s fees and costs incurred were not awarded. There was another $300,000 or so that was not included, only because Klipsch could not produce proper credentials for one of the billing attorneys. Ouch. That is extremely rare and odd. Details are so very important.

The Second Circuit Court’s Rare Ruling on e-Discovery and Sanctions

The Klipsch opinion has language that will carry great weight, not only because it makes sense, but also because it is from a United States Court of Appeals panel. There are not that many appeals court opinions on sanctions or e-discovery so this is a rare and important opinion. It will certainly encourage more courts to do as the trial judge did here and sanction offending parties. It is also controlling law for all federal courts in its jurisdiction, which is Vermont, Connecticut and the all important New York.

In the rest of this blog I will let the language of the Klipsch opinion speak for itself with a few of my favorites selections. Klipsch Group, Inc. v. ePRO E-Commerce Ltd., 880 F. 3d 620 (2d Cir, 2018). The first quote from Circuit Judge Gerard Lynch’s opinion lays out the background.

In the course of defending against claims that it sold counterfeit products, defendant-appellant ePRO E-Commerce Limited (“ePRO”) engaged in persistent discovery misconduct: it failed to timely disclose the majority of the responsive documents in its possession, restricted a discovery vendor’s access to its electronic data, and failed to impose an adequate litigation hold even after the court directed it to do so, which omission allowed custodians of relevant electronic data to delete thousands of documents and significant quantities of data, sometimes permanently. As a result, the United States District Court for the Southern District of New York (Vernon S. Broderick, J.) concluded that ePRO had willfully engaged in spoliation. It accordingly granted in substantial part plaintiff-appellee Klipsch Group, Inc.’s (“Klipsch”) motion for discovery sanctions, including a $2.7 million monetary sanction to compensate Klipsch for its corrective discovery efforts and a corresponding asset restraint in that amount, permissive and mandatory jury instructions, and an additional $2.3 million bond to preserve Klipsch’s ability to recover damages and fees at the end of the case. . . .

We find no error in the district court’s factual findings, and we conclude that the monetary sanctions it awarded properly compensated Klipsch for the corrective discovery efforts it undertook with court permission in response to ePRO’s misconduct. In particular, we emphasize that discovery sanctions should be commensurate with the costs unnecessarily created by the sanctionable behavior. A monetary sanction in the amount of the cost of discovery efforts that appeared to be reasonable to undertake ex ante does not become impermissibly punitive simply because those efforts did not ultimately uncover more significant spoliation and fraud, or increase the likely damages in the underlying case. The district court’s orders imposing sanctions *624 are accordingly AFFIRMED in all respects.

The next quote gives you a good glimpse into the degree of frustration that recalcitrant attorneys who excel in gamesmanship can engender, even in an appellate court panel located in New York City:

[T]he history of the case makes clear that the sanctions and fees awarded in this case were carefully limited to costs Klipsch incurred in direct response to ePRO’s misconduct. Klipsch obtained approval from the magistrate judge prior to each of its substantive efforts, and in each case, that approval was given only after ePRO had already squandered an opportunity to correct its own errors.

For example, ePRO’s failure to implement a litigation hold was first discovered in March 2013, during Klipsch’s first round of depositions with ePRO employees, but ePRO was not sanctioned at that time, nor was Klipsch given carte blanche to explore ePRO’s files. Instead, ePRO was permitted to hire its own discovery expert to correct the error, which resulted in the production of substantial additional discovery. Klipsch then spent approximately $550,000 on a second round of depositions occasioned by that late production.

Paul Klipsch

It is evident that the district court did not detect any abusive conduct on the part of Klipsch, such as the piling on of discovery demands and investigatory initiatives in order to burden its *633 adversary with wasteful expenses, motions practice, and sanctions. ePRO does not appear to contest the reasonableness of permitting Klipsch to take those remedial depositions, nor can it plausibly assert that Klipsch would have insisted on doing so even if ePRO’s initial production had been complete or timely. And only in March 2014, after ePRO had repeatedly shown itself to be an untrustworthy participant in the discovery process, did the magistrate judge determine that Klipsch was “fully justified” in seeking to undertake an independent forensic examination. Joint App. at 1187.

Because the costs for which Klipsch is being compensated were reasonably incurred in direct response to ePRO’s misconduct, we cannot conclude that the district court abused its discretion by requiring ePRO to pay monetary sanctions in that amount.^[7]

The defendant ePro then tried the “proportionality card” arguing that the multi-million dollar punitive damages exceeded the amount at issue in the case. That did not work. The Second Circuit explained that is not how proportionality works in sanctions. It has no bearing to the value of the case itself, just the amount of additional reasonable costs incurred because of the unreasonable conduct to be sanctioned. One party should not be damaged by the unreasonable conduct of the other. The offended party, here Klipsch, should, in effect, be indemnified from all of the burden and expense incurred because of what the offending party did or did not do. They are the innocent party. Here is how the court put it.

In sum, we see nothing in ePRO’s proportionality arguments compelling us to conclude that the district court abused its discretion by awarding full compensation for efforts that were ex ante a reasonable response to ePRO’s own evasive conduct.^[9] The proportionality that matters here is that the amount of the sanctions was plainly proportionate — indeed, it was exactly equivalent — to the costs ePRO inflicted on Klipsch in its reasonable efforts to remedy ePRO’s misconduct.

The Second Court was cognizant of the issues and problems concerning electronic discovery. They had been briefed on the potential for misuses of spoliation sanctions motions. Although this was not at all present in the Klipsch, Judge Lynch address this concern near the end of the opinion.

Nothing that we say in this opinion should be taken as condoning excessive and disproportionate discovery demands, countenancing the tactical use of discovery sanction motions to inflict gratuitous costs on adversaries, or derogating from the responsibility *636 of district courts to ensure that litigation proceeds in a responsible and cost-efficient manner. See Fed. R. Civ. P. 1 (directing that the Rules of Civil Procedure “should be construed, administered, and employed by the court and the parties to secure the just, speedy, and inexpensive determination of every action”) (emphasis added); Fed. R. Civ. P. 26(b)(1) (scope of discovery should be “proportional to the needs of the case, considering [inter alia] the amount in controversy”). If it turns out, as the district court has estimated, that the amount of actual damages in this case is modest in relation to the costs spent on the litigation, that would be a highly regrettable outcome.

But the question before the district court, and before us, is which party should be held responsible for those costs. ePRO does not ever contend that Klipsch’s initial discovery demands were unreasonable or disproportionate to the merits of the case. Nor does it seriously argue that the magistrate judge erred in allowing Klipsch to take the steps it took to remedy ePRO’s refusal to comply with those demands. The district court reasonably concluded, after a full and fair hearing, that it was ePRO’s noncompliance with its legal obligations that occasioned the excessive costs in this case, and we find no reason why ePRO should not therefore be required to pay them.

I leave you with a short video biography of Paul Klipch, a recognized audio engineer leader whom I have long admired.

2 Comments | Forensic Exam, Lawyers Duties, New Rules, Review, Search, Spoliation/Sanctions, Technology | Tagged: ethics, evidence, preservation, PROPORTIONALITY, sanctions | Permalink
Posted by Ralph Losey

Responding Party’s Complaints of Financial Burden of Document Review Were Unsupported by the Evidence, Any Evidence

August 5, 2018

One of the largest cases in the U.S. today is a consolidated group of price-fixing cases in District Court in Chicago. In Re Broiler Chicken Antitrust Litigation, 290 F. Supp. 3d 772 (N.D. Ill. 2017) (order denying motions to dismiss and discussing the case). The consolidated antitrust cases involve allegations of a wide spread chicken price-fixing. Big Food Versus Big Chicken: Lawsuits Allege Processors Conspired To Fix Bird Prices (NPR 2/6/18).

The level of sales and potential damages are high. For instance, in 2014 the sales of broiler chickens in the U.S. was $32.7 Billion. That’s sales for one year. The classes have not been certified yet, but discovery is underway in the consolidated cases.

The Broiler Chicken case is not only big money, but big e-discovery. A Special Master (Maura Grossman) was appointed months ago and she developed a unique e-discovery validation protocol order for the case. See: TAR for Smart Chickens, by John Tredennick and Jeremy Pickens that analyzes the validation protocol.

Maura was not involved in the latest discovery dispute where, Agri Stats, one of many defendants, claimed a request for production was too burdensome as to it. The latest problem went straight to the presiding Magistrate Judge Jeffrey T. Gilbert who issued his order on July 26, 2018. In re Broiler Chicken Antitrust Litig., 2018 WL 3586183 (N.D. Ill. 7/26/18).

Agri Stats had moved for a protective order to limit an email production request. Agri Stats claimed that the burden imposed was not proportional because it would be too expensive. Its lawyers told Judge Gilbert that it would cost between $1,200,000 and $1,700,00 to review the email using the keywords negotiated.

Fantasy Hearing

I assume that there were hearings and attorney conferences before the hearings. But I do not know that for sure. I have not seen a transcript of the hearings with Judge Gilbert. All we know is that defense counsel told the judge that under the keywords selected the document review would cost between $1,200,000 and $1,700,000, and that they had no explanation on how the cost estimate was prepared, nor any specifics as to what it covered. Although I was not there, after four decades of doing this sort of work, I have a pretty good idea of what was or might have been said at the hearing.

This representation of million dollar costs by defense counsel would have gotten the attention of the judge. He would naturally have wanted to know how the cost range was calculated. I can almost hear the judge say from the bench: “$1.7 Million Dollars to do a doc review. Yeah, ok. That is a lot of money. Why so much counsel? Anyone?” To which the defense attorneys said in response, much like the students in Ferris Beuller’s class:

“. . . . . .”

Yes. That’s right. They had Nothing. Just Voodoo Economics

Well, Judge Gilbert’s short opinion makes it seem that way. In re Broiler Chicken Antitrust Litig., 2018 WL 3586183 (N.D. Ill. 7/26/18).

If a Q&A interchange like this happened, either in a phone hearing, or in person, then the lawyers must have said something. You do not just ignore a question by a federal judge. The defense attorneys probably did a little hemming and hawing, conferred among themselves, and then said something to the judge like: “We are not sure how those numbers were derived, $1.2M to $1.5M, and will have to get back to you on that question, Your Honor.” And then, they never did. I have seen this kind of thing a few times before. We all try to avoid it. But it is even worse to make up a false story, or even present an unverified story to the judge. Better to say nothing and get back to the judge with accurate information.

Discovery Order of July 26, 2018

Here is a quote from Judge Gilbert’s Order so you can read for yourself the many questions the moving party left unanswered (detailed citations to record removed; graphics added):

Agri Stats represents that the estimated cost to run the custodial searches EUCPs propose and to review and produce the ESI is approximately $1.2 to $1.7 million. This estimated cost, however, is not itemized nor broken down for the Court to understand how it was calculated. For example, is it $1.2 to $1.7 million to review all the custodial documents from 2007 through 2016? Or does this estimate isolate only the pre-October 2012 custodial searches that Agri Stats does not want to have to redo, in its words? More importantly, Agri Stats also admits that this estimate is based on EUCPs’ original proposed list of search terms. But EUCPs represent (and Agri Stats does not disagree) that during their apparently ongoing discussions, EUCPs have proposed to relieve Agri Stats of the obligation to produce various categories of documents and data, and to revise the search terms to be applied to data that is subject to search. Agri Stats does not appear to have provided a revised cost estimate since EUCPs agreed to exclude certain categories of documents and information and revised their search terms. Rather, Agri Stats takes the position that custodial searches before October 3, 2012 are not proportional to the needs of the case — full stop — so it apparently has not fully analyzed the cost impact of EUCPs’ revised search terms or narrowed document and data categories.

The Court wonders what the cost estimate is now after EUCPs have proposed to narrow the scope of what they are asking Agri Stats to do. (emphasis added) EUCPs say they already have agreed, or are working towards agreement, that 2.5 million documents might be excluded from Agri Stats’s review. That leaves approximately 520,000 documents that remain to be reviewed. In addition, EUCPs say they have provided to Agri Stats revised search terms, but Agri Stats has not responded. Agri Stats says nothing about this in its reply memorandum.

EUCPs contend that Agri Stats’s claims of burden and cost are vastly overstated. The Court tends to agree with EUCPs on this record. It is not clear what it would cost in either time or money to review and produce the custodial ESI now being sought by EUCPs for the entire discovery period set forth in the ESI Protocol or even for the pre-October 3, 2102 period. It seems that Agri Stats itself also does not know for sure what it would have to do and how much it would cost because the parties have not finished that discussion. Because EUCPs say they are continuing to work with Agri Stats to reduce what it must do to comply with their discovery requests, the incremental burden on what Agri Stats now is being asked to do is not clear.

For all these reasons, Agri Stats falls woefully short of satisfying its obligation to show that the information [*10] EUCPs are seeking is not reasonably accessible because of undue burden or cost.

Estimations for Fun and Profit

In order to obtain a protective order you need to estimate the costs that will likely be involved in the discovery from which you seek protection. Simple. Moreover, it obviously has to be a reasonable estimate, a good faith estimate, supported by the facts. The Brolier Chicken defendant, Agri Stats, came up with an estimate. They got that part right. But then they stopped. You never do that. You do not just throw up a number and hope for the best. You have to explain how it was derived. Blushing at any price higher than that is not a reasonable explanation, but is often honest.

Be ready to explain how you came up with the cost estimate. To break down the total into its component parts and allow the “Court to understand how it was calculated.” Agri Stats did not do that. Instead, they just used a cost estimate of between $1.2 to $1.7 million. So of course Agri Stats’ motion for protective order was denied. The judge had no choice because no evidence to support the motion was presented, neither factual or expert evidence. There was no need for Judge Gilbert to go into the secondary questions of whether expert testimony was also needed and whether it should be under Rule 702. He got nothing remember. No explanation for the $1.7 Million.

The lesson of the latest discovery order in Broiler Chicken is pretty simple. In re Broiler Chicken Antitrust Litig., 2018 WL 3586183 (N.D. Ill. 7/26/18). Get a real cost estimate from an expert. The expert needs to know and understand document review, search and costs of review. They need to know how to make reasonable search and retrieval efforts. They also need to know how to make reliable estimates. You may need two experts for this, as not all have expertise in both fields, but they are readily available. Many can even talk pretty well too, but not all! Seriously, everybody knows we are the most fun and interesting lawyer subgroup.

The last thing to do is skimp on an expert and just pull out a number from your hat (or your vendor’s hat) and hope for the best.

This is federal court, not a political rally. You do not make bald assertions and leave the court wondering. Facts matter. Back of the envelope type guesses are not sufficient, especially in a big case like Broiler Chicken. Neither are guesstimates by people who do not know what they are doing. Make disclosure and cooperate with the requesting party to reach agreement. Do not just rush to the courthouse hoping to dazzle with smoke and mirrors. Bring in the experts. They may not dazzle, but they can get you beyond the magic mirrors.

Case Law Background

Judge Paul S. Grewal, who is now Deputy G.C. of Facebook, said quoting The Sedona Conference in Vasudevan: “There is no magic to the science of search and retrieval: only mathematics, linguistics, and hard work.” Vasudevan Software, Inc. v. Microstrategy Inc., No. 11-cv-06637-RS-PSG, 2012 US Dist LEXIS 163654 (ND Cal Nov 15, 2012) (quoting The Sedona Conference, Best Practices Commentary on the Use of Search and Information and Retrieval Methods in E-Discovery, 8 Sedona Conf. J. 189, 208 (2007). There is also no magic to the art of estimation, no magic to calculating the likely range of cost to search and retrieve the documents requested. Judge Grewal refused to make any decision in Vasudevan without expert assistance, recognizing that this area is “fraught with traps for the unwary” and should not be decided on mere arguments of counsel.

Judge Grewal did not address the procedural issue of whether Rule 702 should govern. But he did cite to Judge Facciola’s case on the subject, United States v. O’Keefe, 537 F. Supp. 2d 14 (D.D.C. 2008). Here Judge Facciola first raised the discovery expert evidence issue. He not only opined that experts should be used, but that the parties should follow the formalities of Evidence Rule 702. That governs things such as whether you should qualify and swear in an expert and follow otherwise follow Rule 702 on their testimony. I discussed this somewhat in my earlier article this year, Judge Goes Where Angels Fear To Tread: Tells the Parties What Keyword Searches to Use.

Judge Facciola in O’Keffe held that document review issues require expert input and that this input should be provided with all of the protections provided by Evidence Rule 702.

Given this complexity, for lawyers and judges to dare opine that a certain search term or terms would be more likely to produce information than the terms that were used is truly to go where angels fear to tread. This topic is clearly beyond the ken of a layman and requires that any such conclusion be based on evidence that, for example, meets the criteria of Rule 702 of the Federal Rules of Evidence. Accordingly, if defendants are going to contend that the search terms used by the government were insufficient, they will have to specifically so contend in a motion to compel and their contention must be based on evidence that meets the requirements of Rule 702 of the Federal Rules of Evidence.

Conclusion

In the Boiler Chicken Antitrust Order of July 27, 2018, a motion for protective order was denied because of inadequate evidence of burden. All the responding party did was quote a price-range, a number presumably provided by an expert, but there was no explanation. More evidence was needed, both expert and fact. I agree that generally document review cost estimation requires opinions of experts. The experts need to be proficient in two fields. They need to know and understand the science of document search and retrieval and the likely costs for these services for a particular set of data.

Although all of the formalities and expense of compliance with Evidence Rule 702 may be needed in some cases, it is probably not necessary in most. Just bring your expert to the attorney conference or hearing. Yes, two experts may well disagree on some things, probably will, but the areas of agreement are usually far more important. That in turn makes compromise and negotiation far easier. Better leave the technical details to the experts to sort out. That follows the Rule 1 prime directive of “just, speedy and inexpensive.” Keep the trial lawyers out of it. They should instead focus and argue on what the documents mean.

2 Comments | Lawyers Duties, New Rules, Review, Search | Tagged: best practices, computer assisted review, discovery, document review, education, evidence, junk science, legal profession, legal search, PROPORTIONALITY, science, search, tar, technology, vendors | Permalink
Posted by Ralph Losey

Evidence Code Revisions and the Grimm/Brady Evidence Admissibility Chart

April 22, 2018

Great fanfare was provided for the changes to the Federal Rules of Civil Procedure in December 2015. But not much attention has been given to the December 2017 changes to the Federal Rules of Evidence. Maybe that has to do with the disappearing trial, the fact that less than one percent of federal cases actually go to trial. Still, you need to know the rules of evidence admissibility, even if you are preparing for a trial that will never come. You need to collect and discover evidence in a way that it can be used, even if it is just in a motion for summary judgment.

Two New Subsections to Rule 902 on Self-Authenticating Evidence

In December 2017 two new subsections were added to Evidence Rule 902, subsections (13) and (14). They are designed to streamline authentication of electronically stored information (ESI). The goal is to eliminate the need to call a witness at trial to authenticate evidence, at least in most instances. Here are the two new provisions:

Rule 902. Evidence That Is Self-Authenticating

The following items of evidence are self-authenticating; they require no extrinsic evidence of authenticity in order to be admitted: . . .

(13) Certified Records Generated by an Electronic Process or System. A record generated by an electronic process or system that produces an accurate result, as shown by a certification of a qualified person that complies with the certification requirements of Rule 902(11) or (12). The proponent must also meet the notice requirements of Rule 902(11).

(14) Certified Data Copied from an Electronic Device, Storage Medium, or File. Data copied from an electronic device, storage medium, or file, if authenticated by a process of digital identification, as shown by a certification of a qualified person that complies with the certification requirements of Rule (902(11) or (12). The proponent also must meet the notice requirements of Rule 902 (11).

The Evidence Rules Committee Notes explain the background of these two new subsections.

Committee Notes on Rules—2017 Amendment

Paragraph (14). The amendment sets forth a procedure by which parties can authenticate data copied from an electronic device, storage medium, or an electronic file, other than through the testimony of a foundation witness. As with the provisions on business records in Rules 902(11) and (12), the Committee has found that the expense and inconvenience of producing an authenticating witness for this evidence is often unnecessary. It is often the case that a party goes to the expense of producing an authentication witness, and then the adversary either stipulates authenticity before the witness is called or fails to challenge the authentication testimony once it is presented. The amendment provides a procedure in which the parties can determine in advance of trial whether a real challenge to authenticity will be made, and can then plan accordingly.

Today, data copied from electronic devices, storage media, and electronic files are ordinarily authenticated by “hash value”. A hash value is a number that is often represented as a sequence of characters and is produced by an algorithm based upon the digital contents of a drive, medium, or file. If the hash values for the original and copy are different, then the copy is not identical to the original. If the hash values for the original and copy are the same, it is highly improbable that the original and copy are not identical. Thus, identical hash values for the original and copy reliably attest to the fact that they are exact duplicates. This amendment allows self-authentication by a certification of a qualified person that she checked the hash value of the proffered item and that it was identical to the original. The rule is flexible enough to allow certifications through processes other than comparison of hash value, including by other reliable means of identification provided by future technology.

Nothing in the amendment is intended to limit a party from establishing authenticity of electronic evidence on any ground provided in these Rules, including through judicial notice where appropriate.

A proponent establishing authenticity under this Rule must present a certification containing information that would be sufficient to establish authenticity were that information provided by a witness at trial. If the certification provides information that would be insufficient to authenticate the record of the certifying person testified, then authenticity is not established under this Rule.

The reference to the “certification requirements of Rule 902(11) or (12)” is only to the procedural requirements for a valid certification. There is no intent to require, or permit, a certification under this Rule to prove the requirements of Rule 803(6). Rule 902(14) is solely limited to authentication, and any attempt to satisfy a hearsay exception must be made independently.

A certification under this Rule can only establish that the proffered item is authentic. The opponent remains free to object to admissibility of the proffered item on other grounds—including hearsay, relevance, or in criminal cases the right to confrontation. For example, in a criminal case in which data copied from a hard drive is proffered, the defendant can still challenge hearsay found in the hard drive, and can still challenge whether the information on the hard drive was placed there by the defendant.

A challenge to the authenticity of electronic evidence may require technical information about the system or process at issue, including possibly retaining a forensic technical expert; such factors will affect whether the opponent has a fair opportunity to challenge the evidence given the notice provided.

The reference to Rule 902(12) is intended to cover certifications that are made in a foreign country.

Also see: Paul Grimm, Gregory Joseph, Daniel Capra, Manual on Best Practices for Authenticating Digital Evidence; Authenticating Digital Evidence, 69 BAYLOR L. REV. 1 (2017).

Grimm/Brady Evidence Admissibility Chart

The rule change is a helpful addition to the litigator’s toolkit, but many challenges remain for attorneys handling electronic evidence. I agree with Kevin Brady, who is a top expert in the field of ESI evidence, who says that “the challenge for lawyers trying to authenticate digital evidence using the traditional rules of evidence can be confusing.” This may be an understatement! Kevin thinks that part of the challenge for attorneys arises from the rapidly-evolving landscape of data sources. He gives examples such as bitcoin, blockchain, smart contracts, social media, IoT, mobile devices, and cloud computing services. Moreover, the use of social media like Facebook, LinkedIn, Instagram and others continues to increase at an unbelievable rate and adds to the problem. Moreover, according to Business Insider, there are more people using the top four social messaging apps (WhatsApp, Messenger, WeChat, and Viber) than the top four social media apps (Facebook, Instagram, Twitter, and LinkedIn). According to Tech Crunch, Facebook’s Messenger alone has more than 1.3 billion monthly active users, and Instagram is officially testing a standalone messaging app, Direct.

Recognizing the problem Kevin Brady teamed up with U.S. District Court Judge Paul Grimm, the leading judicial expert in the field, to create the Grimm/Brady Evidence Admissibility Chart shown below.

The detailed reference chart provides discovery lawyers and trial attorneys with a quick reference guide for handling many different sources of ESI evidence. It covers Rule 104 to Rule 803(6) to Rule 901 and 902. The chart provides a step by step approach for authenticating digital information and successfully getting that information admitted into evidence.

The e-Discovery Team highly recommends that you carefully study this chart. Click on the photos and they will open in a larger size. Also suggest you download your own copy here: Grimm Brady Evidence Admission Chart 2018. Many thanks to Kevin Brady for helping me with this blog.

1 Comment | Evidence, Lawyers Duties, New Rules, Technology | Tagged: evidence, law | Permalink
Posted by Ralph Losey

TAR Course Expands Again: Standardized Best Practice for Technology Assisted Review

February 11, 2018

The TAR Course has a new class, the Seventeenth Class: Another “Player’s View” of the Workflow. Several other parts of the Course have been updated and edited. It now has Eighteen Classes (listed at end). The TAR Course is free and follows the Open Source tradition. We freely disclose the method for electronic document review that uses the latest technology tools for search and quality controls. These technologies and methods empower attorneys to find the evidence needed for all text-based investigations. The TAR Course shares the state of the art for using AI to enhance electronic document review.

The key is to know how to use the document review search tools that are now available to find the targeted information. We have been working on various methods of use since our case before Judge Andrew Peck in Da Silva Moore in 2012. After we helped get the first judicial approval of predictive coding in Da Silva, we began a series of several hundred document reviews, both in legal practice and scientific experiments. We have now refined our method many times to attain optimal efficiency and effectiveness. We call our latest method Hybrid Multimodal IST Predictive Coding 4.0.

The Hybrid Multimodal method taught by the TARcourse.com combines law and technology. Successful completion of the TAR course requires knowledge of both fields. In the technology field active machine learning is the most important technology to understand, especially the intricacies of training selection, such as Intelligently Spaced Training (“IST”). In the legal field the proportionality doctrine is key to the pragmatic application of the method taught at TAR Course. We give-away the information on the methods, we open-source it through this publication.

All we can transmit by online teaching is information, and a small bit of knowledge. Knowing the Information in the TAR Course is a necessary prerequisite for real knowledge of Hybrid Multimodal IST Predictive Coding 4.0. Knowledge, as opposed to Information, is taught the same way as advanced trial practice, by second chairing a number of trials. This kind of instruction is the one with real value, the one that completes a doc review project at the same time it completes training. We charge for document review and throw in the training. Information on the latest methods of document review is inherently free, but Knowledge of how to use these methods is a pay to learn process.

The Open Sourced Predictive Coding 4.0 method is applied for particular applications and search projects. There are always some customization and modifications to the default standards to meet the project requirements. All variations are documented and can be fully explained and justified. This is a process where the clients learn by doing and following along with Losey’s work.

What he has learned through a lifetime of teaching and studying Law and Technology is that real Knowledge can never be gained by reading or listening to presentations. Knowledge can only be gained by working with other people in real-time (or near-time), in this case, to carry out multiple electronic document reviews. The transmission of knowledge comes from the Q&A ESI Communications process. It comes from doing. When we lead a project, we help students to go from mere Information about the methods to real Knowledge of how it works. For instance, we do not just make the Stop decision, we also explain the decision. We share our work-product.

Knowledge comes from observing the application of the legal search methods in a variety of different review projects. Eventually some Wisdom may arise, especially as you recover from errors. For background on this triad, see Examining the 12 Predictions Made in 2015 in “Information → Knowledge → Wisdom” (2017). Once Wisdom arises some of the sayings in the TAR Course may start to make sense, such as our favorite “Relevant Is Irrelevant.” Until this koan is understood, the legal doctrine of Proportionality can be an overly complex weave.

The TAR Course is now composed of eighteen classes:

First Class: Background and History of Predictive Coding
Second Class: Introduction to the Course
Third Class: TREC Total Recall Track, 2015 and 2016
Fourth Class: Introduction to the Nine Insights from TREC Research Concerning the Use of Predictive Coding in Legal Document Review
Fifth Class: 1st of the Nine Insights – Active Machine Learning
Sixth Class: 2nd Insight – Balanced Hybrid and Intelligently Spaced Training (IST)
Seventh Class: 3rd and 4th Insights – Concept and Similarity Searches
Eighth Class: 5th and 6th Insights – Keyword and Linear Review
Ninth Class: 7th, 8th and 9th Insights – SME, Method, Software; the Three Pillars of Quality Control
Tenth Class: Introduction to the Eight-Step Work Flow
Eleventh Class: Step One – ESI Communications
Twelfth Class: Step Two – Multimodal ECA
Thirteenth Class: Step Three – Random Prevalence
Fourteenth Class: Steps Four, Five and Six – Iterative Machine Training
Fifteenth Class: Step Seven – ZEN Quality Assurance Tests (Zero Error Numerics)
Sixteenth Class: Step Eight – Phased Production
Seventeenth Class: Another “Player’s View” of the Workflow (class added 2018)
Eighteenth Class: Conclusion

With a lot of hard work you can complete this online training program in a long weekend, but most people take a few weeks. After that, this course can serve as a solid reference to consult during complex document review projects. It can also serve as a launchpad for real Knowledge and eventually some Wisdom into electronic document review. TARcourse.com is designed to provide you with the Information needed to start this path to AI enhanced evidence detection and production.