e-Discovery Team

Here is the e-Discovery Team’s most interesting e-discovery opinion of 2016: Hyles v. New York City, No. 10 Civ. 3119 (AT)(AJP), 2016 WL 4077114 (S.D.N.Y. Aug. 1, 2016)!!!!! The author is well-known e-discovery expert, Judge Andrew Peck of the SDNY.

ONE – Hyles v. New York City

This opinion, like that of Number Two, Dynamo Holdings, is on the e-Discovery Team’s favorite topic, predictive coding. Admittedly, that had a lot to do with the Team’s pick of Hyles as this year’s most interesting e-discovery opinion. Hyles v. New York City, No. 10 Civ. 3119 (AT)(AJP), 2016 WL 4077114 (S.D.N.Y. Aug. 1, 2016). So too did the fact that it was written by Judge Peck. He is known for his excellent legal analysis, especially on the legal search topics. The Hyles opinion was not only spot on, it had the clarity and writing quality demanded of any opinion to be ranked one in 2016.

Although Hyles did not make new law, it clarified existing law on predictive coding. We like that. The specific issue addressed in Hyles has been discussed before, but never squarely ruled on by Judge Peck. In Hyles the plaintiff wanted to require the defendant City (i.e., the responding party) to use TAR (technology assisted review, aka predictive coding), instead of the method the City preferred of keyword searching. As expected, Judge Peck ruled that a party cannot be forced to do predictive coding, even if it is a better method than what the party wants to do, in this case, keyword search; and, even if the party’s preferred method had not yet started. As expected, the reason for this ruling was old Sedona Principle Six.

We liked the Hyles opinion, over Dynamo Two, because Hyles does not include descriptions of cockamamy methods of predictive coding, like Dynamo does. Instead, Hyles involves a more basic methodology, one faced by most e-discovery practitioners today, not just predictive coding specialists, on how to cull down the ESI universe subject to review for relevance by: (1) custodian priority; and, (2) date range. This is a predictive coding case that covers pre-predictive coding methods. It that sense Hyles is like Judge Peck’s other classic legal search opinion from the pre-predictive coding era, Gross Construction. William A. Gross Constr. Assocs., Inc. v. Am. Mutual Mfrs. Ins. Co., 256 F.R.D. 134, 134 (S.D.N.Y. 2009).

Here is how Judge Peck described counsels efforts to agree upon a method to cull the universe of documents to be reviewed for relevance.

As to date range, the parties agreed on a start date of September 1, 2005 but disagreed on the end date. … After hearing the parties’ arguments at the conference, the Court ruled that the end date would be April 30, 2010 (when defendant Patricoff was reassigned from her First Deputy Commissioner position), without prejudice to Hyles seeking documents or ESI from a later period, if justified, on a more targeted inquiry basis.

Notice how Judge Peck ruled, but without prejudice for the requesting party to come back later, if need be. Most discovery rulings on issues like that should be open-ended, with the idea that any follow-up requests must be narrow and focused, and thus relatively inexpensive to fulfill.

Judge Peck makes the same kind of ruling at to the total number of custodians whose ESI must be reviewed.

As to custodians, the City agreed to search the files of nine custodians (including Hyles), but not six additional custodians that Hyles requested. (7/18/16 Ltr. at 5, 7.) The Court ruled that discovery should be staged, by starting with the agreed upon nine custodians (Hyles, Stark, Patricoff and six others). After reviewing the production from the nine custodians, if Hyles could demonstrate that other custodians had relevant, unique and proportional ESI, the Court would consider targeted searches from such other custodians.^[1]

Here is how Judge Peck quickly frames the dispute that the parties brought to him for resolution. (All record citations omitted.)

After the parties had initial discussions about the City using keywords, Hyles’ counsel consulted an ediscovery vendor and proposed that the City should use TAR as a “more cost-effective and efficient method of obtaining ESI from Defendants.”  The City declined, both because of cost and concerns that the parties, based on their history of scope negotiations, would not be able to collaborate to develop the seed set for a TAR process.

Judge Peck began his analysis as you might expect by agreeing with the plaintiff that “in general, TAR is cheaper, more efficient and superior to keyword searching.” Then he set out the legal precedent history embodying his thinking on predictive coding and whether a party should be required to use TAR against their will.

In March 2009, the “dark ages” in terms of ediscovery advances, this Court described problems with keywords and the need for “careful thought, quality control, testing, and cooperation with opposing counsel in designing search terms or `keywords.'” William A. Gross Constr. Assocs., Inc. v. Am. Mutual Mfrs. Ins. Co., 256 F.R.D. 134, 134 (S.D.N.Y. 2009) (Peck, M.J.). Further elaborating on the deficiencies of keyword searching, my seminal Da Silva Moore decision in 2012 approved the use of predictive coding, aka TAR, in appropriate cases. Da Silva Moore v. Publicis Groupe, 287 F.R.D. 182, 190-91, 193 (S.D.N.Y. 2012) (Peck, M.J.). In again approving the use of TAR in 2015, I wrote that “the case law has developed to the point that it is now black letter law that where the producing party wants to utilize TAR for document review, courts will permit it.” Rio Tinto PLC v. Vale S.A., 306 F.R.D. 125, 127 (S.D.N.Y. 2015) (Peck, M.J.).^[3] Dicta in a footnote in Rio Tinto stated that “[i]n contrast, where the requesting party has sought to force the producing party to use TAR, the courts have refused.” Rio Tinto PLC v. Vale S.A., 306 F.R.D. at 127 n.1. “The Court note[d], however, that in [the cited] cases the producing party had spent over $1 million using keyword search (in Kleen) or keyword culling followed by TAR (in Biomet), so it is not clear what a court might do if the issue were raised before the producing party had spent any money on document review.” Rio Tinto PLC v. Vale S.A., 306 F.R.D. at 127 n.1. Since the search methodology issue arose in this case before the City spent much, if any, money on searching for responsive ESI, this case squarely raises the issue of whether the requesting party can have the Court force the responding party to use TAR.

The plaintiff also argued that since parties should cooperate in discovery the City should cooperate and use the best technology available to find relevant evidence. Judge Peck rejected this argument as follows:

Hyles’ counsel is correct that parties should cooperate in discovery. I am a signatory to and strong supporter of the Sedona Conference Cooperation Proclamation, and I believe that parties should cooperate in discovery. See William A. Gross Constr. Assocs., Inc. v. Am. Mfrs. Mut. Ins. Co., 256 F.R.D. at 136; Rio Tinto PLC v. Vale S.A., 306 F.R.D. at 129 n.6. The December 1, 2015 Advisory Committee Notes to amended Fed. R. Civ. P. 1 emphasized the need for cooperation. Cooperation principles, however, do not give the requesting party, or the Court, the power to force cooperation or to force the responding party to use TAR.

His comment that a Court does not have the “power to force cooperation” is also interesting and somewhat controversial. Perhaps it is a matter of semantics, but I have seen many judges “order” parties to cooperate. Still, his second point on cooperation is that the doctrine cooperation does not require a responding party to use TAR if they do not want to. Cooperation does not mean capitulation.

Judge Peck then goes on to articulate the main reason that a judge should not ordinarily force a party to use a particular tool or technique to mine client data for useful evidence. That should be the litigant’s independent duty and the court should not interfere without cause. This is part of what is known as The Sedona Conference Principle Six as is well explained by Judge Peck in Hyles.

It certainly is fair to say that I am a judicial advocate for the use of TAR in appropriate cases. I also am a firm believer in the Sedona Principles, particularly Principle 6, which clearly provides that:

Responding parties are best situated to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own electronically stored information.

The Sedona Principles: Second Edition, Best Practices Recommendations & Principles for Addressing Electronic Document Production, Principle 6 (available at http://www.TheSedonaConference.org).

Under Sedona Principle 6, the City as the responding party is best situated to decide how to search for and produce ESI responsive to Hyles’ document requests. Hyles’ counsel candidly admitted at the conference that they have no authority to support their request to force the City to use TAR. The City can use the search method of its choice. If Hyles later demonstrates deficiencies in the City’s production, the City may have to re-do its search.^[4] But that is not a basis for Court intervention at this stage of the case.

Notice how the decision to not compel the use of TAR is without prejudice. The plaintiff can revisit the request by demonstrating deficiencies in the defendant’s production.

Judge Peck then quotes with approval the recent Dynamo Two opinion where Judge Buch held that it was not the court’s business to dictate to attorneys how to do document review, and again relied on Sedona Six. Here is Judge Peck’s concluding words.

Here, too, it is not up to the Court, or the requesting party (Hyles), to force the City as the responding party to use TAR when it prefers to use keyword searching. While Hyles may well be correct that production using keywords may not be as complete as it would be if TAR were used (7/18/16 Ltr. at 4-5), the standard is not perfection, or using the “best” tool (see 7/18/16 Ltr. at 4), but whether the search results are reasonable and proportional. Cf. Fed. R. Civ. P. 26(g)(1)(B).

To be clear, the Court believes that for most cases today, TAR is the best and most efficient search tool. That is particularly so, according to research studies (cited in Rio Tinto), where the TAR methodology uses continuous active learning (“CAL”), which eliminates issues about the seed set and stabilizing the TAR tool. The Court would have liked the City to use TAR in this case. But the Court cannot, and will not, force the City to do so. There may come a time when TAR is so widely used that it might be unreasonable for a party to decline to use TAR. We are not there yet. Thus, despite what the Court might want a responding party to do, Sedona Principle 6 controls. Hyles’ application to force the City to use TAR is DENIED.

Our view of the most interesting opinion of 2916. The e-Discovery Team agrees with Judge Peck that “for most cases today, TAR is the best and most efficient search tool.” To be clear, however, our agreement is predicated upon the TAR tool being used properly. The method of TAR matters. The e-Discovery Team would much rather work on a well-run, well-designed keyword search project, than a mismanaged, poorly designed predictive coding project.

We think the wise words of Judge Facciola in O’Keefe in 2008 about angels have been too early forgotten:

Whether search terms or ‘keywords’ will yield the information sought is a complicated question involving the interplay, at least, of the sciences of computer technology, statistics and linguistics…. Given this complexity, for lawyers and judges to dare opine that a certain search term or terms would be more likely to produce information than the terms that were used is truly to go where angels fear to tread. This topic is clearly beyond the ken of a layman.

United States v. O’Keefe, 537 F. Supp. 2d 14 (D.C. 2008). What Judge Facciola said about keyword search is as true today as when written. When it comes to legal search today using active machine learning, which is the true meaning of TAR, the expertise required is even greater. Predictive coding requires special skills and a unique knowledge set to do right. It is clearly beyond the ken of almost all attorneys practicing law today. We do not see this gap narrowing, not because the education is not available, but because most lawyers are disinterested. For this reason the competency gap is widening and the problem noted by Judge Facciola in 2008 is still alive and well today.

In spite of this competency gap, and the stupid fearlessness of many trial lawyers, those who do not even know what they do not know, courts continue to approve the use of TAR carte blanche, with no requirement of expert assistance or use of proven methodologies. For that reason our agreement with Judge Peck on the superiority of predictive coding must be qualified. Still, we agree, because when active machine learning is done right it is a thing of beauty, far more effective than keywords in all but the simplest projects. Why I Love Predictive Coding: Making document review fun with Mr. EDR and Predictive Coding 3.0. (e-discovery team, 2/14/16).

We also agree with Judge Peck’s speculation that there may come a time when a court forces the use of best practices by recalcitrant lawyers. Judge Peck may even reverse himself on this point before TAR is more widely used, especially if  Sedona Principle Six is revised or shown to be inapplicable to a particular case. Is it really true, as Principle Six asserts, that “Responding parties are best situated to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own electronically stored information.” They may be in the best position for preservation, but are they for search and production? Legal search, especially active machine learning, is a speciality well beyond the IT capabilities and skills of responding parties. Legal search does not require special knowledge of the data itself, it requires special knowledge of the procedures, methodologies, and technologies for electronic document review, including especially predictive coding. This is knowledge possessed by e-discovery specialists, by lawyers who specialize in legal search, not by litigant’s IT departments.

It is hard to see how Principle Six applies to choice of document review software and feature utilization. We need to think this through and have a vigorous debate on the continued application of Principle Six to document review defensibility. See Ball, Craig, Sedona Principle Six: Overdue for an Overhaul (10/10/14) (“It’s time to deep six Sedona Six.“)

In the not too remote future, when Hyles is someday reversed (perhaps by Judge Peck himself) and a party is ordered to use TAR, we expect (hope) that the opinion will also specify the particular method or methods of use of TAR. Otherwise, the order is too general to have any meaning. You might as well order an attorney to use a computer to do document review. There are many, many ways to do TAR. Most of them are wrong. Any attorney angel should fear to tread TAR without the help of experts.

If ESI continues to grow more complicated, and the volumes of data continue to explode, then in the future legal search that includes predictive coding may well be the only way document discovery can be conducted. Litigation lawyers of the future may still do depositions, motion practice, trials and the like, but it is unlikely they will also continue to do large volume document review. They will leave that to active machine learning experts. The improvements we see in the use of artificial intelligence and easier-to-use software will help expand the group of experts, the specialists in document review, but it is likely to bring it within the reach of the general litigator.

The Bar is already faced with a large competency gap. A new type of legal work is emerging to fill that competency gap, a new job, where specialists in AI enhanced evidence search handle all document discovery. A dual track for trial preparation is emerging. One group of lawyers will be concerned with electronic document discovery and another group will handle all of the other litigation tasks.

Looking a little further into the future, we expect courts may eventually turn over the entire ESI search and production process over to neutral expert specialists serving as discovery masters (or something like that). That may well be the best means for the just, speedy and efficient resolution of most law suits.

This entry was posted on Sunday, January 1st, 2017 at 8:44 am and is filed under Lawyers Duties, Review, Search, Technology, VENDORS. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.