The Jimi Hendrix Experience: Sanctions For Destruction

December 2, 2018

James Marshall “Jimi” Hendrix, whom Rolling Stone ranked the greatest guitarist of all time, died intestate in 1970 at twenty-seven. His heirs have been embroiled in litigation ever since. They have recently entered the fiery realm of e-discovery and sanctions. Experience Hendrix, LLC v. Pitsicalis, No. 17-cv-1927 (PAE) (S.D.N.Y., 11/27/18). The opinion by District Court Judge Paul A. Engelmayer is interesting in its own right, but when you add the Hendrix name and family feud, you have a truly memorable order. After all, we are talking about the artist who created “Purple Haze,” “Foxy Lady,” “The Star-Spangled Banner,” “Hey Joe” and my personal favorite, his rendition of Bob Dylan’s “All Along the Watchtower”.

Case Background: The Hendrix Family Feud

The latest suit involves the usual serial litigants. On one side is Jimi’s step-sister, Janie Hendrix (shown right). She is, as Jimi would have said, a “Foxy Lady”. Janie assumed control of the Estate from Jimi’s natural father, Al Hendrix, when he died in 2002. On the other side is Jimi’s brother, Leon Hendrix and Leon’s business partner, Andrew Pitsicalis. Kerzner, Hendrix Sues Serial Infringer Andrew Pitsicalis (American Blues Scene, 3/20/17). There can be big money in the Hendrix name, the top guitarist of all time. I for one still get choked up when I hear his rendition of “The Star-Spangled Banner”:

Jimi’s father, Al Hendrix, won bitterly fought estate litigation in the seventies, but the trademark litigation has never stopped. Multiple injunctions are in place under the corporate name, Experience Hendrix, LLC, but the infringements just keep coming. Companies keep popping up to sell Jimi Hendrix branded goods. Andrew Pitsicalis and Leon Hendrix are frequently involved. Their latest attempts to profit from the Jimi brand include marijuana related products (apparently “Purple Haze” has long been a well know strain of cannabis. See: Legal Battle Ensues Over Jimi Hendrix Usage Rights (High Times, 3/29/17); Jimi Hendrix’s heirs wage court battle over branded cannabis, other products (Cannifornan, 3/22/17).

I am reminded of the closing line of the Hendrix classic, Are You Experienced:

Ah! But Are You Experienced?
Have you ever been experienced?

Not necessarily stoned, but beautiful.

Jimi’s brother, Leon Hendrix (shown right), is an artist and musician himself with his own following. Some think he was treated unfairly by his Dad and Step-Sister. For a variety of reasons, especially I suspect the impact of Pitsicalis, the CEO of “Purple Haze Properties” and Leon’s business partner, there is still bad blood. Chris Fry, Jimi Hendrix’s Brother Fires Back Against Estate (Courthouse News, 3/28/17).

This kind of family feud mentality is not uncommon in litigation, especially in cases involving the intentional destruction of evidence. I am reminded of a Hendrix line from Voodoo Child:

Well, the night I was born. Lord I swear the moon turned a fire red. The night I was born I swear the moon turned a fire red. Well my poor mother cried out “lord, the gypsy was right!” And I seen her, fell down right dead. Have mercy.

Spoliation sanctions generally arise from a haze, just not a stoned purple haze, more like an angry moon turned a fire red haze. Even a seasoned District Court Judge in the SDNY, Paul Engelmayer, was “dismayed” by the conduct of Pitsicalis and Leon Hendrix. Well, what did you expect in matters involving the Estate of a Voodoo Child musical genius? The best guitarist that ever lived?

Judge Engelmayer’s Sanction Order

The scholarly and well-written opinion by District Court Judge Paul A. Engelmayer (shown right) begins by observing:

As the docket in this matter reflects, the Court has been called upon dismayingly often to act when presented with evidence of the PHP defendants’ persistent non-compliance with basic discovery obligations. Plaintiffs now move this Court to sanction these defendants for (1) spoliation of evidence and, more generally, (2) “consistent, pervasive[,] and relentless discovery abuses by [d]efendants and their counsel, Thomas Osinski.” Dkt. 245. Plaintiffs request, inter alia, a preliminary injunction, an order of attachment, an adverse inference instruction at trial, and terminating sanctions. See Dkts. 237, 244. For the reasons below, the Court grants the motion for an adverse inference instruction and directs the PHP defendants to pay the reasonable fees and costs incurred by plaintiffs in bringing this motion.

Experience Hendrix, LLC v. Pitsicalis, No. 17-cv-1927 (PAE) (S.D.N.Y., 11/27/18). Expressing “dismay” is about as emotional as Judge Engelmayer gets in writing an opinion, even one sanctioning a party for destroying evidence and disobeying court orders.

The PHP defendants mentioned are Leon Hendrix, Andrew Pitsicalis and their corporation, Purple Haze Properties (PHP). As you can see from the first quote, the attorney who represents them, Thomas Osinski, was also accused of discovery abuse. That often happens in joker and the thief type cases like this.

A good sanctions case will always have a “parade of horribles” consisting of a list of things the spoliating party supposedly did wrong. Hendrix is no exception. That is how the severe sanctions are justified. It would take too long to list all of the abuses justifying sanctions in Hendrix, but here are the four main ones:

  1. PHP Defendants’ Failure to Produce Forensic Images as Ordered. Apparent intentional disobedience of court orders to make forensic copies of and produce certain drives, even after daily fines are imposed for late production. One of the excuses PHP offered was especially humorous, especially considering the NYC venue, but they actually claimed “that they had had difficulty hiring an expert technician who could image the hard drives.” Yeah, it’s real hard. Need I say more about Osinski’s veracity? When they finally did produce some, but not all of the forensic images, they were not “forensic” images. They were just copies of all active files (a “ghost” copy) with no forensic copy of the slack space. That is what a forensic copy means. It allows for search and examination of deleted files, which was the whole point of the court order.
  2. PHP Defendants’ Use of Anti-Forensic Software. Software allowing for the complete wiping of files was found installed on several of the computer images that were produced. In some there was evidence the software was installed immediately after a court order was entered requiring production. In these the plaintiff’s forensic expert could also show that the software, CleanMyMac, was actually used to wipe files and when, although it was not possible to know what files were destroyed. The moving party (Janie Hendrix and her company Experience Hendrix, L.L.C.)  proved the use was knowing when their expert, John T. Myers, showed how the software was configured to have a pop-out and warn the user to confirm complete elimination of the file (it cannot be recovered after that). The defendants testified that they did not recall ever using it. Sure. Spoliate evidence and then cover-up, or try to.
  3. Andrew Pitsicalis Deleted “Jimi”-related Text Messages from his iPhone. Plaintiff’s forensic expert was able to prove that more than 500 text messages had been deleted from Pitsicalis cell phone after the duty to preserve had arisen (suit was filed). Moreover, they were able to recover nine text messages pertaining to Jimi. As Judge Engelmayer explained: “Fortuitously, Myers was able to recover the deleted text messages from the imaged phone because those communications had been stored not in the applications used to send and receive them (e.g., iMessage), but in databases where files exist until overwritten or otherwise purged.”
  4. Key Computer at First Hidden, then After Discovery in Photograph, Goes South to Florida and is Never Examined. This one sounds like a bad game of Where’s Waldo. A “Seventh Computer” was found, one never reported by PHP, by plaintiff’s study of photos on PHP’s Facebook page. Very clever. One picture on FB showed Andrew Pitsicalis, sitting in his office, in immediate proximity to a mystery desktop computer. When asked about it under oath PHP’s fine attorney, Osinski, swore that he thought it was just a dummy Apple monitor on the office desk, not a computer. He said he did not know that the monitor, and key board next to it, were a real, functional computer, an iMac. What? Did he think these were IKEA props in a display room? They were sitting on his client’s desk in a Facebook photo taken after suit was filed. But wait, there is more, Osinski went on to swear that the computer had moved to Florida. As Judge Engelmayer explained:

“Osinski testified that his present understanding is that the desktop computer belonged to an individual named Hector David, Jr. who has moved to Florida and who, Osinski assumes, took the computer with him. Osinski, however, did not have personal knowledge of this, or of the contents of the desktop in Andrew Pitsicalis’ office. … Andrew Pitsicalis, for his part, denied owning the computer and testified that David was not employed by PHP.”

Apparently no one has been able to locate this mysterious Hector David or know where he took the iMac computer sitting on Pitsicalis desk.

Legal Standards of Spoliation in Hendrix

Judge Engelmayer’s opinion in Hendrix examines two legal standards, Rule 37 and Spoliation. He begins the discussion with the duty to preserve, the threshold issue in spoliation:

The first issue is whether the PHP defendants had an obligation to preserve the categories of evidence at issue. A party has an obligation to preserve evidence when it “has notice that the evidence is relevant to litigation . . . [or] should have known that the evidence may be relevant to future litigation.” Kronisch v. United States, 150 F.3d 112 , 126 (2d Cir. 1998) (internal citations omitted).

That standard is easily met here.

He then goes on to discuss whether that duty as breached, another no-brainer based on the impressive parade of horribles in this case:

The Court further finds—and the evidence to this effect is overwhelming—that the PHP defendants repeatedly breached this duty. The breaches fall in three categories: (1) the use of cleaning software on covered computing devices, (2) the failure to disclose the existence of a seventh computing device containing potentially relevant documents, and (3) the deletion of relevant text messages.

Judge Engelmayer’ then discusses the key issue of intent, the mens rea to spoliate.

Much of the PHP defendants’ spoliation of evidence, the Court finds, was intentional. …

The Court finds that, by installing anti-cleaning software on his own computer and causing it to be installed on Schmitt’s in the face of an unambiguous and known duty to preserve potentially relevant evidence, Pitsicalis intentionally caused the destruction of such evidence.

The defenses proffered by the PHP defendants are unavailing. That Schmitt personally may not have acted with the intent to deleted responsive files is beside the point. The relevant mens rea here is that of Andrew Pitsicalis, who owned PHP, for which Schmitt worked as an independent contractor, and who, despite being a repeat litigant amply on notice of his duty to preserve potentially relevant evidence, urged Schmitt to run this software to delete files. Tr. 125, 133. Also unhelpful is Pitsicalis’ [*13] explanation that, at some unspecified point, he went on “Google to search for ‘top anti-forensic software’ and went through the first 10 pages of search” without finding anything for CleanMyMac. Andrew Pitsicalis Decl. at 2-3. Regardless what Pitsicalis’ internet research may have shown, the evidence adduced at the hearing clearly established both that the CleanMyMac software had the capacity to cause the deletion (and shredding) of files, and that Pitsicalis knew this, not least because the software’s causation of such deletion was made explicit to the user each time. Pitsicalis does not offer any reason for installing and using this software on his computer, let alone for having done so without first creating an image of the full contents of the computer that would have assured preservation of the computer’s contents.

Andrew Pitsicalis’ deletion of relevant text messages was also clearly intentional. By his admission, he personally and deliberately deleted, among other text messages, a series of texts concerning the marketing of “Jimi”-related products, the very subject of this lawsuit. He did so one day after the Court issued an order requiring the Purple Haze Properties defendants to: “produce to plaintiffs the forensic images of” every device, including phones, containing files that are relevant to this action. Pitsicalis did not offer any coherent defense to this misconduct. The Court finds it to have been a willful and blatant violation of the duty to preserve relevant evidence.

Finally, the Court finds that the removal of a computer from Andrew Pitsicalis’ office and its transfer to a Floridian, Hector David, Jr., was an act of intentional spoliation. To be sure, the question is a closer one, if only because the contents of that computer are unknown, and so the Court cannot rule out the possibility that these contents were wholly extraneous to this litigation. The location of the computer in Pitsicalis’ office, however, suggests otherwise. Had the Court been notified of the existence of this computer, it assuredly would have ordered that the computer’s contents be searched for responsive materials. It is also noteworthy that Andrew Pitsicalis did not inform his attorney of the existence of this computer. While conceivably these circumstances, in isolation, might have been consistent with the merely reckless disposal of evidence, when this episode is viewed in the light of Pitsicalis’ other acts of willful spoliation, the Court has little difficulty finding it, too, to bespeak intentional misconduct.

Sanctions Imposed

Judge Engelmayer begins his analysis of the appropriate, proportional sanctions by stating the black letter law:

The trial judge must determine the appropriate sanction for spoliation [*14] of evidence on a case-by-case basis. F, 247 F.3d at 436 . Such sanctions should be designed to:

(1) deter parties from engaging in spoliation; (2) place the risk of an erroneous judgment on the party who wrongfully created the risk; and (3) restore the prejudiced party to the same position [they] would have been in absent the wrongful destruction of evidence by the opposing party.

West, 167 F.3d at 779 . Case-dispositive sanctions, however, “should be imposed only in extreme circumstances, usually after consideration of alternative, less drastic sanctions.” Id.

Based on these objectives Judge Engelmayer sanctioned defendants as follows:

Considering these objectives, the Court imposes the following two sanctions, regarding (1) Andrew Pitsicalis’ computer, iPhone, and desktop computer; and (2) Schmitt’s computer, as to each of which the Court has found intentional spoliation. First, the Court will instruct the finder of fact that it may draw an adverse inference from the PHP parties’ failure adequately to preserve and produce these materials, to wit, that the devices in question contained evidence of conduct by the PHP defendants in breach of their legal duties to plaintiffs in connection with the sale and marketing of Jimi Hendrix-related materials.8

Second, given the resources plaintiffs again have had to expend in establishing the above-chronicled acts of non-compliance by the PHP defendants with the Court’s discovery orders, plaintiffs are entitled to an award reflecting the reasonable attorneys’ fees and costs incurred in connection with bringing and litigating the instant successful motion.

Judge Engelmayer went on to explain why a lesser sanction was inappropriate:

The Court has carefully considered whether lesser sanctions are adequate to cure the harm caused by the disposition of these materials. The Court’s firm conclusion is that no lesser sanction than the combination of an adverse inference instruction and an order directing the prompt recompense of plaintiffs for costs reasonably incurred litigating the meritorious motions for sanctions based on spoliation would adequately remedy plaintiffs’ injury. See, e.g., Moody v. CSX Transp., Inc., 271 F. Supp. 3d 410 , 432 (W.D.N.Y. 2017) (finding adverse inference appropriate where defendants intentionally lost material evidence); Ottoson v. SMBC Leasing and Finance, Inc., 268 F. Supp. 3d 570 , 584 (S.D.N.Y. 2017) (granting an adverse inference instruction where plaintiff “has acted willfully or in bad faith” in [*15] violation of her duty to preserve certain emails); First Fin. Sec., Inc. v. Freedom Equity Grp., LLC, No. 15-CV-1893-HRL, 2016 U.S. Dist. LEXIS 140087 , [2016 BL 337069], 2016 WL 5870218 (N.D. Cal. Oct. 7, 2016) (imposing adverse inference instruction for intentional deletion of text messages and awarding plaintiffs attorneys fees incurred in bringing sanctions motions). The Court has also carefully considered whether this is the rare case in which terminating sanctions are merited, as plaintiffs have urged. See Dkt. 237. At the present time, the Court’s judgment is that such extreme sanctions are not warranted, although further acts of spoliation and/or other discovery abuses could produce a different result.

The footnotes in these last paragraphs are interesting. Footnote 8 explains that “The Court defers decision on the precise formulation of the adverse inference instruction until closer to trial.” That means it could become a mandatory presumption, or merely permissive. Footnote 9 acknowledges that there may be more discovery misconduct in the works. The court noted it could still strike all defenses, if the conduct continues, and save everyone the cost of a trial.

Conclusion

Even with just a permissive presumption, the case at this point will almost certainly be won by Janie Hendrix’ company, Experience Hendrix, L.L.C.. Experience Hendrix, LLC v. Pitsicalis. Yet another loss for Jimi’s brother, Leon, in a long list of losses. Another injunction and businesses shut-down, but for how long? The Estate and L.L.C. have won so many times before. Yet they keep coming back. Is this yet another Pyrrhic Victory in a long line of pointless litigation? How long before the next suit? Some things are just beyond Law’s reach. Purple Haze.

Purple Haze

Purple haze all in my brain
Lately things just don’t seem the same
Actin’ funny, but I don’t know why
‘Scuse me while I kiss the sky

Purple haze all around
Don’t know if I’m comin’ up or down
Am I happy or in misery?
Whatever it is, that girl put a spell on me

Help me help me
Oh no no… No

Yeah
Purple haze all in my eyes
Don’t know if it’s day or night
You’ve got me blowin, blowin my mind
Is it tomorrow or just the end of time?

No, help me aw yeah! Oh no no oh help me…

When you are the best in the world at something, like Jimi Hendrix was at guitar playing, and when you are still famous and admired by millions fifty years after your death, there will be profiteers around. When you add sibling rivalry and family resentments to the mix, then the trouble goes from bad to worse.

The Hendrix family saga, and this lawsuit, are tragedies. So too is the destruction of evidence and this Sanctions Order. It is part of his guitar star legend. Jimi Hendrix’ boy genius was born out of a troubled childhood and family. Diamond in the rough. Bigger than life. Exploded with art, fame and fortune in just three years. Dead at age 27 of an overdose. The day he was born the moon turned a fire red, “Lord, the gypsy was right!”

The greatest guitarist of all time was a Phoenix – tragic, fiery, short-lived, but beautiful and spell-binding too. Where will musical genius appear like that again?


Transparency in a Salt Lake TAR Pit?

November 11, 2018

A Salt Lake City Court braved the TAR pits to decide a “transparency” issue. Entrata, Inc. v. Yardi Systems, Inc., Case No. 2:15-cv-00102 (D.C. Utah, 10/29/18). The results were predictable for TAR, which is usually dark. The requesting party tried to compel the respondent to explain their TAR. Tried to force them to disclose the hidden metrics of Recall and Richness. The motion was too little, too late and was denied. The TAR pits of Entrata remain dark. Maybe TAR was done well, maybe not. For all we know the TAR was done by Sponge Bob Square Pants using Bikini Bottom software. We may never know.

Due to the Sponge Bobby type motion leading to an inevitable denial, the requesting party, Yardi Systems, Inc., remains in dark TAR. Yardi still does not know whether the respondent, Entrata,Inc., used active machine learning? Maybe they used a new kind of Bikini Bottom software nobody has ever heard of? Maybe they used KL’s latest software? Or Catalyst? Maybe they did keyword search and passive analytics and never used machine training at all? Maybe they threw darts for search and used Adobe for review? Maybe they ran a series of random and judgmental samples for quality control and assurance? Maybe the results were good? Maybe not?

The review by Entrata could have been a very well managed project. It could have had many built-in quality control activities. It could have been an exemplar of Hybrid Multimodal Predictive Coding 4.0. You know, the method we perfected at NIST’s TREC Total Recall Track? The one that uses the more advanced IST, instead of simple CAL? I am proud to talk about these methods all day and how it worked out on particular projects. The whole procedure is transparent, even though disclosure of all metrics and tests is not. These measurements are anyway secondary to method. Yardi’s motion to compel disclosure should not have been so focused on a recall and richness number. It should instead of focused on methods. The e-Discovery Team methods are spelled out in detail in the TAR Course. Maybe that is what Entrata followed? Probably not. Maybe, God forbid, Entrata used random driven CAL? Maybe the TAR was a classic Sponge Bob Square Pants production of shame and failure? Now Yardi will never know. Or will they?

Yardi’s Quest for True Facts is Not Over

About the only way the requesting party, Yardi, can possibly get TAR disclosure in this case now is by proving the review and production made by Entrata was negligent, or worse, done in bad faith. That is a difficult burden. The requesting party has to hope they find serious omissions in the production to try to justify disclosure of method and metrics. (At the time of this order production by Entrata had not been made.) If expected evidence is missing, then this may suggest a record cleansing, or it may prove that nothing like that ever happened. Careful investigation is often required to know the difference between a non-existent unicorn and a rare, hard to find albino.

Remember, the producing party here, the one deep in the secret TAR, was Entrata, Inc. They are Yardi Systems, Inc. rival software company and defendant in this case. This is a bitter case with history. It is hard for attorneys not to get involved in a grudge match like this. Looks like strong feelings on both sides with a plentiful supply of distrust. Yardi is, I suspect, highly motivated to try to find a hole in the ESI produced, one that suggests negligent search, or worse, intentional withholding by the responding party, Entrata, Inc. At this point, after the motion to compel TAR method was denied, that is about the only way that Yardi might get a second chance to discover the technical details needed to evaluate Entrata’s TAR. The key question driven by Rule 26(g) is whether reasonable efforts were made. Was Entrata’s TAR terrible or terrific? Yardi may never know.

What about Yardi’s discovery? Do they have clean hands? Did Yardi do as good a job at ESI search as Entrata? (Assuing that Yardi used TAR too.) How good was Yardi’s TAR? (Had to ask that!) Was Yardi’s TAR as tardy as its motion? What were the metrics of Yardi’s TAR? Was it dark too? The opinion does not say what Yardi did for its document productions. To me that matters a lot. Cooperation is a mutual process. It is not capitulation. The same goes for disclosure. Do not come to me demanding disclosure but refusing to reciprocate.

How to Evaluate a Responding Party’s TAR?

Back to the TAR at issue. Was Entrata’s TAR riddled with errors? Did they oppose Yardi’s motion because they did a bad job? Was this whole project a disaster? Did Entrata know they had driven into a TAR pit? Who was the vendor? What software was used? Did it have active machine learning features? How were they used? Who was in charge of the TAR? What were their qualifications? Who did the hands-on review? What problems did they run into? How were these problems addressed? Did the client assist? Did the SMEs?

Perhaps the TAR was sleek and speedy and produced the kind of great results that many of us expect from active machine learning. Did sampling suggest low recall? Or high recall? How was the precision? How did this change over the rounds of training. The machine training was continuous, correct? The “seed-set nonsense” was not used, was it? You did not rely on a control set to measure results, did you? You accounted for natural concept drift, didn’t you, where the understanding of relevance changes over the course of the review? Did you use ei-Recall statistical sampling at the end of the project to test your work? Was a “Zero Error” policy followed for the omission of Highly Relevant documents as I recommend?. Are corrective supplemental searches now necessary to try to find missing evidence that is important to the outcome of the case? Do we need to force them to use an expert? Require that they use the state of the art standard, the e-Discovery Team’s Predictive Coding 4.0 Hybrid Multimodal IST?

Yardi’s motion was weak and tardy so Entrata, Inc. could defend its process simply by keeping it secret. This is the work-product defense approach. This is NOT how I would have defended a TAR process. Or rather, not the only way. I would have objected to interference, but also made controlled, limited disclosures. I would have been happy, even proud to show what state of the art search looks like. I would introduce our review team, including our experts, and provide an overview of the methods, the work-flow.

I would also have demanded reciprocal disclosures. What method, what system did you use? TAR is an amazing technology, if used correctly. If used improperly, TAR can be a piece of junk. How did the Subject Matter Experts in this case control the review? Train the machine? Is that a scary ghost in the machine or just a bad SMI?

How did Entrata do it? How for that matter did the requesting party, Yardi, do it? Did it use TAR as part of its document search? Is Yardi proud of its TAR? Or is Yardi’s TAR as dark and hardy har har as Entrata’s TAR. Are all the attorneys and techs walking around with their heads down and minds spinning with secret doc review failures?

e-Discovery Team reviews routinely exceed minimal reasonable efforts; we set the standards of excellence.  I would have made reasonable reassurances by disclosure of method. That builds trust. I would have pointed them to the TAR Course and the 4.0 methods. I would have sent them the below eight-step work-flow diagram. I would have told then that we follow these eight steps or if any deviations were expected, explained why.

I would have invited opposing counsel to participate in the process with any suggested keywords, hot documents to use to train. I would even allow them to submit model fictitious training documents. Let them create fake documents to help us to find any real ones that might be like it, no matter what specific words are used. We are not trying to hide anything. We are trying to find all relevant documents. All relevant documents will be produced, good or bad. Repeat that often. Trust is everything. You can never attain real cooperation without it. Trust but verify. And clarify by talk. Talk to your team, your client, witnesses, opposing counsel and the judge. That is always the first step.

Of course, I would not spend unlimited time going over everything. I dislike meetings and talking to people who have little or no idea what I am saying. Get your own expert. Do the work. These big document review projects often go on for weeks and you could waste and spend a small fortune with too much interaction and disclosures. I don’t want people looking over my shoulder and I don’t want to reveal all of my tricks and work-product secrets, just the general stuff you could get by study of my books. I would have drawn some kind of line of secrecy in the sand, hopefully not quicksand, so that our disclosures were reasonable and not over-burdensome. In Entrata the TAR masters doing the search did not want reveal much of anything. They were very distrustful of Yardi and perhaps sensed a trap. More on that later. Or maybe Entrata did have something to hide? How do we get at the truth of this question without looking at all of the documents ourselves? That is very difficult, but one way to get at the truth is to look at the search methods used, the project metadata.

The dark TAR defense worked for Entrata, but do not count on it working in your case. The requesting party might not be tardy like Yardi. They might make a far better argument.

Order Affirming Denial of Motion to Compel Disclosure of TAR

The well-written opinion in Entrata, Inc. v. Yardi Systems, Inc., (D.C. Utah, 10/29/18) was  by United States District Judge Clark Waddoups. Many other judges have gone over this transparency issue before and Judge Waddoups has a good summary of the ones cited to him by the moving party. I remember tackling these transparency issues with Judge Andrew Peck in Da Silva Moore v. Publicis Groupe, 287 F.R.D. 182 (S.D.N.Y. 2012), which is one of the cases that Judge Waddoups cites. At that time, 2012, there was no precedent even allowing Predictive Coding, much less discussing details of its use, including disclosure best-practices. We made strong efforts of cooperation on the transparency issues after Judge Peck approved predictive coding. Judge Peck was an expert in TAR and very involved in the process. That kind of cooperation that can be encouraged by a very active judge did not happen in Entrata. The cooperative process failed. That led to a late motion by the Plaintiff to force disclosure of the TAR.

The plaintiff, Yardi Systems, Inc, is the party who requested ESI from defendants in this software infringement case. It wanted to know how the defendant was using TAR to respond to their request. Plaintiff’s motion to compel focused on disclosure of the statistical analysis of the results, Recall and Prevalence (aka Richness). That was another mistake. Statistics alone can be meaningless and misleading, especially if range is not considered, including the binomial adjustment for low prevalence. This is explained and covered by my ei-Recall test. Introducing “ei-Recall” – A New Gold Standard for Recall Calculations in Legal SearchPart One, Part Two and Part Three (e-Discovery Team, 2015). Also see: In Legal Search Exact Recall Can Never Be Known.

Disclosure of the whole process, the big picture, is the best Defense Of Process evidence, not just a couple of random sample test results. Looks like the requesting party here might have just been seeking “gotcha material” by focusing so much on the recall numbers. That may be another unstated reason both the Magistrate and District Court Judges denied their late request for disclosure. That could why the attorneys for Entrata kept their TAR dark, even though they were not negligent or in bad faith. Maybe they were proud of their efforts, but were tired of bad faith hassling by the requesting party. Hard to know based on this opinion alone.

After Chief Magistrate Judge Warner denied Yardi’s motion to compel, Yardi appealed to the District Court Judge Waddoups and argued that the Magistrate’s order was “clearly erroneous and contrary to law.”  Yardi argued that “the Federal Rules of Civil Procedure and case law require Entrata, in the first instance, to provide transparent disclosures as a requirement attendant to its use of TAR in its document review.”

Please, that is not an accurate statement of the governing legal precedent. It was instead “wishful thinking” on the part of plaintiff’s counsel. Sounds like a Sponge Bob Square Pants move to me. Judge Waddoups considered taxing fees against the plaintiff under Rule 37(a)(5)(B) because of this near frivolous argument, but ultimately let them off by finding the position was not “wholly meritless.”

Judge Waddoups had no choice but to deny a motion like this filed under these procedures. Here is a key paragraph explaining his reasoning for denial.

The Federal Rules of Civil Procedure assume cooperation in discovery. Here, the parties never reached an agreement regarding search methodology. In the court’s view, the lack of any agreement regarding search methodology is a failure on the part of both parties. Nevertheless, Yardi knew, as early as May of 2017, that Entrata intended to use TAR. (See ECF No. 257-1 at 2.) The Magistrate Court’s September 20, 2017 Order stated, in part, that “[i]f the parties are unable to agree on . . . search methodology within 30 days of the entry of this Order, the parties will submit competing proposals . . . .” (ECF No. 124 at 2.) Yardi, as early as October 2, 2017, knew that “Entrata [was] refus[ing] to provide” “TAR statistics.” (See ECF No. 134 at 3.) In other words, Yardi knew that the parties had not reached an agreement regarding search methodology well before the thirty day window closed. Because Yardi knew that the parties had not reached an agreement on search methodology, it should have filed a proposal with the Magistrate Court. This would have almost certainly aided in resolving this dispute long before it escalated. But neither party filed any proposal with the Magistrate Court within 30 days of entry of its Order. Yardi has not pointed to any Federal Rule of Civil Procedure demonstrating that the Magistrate Court’s Order was contrary to law. This court rejects Yardi’s argument relating to the Federal Rules of Civil Procedure.

Conclusion

The requesting party in Entrata did not meet the high burden needed to reverse a magistrate,s discovery ruling as clearly erroneous and contrary to law. If you are ever going to win on a motion like this, it will likely be on a Magistrate level. Seeking to overturn a denial and meet this burden to reverse is extremely difficult, perhaps impossible in cases seeking to compel TAR disclosure. The whole point is that there is no clear law on the topic yet. We are asking judges to make new law, to establish new standards of transparency. You must be open and honest to attain this kind of new legal precedent. You must use great care to be accurate in any representations of Fact or Law made to a court. Tell them it is a case of first impression when the precedent is not on point as was the situation in Entrata, Inc. v. Yardi Systems, Inc., Case No. 2:15-cv-00102 (D.C. Utah, 10/29/18). Tell them the good and the bad. There was never a perfect case and there always has to be a first for anything. Legal precedent moves slowly, but it moves continuously. It is our job as practicing attorneys to try to guide that change.

The requesting party seeking disclosure of TAR methods in Entrata doomed their argument by case law  misstatements and in-actions. They might have succeeded by making full disclosures themselves, both of the law and their own TAR. The focus of their argument should be on the benefits of doing TAR right and the dangers of doing it wrong. They should have talked more about what TAR – Technology Assisted Review – really means. They should have stressed cooperation and reciprocity.

To make new precedent in this area you must first recognize and explain away a number of opposing principles,  including especially The Sedona Conference Principle Six. That says responding parties always know best and requesting parties should stay out of their document reviews. I have written about this Principle and why it should be updated. Losey, Protecting the Fourteen Crown Jewels of the Sedona Conference in the Third Revision of its Principles (e-Discovery Team, 2//2/17). The Sedona Principle Six argument  is just one of many successful defenses that can be used to protect against forced TAR disclosure. There are also good arguments based on the irrelevance of this search information to claims or defenses under Rule 26(b)(2) and under work-product confidentiality protection.

Any party who would like to force another to make TAR disclosure should make such voluntary disclosures themselves. Walk your talk to gain credibility. The disclosure argument will only succeed, at least for the first time (the all -important test case), in the context of proportional cooperation. An extended 26(f) conference is a good setting and time. Work-product confidentiality issues should be raised in the first days of discovery, not the last day. Timing is critical.

The 26(f) discovery conference dialogue should be directed towards creating a uniform plan for both sides. This means the TAR disclosures should be reciprocal. The ideal test case to make this law would be a situation where the issue is decided early at a Rule 16(b) hearing. It would involve a situation where one side is willing to disclose, but the other is not, or where the scope of disclosures is disputed. At the 16(b) hearing, which usually takes place in the first three months, the judge is supposed to consider the parties’ Rule 26(f) report and address any discovery issues raised, such as TAR method and disclosures.

The first time disclosure is forced by a judge it will almost certainly be a mutual obligation. Each side should will be required to assume the same disclosure obligations. This could include  a requirement for statistical sampling  and disclosure of certain basic metrics such as Recall range, Prevalence and Precision? Sampling tests like this can be run no matter what search method is used, even little old keyword search.

It is near impossible to come into court when both sides have extensive ESI and demand that your opponent do something that you yourself refuse to do. If you expect to be able to force someone to use TAR, or to disclose basic TAR methods and metrics, then you had better be willing to do that yourself. If you are going to try to force someone to disclose work-product protected information, such as an attorney’s quality control tests for Recall range in document review, then you had better make such a limited waiver yourself.

 



Do TAR the Right Way with “Hybrid Multimodal Predictive Coding 4.0”

October 8, 2018

The term “TAR” – Technology Assisted Review – as we use it means document review enhanced by active machine learning. Active machine learning is an important tool of specialized Artificial Intelligence. It is now widely used in many industries, including Law. The method of AI-enhanced document review we developed is called Hybrid Multimodal Predictive Coding 4.0. Interestingly, reading these words in the new Sans Forgetica font will help you to remember them.

We have developed an online instructional program to teach our TAR methods and AI infused concepts to all kinds of legal professionals. We use words, studies, case-law, science, diagrams, math, statistics, scientific studies, test results and appeals to reason to teach the methods. To balance that out, we also make extensive use of photos and videos. We use right brain tools of all kinds, even subliminals, special fonts, hypnotic images and loads of hyperlinks. We use emotion as another teaching tool. Logic and Emotion. Sorry Spock, but this multimodal, holistic approach is more effective with humans than an all-text, reason-only approach of Vulcan law schools.

We even try to use humor and promote student creativity with our homework assignments. Please remember, however, this is not an accredited law school class, so do not expect professorial interaction. Did we mention the TAR Course is free?

By the end of study of the TAR Course you will know and remember exactly what Hybrid Multimodal means. You will understand the importance of using all varieties of legal search, for instance: keywords, similarity searches, concept searches and AI driven probable relevance document ranking. That is the Multimodal part. We use all of the search tools that our KL Discovery document review software provides.

 

The Hybrid part refers to the partnership with technology, the reliance of the searcher on the advanced algorithmic tools. It is important than Man and Machine work together, but that Man remain in charge of justice. The predictive coding algorithms and software are used to enhance the lawyers, paralegals and law tech’s abilities, not replace them.

By the end of the TAR Course you will also know what IST means, literally Intelligently Spaced Training. It is our specialty technique of AI training where you keep training the Machine until first pass relevance review is completed. This is a type of Continuous Active Learning, or as Grossman and Cormack call it, CAL. By the end of the TAR Course you should also know what a Stop Decision is. It is a critical point of the document review process. When do you stop the active machine teaching process? When is enough review enough? This involves legal proportionality issues, to be sure, but it also involves technological processes, procedures and measurements. What is good enough Recall under the circumstances with the data at hand? When should you stop the machine training?

We can teach you the concepts, but this kind of deep knowledge of timing requires substantial experience. In fact, refining the Stop Decision was one of the main tasks we set for ourself for the  e-Discovery Team experiments in the Total Recall Track of the National Institute of Standards and Technology Text Retrieval Conference in 2015 and 2016. We learned a lot in our two years. I do not think anyone has spent more time studying this in both scientific and commercial projects than we have. Kudos again to KL Discovery for helping to sponsor this kind of important research  by the e-Discovery Team.

 

 

Working with AI like this for evidence gathering is a newly emerging art. Take the TAR Course and learn the latest methods. We divide the Predictive Coding work flow into eight-steps. Master these steps and related concepts to do TAR the right way.

 

Pop Quiz: What is one of the most important considerations on when to train again?

One Possible Correct Answer: The schedule of the humans involved. Logistics and project planning is always important for efficiency. Flexibility is easy to attain with the IST method. You can easily accommodate schedule changes and make it as easy as possible for humans and “robots” to work together. We do not literally mean robots, but rather refer to the advanced software and the AI that arises from the machine training as an imiginary robot.

 

 

 

 

 

 

 

 

 


%d bloggers like this: