Robophobia: Great New Law Review Article – Part 2

May 26, 2022
Professor Andrew Woods

This article is Part Two of my review of Robophobia by Professor Andrew Woods. See here for Part 1.

I want to start off Part 2 with a quote from Andrew Woods in the Introduction to his article, Robophobia, 93 U. Colo. L. Rev. 51  (Winter, 2022). Footnotes omitted.

Deciding where to deploy machine decision-makers is one of the most important policy questions of our time. The crucial question is not whether an algorithm has any flaws, but whether it outperforms current methods used to accomplish a task. Yet this view runs counter to the prevailing reactions to the introduction of algorithms in public life and in legal scholarship. Rather than engage in a rational calculation of who performs a task better, we place unreasonably high demands on robots. This is   robophobia – a bias against robots, algorithms, and other nonhuman deciders. 

Robophobia is pervasive. In healthcare, patients prefer human diagnoses to computerized diagnoses, even when they are told that the computer is more effective.  In litigation, lawyers are reluctant to rely on – and juries seem suspicious of – [*56] computer-generated discovery results, even when they have been proven to be more accurate than human discovery results. . . .

In many different domains, algorithms are simply better at performing a given task than people. Algorithms outperform humans at discrete tasks in clinical health, psychology, hiring and admissions, and much more. Yet in setting after setting, we regularly prefer worse-performing humans to a robot alternative, often at an extreme cost. 

Woods, Id. at pgs. 55-56

Bias Against AI in Electronic Discovery

Electronic discovery is a good example of the regular preference of worse-performing humans to a robot alternative, often at an extreme cost. There can be no question now that any decent computer assisted method will significantly outperform human review. We have made great progress in the law through the outstanding leadership of many lawyers and scientists in the field of ediscovery, but there is still a long way to go to convince non-specialists. Professor Woods understands this well and cites many of the leading legal experts on this topic at footnotes 137 to 148. Even though I am not included in his footnotes of authorities (what do you expect, the article was written by a mere human, not an AI), I reproduce them below in the order cited as a grateful shout-out to my esteemed colleagues.

  • Maura R. Grossman & Gordon V. Cormack, Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient than Exhaustive Manual Review, 17 Rich. J.L. & Tech. 1 (2011).
  • Sam Skolnik, Lawyers Aren’t Taking Full Advantage of AI Tools, Survey Shows, Bloomberg L. (May 14, 2019) (reporting results of a survey of 487 lawyers finding that lawyers have not well utilized useful new tools).
  • Moore v. Publicis Groupe, 287 F.R.D. 182, 191 (S.D.N.Y. 2012) (“Computer-assisted review appears to be better than the available alternatives, and thus should be used in appropriate cases.”) Judge Andrew Peck.
  • Bob Ambrogi, Latest ABA Technology Survey Provides Insights on E-Discovery Trends, Catalyst: E-Discovery Search Blog (Nov. 10, 2016) (noting that “firms are failing to use advanced e-discovery technologies or even any e-discovery technology”).
  • Doug Austin, Announcing the State of the Industry Report 2021, eDiscovery Today (Jan. 5, 2021),
  • David C. Blair & M. E. Maron, An Evaluation of Retrieval Effectiveness for a Full-Text Document-Retrieval System, 28 Commc’ns ACM 289 (1985).
  • Thomas E. Stevens & Wayne C. Matus, Gaining a Comparative Advantage in the Process, Nat’l L.J. (Aug. 25, 2008) (describing a “general reluctance by counsel to rely on anything but what they perceive to be the most defensible positions in electronic discovery, even if those solutions do not hold up any sort of honest analysis of cost or quality”).
  • Rio Tinto PLC v. Vale S.A., 306 F.R.D. 125, 127 (S.D.N.Y. 2015). Judge Andrew Peck.
  • See The Sedona Conference, The Sedona Conference Best Practices Commentary on the Use of Search & Information Retrieval Methods in E-Discovery, 15 Sedona Conf. J. 217, 235-26 (2014) (“Some litigators continue to primarily rely upon manual review of information as part of their review process. Principal rationales [include] . . . the perception that there is a lack of scientific validity of search technologies necessary to defend against a court challenge . . . .”).
  • Doug Austin, Learning to Trust TAR as Much as Keyword Search: eDiscovery Best Practices, eDiscovery Today (June 28, 2021).
  • Robert Ambrogi, Fear Not, Lawyers, AI Is Not Your Enemy, Above Law (Oct. 30, 2017).

Robophobia Article Is A First

Robophobia is the first piece of legal scholarship to address our misjudgment of algorithms head-on. Professor Woods makes this assertion up front and I believe it. The Article catalogs different ways that we now misjudge poor algorithms. The evidence of our robophobia is overwhelming, but, before Professor Woods work, it had all been in silos and was not seriously considered. He is the first to bring it all together and consider the legal implications.

His article goes on to suggests several reforms, also a first. But before I get to that, a more detailed overview is in order. The Article is in six parts. Part I provides several examples of robophobia. Although a long list, he says it is far from exhaustive. Part II distinguishes different types of robophobia. Part III considers potential explanations for robophobia. Part IV makes a strong, balanced case for being wary of machine decision-makers, including our inclination to, in some situations, over rely on machines. Part V outlines the components of his case against robophobia. The concluding Part VI offers “tentative policy prescriptions for encouraging rational thinking – and policy making – when it comes to nonhuman deciders.

Part II of the Article – Types of Robophobia

Professor Woods identifies five different types of robophobia.

  • Elevated Performance Standards: we expect algorithms to greatly outperform the human alternatives and often demand perfection.
  • Elevated Process Standards: we demand algorithms explain their decision-making processes clearly and fully; the reasoning must be plain and understandable to human reviewers.
  • Harsher Judgments: algorithmic mistakes are routinely judges more severely than human errors. A corollary of elevated performance standards.
  • Distrust: our confidence in automated decisions is week and fragile. Would you rather get into an empty AI Uber, or one driven by a scruffy looking human?
  • Prioritizing Human Decisions: We must keep “humans in the loop” and give more weight to human input than algorithmic.

Part III – Explaining Robophobia

Professor Woods considers seven different explanations for robophobia.

  • Fear of the Unknown
  • Transparency Concerns
  • Loss of Control
  • Job Anxiety
  • Disgust
  • Gambling for Perfect Decisions
  • Overconfidence in Human Decisions

I’m limiting my review here, since the explanations for most of these should be obvious by now and I want to limit the length of my blog. But the disgust explanation was not one I expected and a short quote by Andrew Woods might be helpful, along with the robot photo I added.

Uncannily Creepy Robot

[T]he more that robots become humanlike, the more they can trigger feelings of disgust. In the 1970s, roboticist Masahiro Mori hypothesized that people would be more willing to accept robots as the machines became more humanlike, but only up to a point, and then human acceptance of nearly-human robots would decline.[227] This decline has been called the “uncanny valley,” and it has turned out to be a profound insight about how humans react to nonhuman agents. This means that as robots take the place of humans with increasing frequency—companion robots for the elderly, sex robots for the lonely, doctor robots for the sick—reports of robots’ uncanny features will likely increase.

For interesting background on the uncanny valley, see these You Tube videos and experience robot disgust for yourself. Uncanny Valley by Popular Science 2008 (old, but pretty disgusting). Here’s a more recent and detailed one, pretty good, by a popular twenty-something with pink hair. Why is this image creepy? by TUV 2022.

Parts IV and V – The Cases For and Against Robophobia

Part IV lays out all the good reasons to be suspect of delegating decision to algorithms. Part V is the new counter-argument, one we have not heard before, why robophobia is bad for us. This is probably the heart of the article and suggest you read this part for sure.

Here is a good quote at the end of Part IV to put the pro versus anti-robot positions into perspective:

Pro-robot bias is no better than antirobot bias. If we are inclined both to over- and underrely on robots, then we need to correct both problems—the human fear of robots is one piece of the larger puzzle of how robots and humans should coexist. The regulatory challenge vis-à-vis human-robot interactions then is not merely minimizing one problem or the other but rather making a rational assessment of the risks and rewards offered by nonhuman decision-makers. This requires a clear sense of the key variables along which to evaluate decision-makers.

In the first two paragraphs of Part V of his article Professor Woods deftly summarizes the case against robophobia.

We are irrational in our embrace of technology, which is driven more by intuition than reasoned debate. Sensible policy will only come from a thoughtful and deliberate—and perhaps counterintuitive—approach to integrating robots into our society. This is a point about the policymaking process as much as it is about the policies themselves. And at the moment, we are getting it wrong—most especially with the important policy choice of where to transfer control from a human decider to a robot decider.

Specifically, in most domains, we should accept much more risk from algorithms than we currently do. We should assess their performance comparatively—usually by comparing robots to the human decider they would replace—and we should care about rates of improvement. This means we should embrace robot decision-makers whenever they are better than human decision-makers. We should even embrace robot decision-makers when they are less effective than humans, as long as we have a high level of confidence that they will soon become better than humans. Implicit in this framing is a rejection of deontological claims—some would say a “right”—to having humans do certain tasks instead of robots.[255] But, this is not to say that we should prefer robots to humans in general. Indeed, we must be just as vigilant about the risks of irrationally preferring robots over humans, which can be just as harmful.[256]


The concluding Part Three of my review of Robophobia is coming soon. In the meantime, take a break and think about Professor Woods policy-based perspective. That is something practicing lawyers like me do not do often enough. Also, it is of value to consider Andrew’s reference to “deontology“, not a word previously in my vocabulary. It is a good ethics term to pick up. Thank you Immanuel Kant.



The Importance of a 502(d) Order and Attorney Candor

October 27, 2019

I always suggest that attorneys ask for a 502(d) Order under Federal Rules of Evidence before production of ESI. A new case out of Texas demonstrates some of the many bad things that can happen if you do not. Bellamy v. Wal-Mart Stores, Texas, LLC, No. SA-18-CV-60-XR, 2019 WL 3936992 (W.D. Tex. Aug. 19, 2019). The opinion is from one of the leading e-discovery jurists in the country, Texas District Court Judge Xavier Rodriguez. Although he allowed the inadvertently produced documents to be clawed back, it was a close call. In the process Judge Rodriguez considered those documents and sanctioned defendants based on what he read. He struck defendant’s comparative negligence defense and awarded fees and costs. It could have been worse. The accidentally disclosed attorney emails suggested multiple rule violations and a disturbing lack of candor to the court.

This is a must read opinion, not only because of who wrote it, Judge Rodriguez, and the quality of his research and analysis, but also because of the facts of the case. There are many things we can learn from the mistakes highlighted in this opinion. Including the all important ethical values of attorney candor to the court and cooperation.

I will let the learned Judge Rodriguez’ own words in Bellamy explain this case, which was colored by the  attorney conduct he uncovered.

This is a slip and fall case. Plaintiff alleges that she . .  tripped over a pallet while walking through sliding doors into the garden center. . . .

There have been several discovery disputes that have arisen in this case. The Magistrate Judge presided over the first round of disputes and eventually ordered that the Plaintiff’s [First] Motion for Sanctions be dismissed without prejudice to allow for the deposition of a Wal-Mart employee who may have been responsible for leaving the pallet unattended. The Magistrate Judge further ordered that Defendant supplement its disclosures and discovery responses, amend its objections, and provide Plaintiff with a privilege log as to any withheld documents.

This latest round of disputes centers on what happened next. In responding to the Magistrate Judge’s Order, a paralegal in counsel for Defendant’s office inadvertently produced documents that Defendant claims are privileged under the attorney-client privilege or work product. Plaintiff responds that some documents are not privileged. With regard to documents that are privileged, Plaintiff argues that these documents nonetheless demonstrate that Defendant’s counsel has acted in bad faith and engaged in discovery abuse.

Id. at pg. 1 of 7.

Judge Rodriguez starts with an analysis of Evidence Rule 502.

This Court encourages parties to enter into a Rule 502(d) Order[1], which states: “A federal court may order that the privilege or protection is not waived by disclosure connected with the litigation pending before the court.” FED. R. EVID. 502(d). Despite this Court’s encouragement, the Defendant did not request such an Order.[2] This was the first of many mistakes by Defendant’s counsel in this case. In the absence of a 502(d) Order, the Court then turns to an analysis under Rule 502(b).  . . .

In this case the privilege log was woefully deficient. Specifically, the Court is unable to ascertain the identities of various recipients of the emails in question.

Id. at pg 2 of 7.

The emails were all submitted to Judge Rodriguez for review in camera. The opinion makes clear that Judge Rodriguez did not think all of these emails were in fact privileged under case law, but plaintiff’s counsel had for some reason, not explained, conceded that they were.

But as stated above, because Plaintiff concedes that the documents are privileged, the Court will not disturb the concession that the documents are covered by the attorney-client privilege.

Id.

The elements of Rule 502(b) were met with this odd concession, so Judge Rodriguez had no choice but to order their return and prevent plaintiff from using the emails at trial, but Judge Rodriguez was not at all happy about the contents of the emails. This is where the hammer falls:

*3 Accordingly, pursuant to Fed. R. Evid. 502(b) and Fed. R. Civ. P. 26(b)(5)(B), Defendant is entitled to “claw back” the documents it inadvertently produced. But that is not the end of this analysis. Although Plaintiff may not further use these documents in this case, preventing their use in analyzing the pending motion for sanctions would result in a perverse result, upending the rules of civil procedure and encouraging discovery abuse.

Id.

Judge Rodriguez starts by noting defense counsel became aware of key witnesses and failed to disclose them.

With regard to the above individuals, Defendant failed to list them in its Fed. R. Civ. P. 26(a)(1) initial disclosures and failed to timely list them in answers to interrogatories. It is apparent from a reading of the materials submitted either Defendant’s counsel was grossly negligent in fulfilling their discovery obligations or they realized they had an uncooperative manager who was refusing to assist in their investigation, and they did not want to disclose the identities of potentially “bad” witnesses. Counsel for Defendant attempts to shift some of this blame by stating that Plaintiff was already aware of the manager and garden center employee because of her prior employment with Wal-Mart. This shifting is unpersuasive. Defendant’s counsel had obligations to provide this information and it unreasonably and untimely did not.

Id. at pg. 3 of 7.

The in camera privileged emails Judge Rodriguez read also showed that a video of the slip and fall once existed. Yikes. That is a real problem.

Counsel for Defendant never disclosed to Plaintiff’s counsel that at one time video may have existed that was now lost. Rather, counsel merely kept repeating that video does not exist.

Id.

That was way too cute. Disclosure to opposing counsel and the court was expected by Judge Rodriguez.

If that were not all bad enough, the emails revealed another hidden fact:

Finally, Plaintiff’s counsel discovered in the inadvertently produced emails that: (9) Defendant hired an
investigator to conduct a full social media/background check on the Plaintiff on June 20, 2018; and (10)
outside counsel for Defendant notified “Travis Rodmon-Legal” that surveillance had been completed on the Plaintiff and “it is debatable if the footage will be beneficial…. The investigator informs me that she moves very slowly, gingerly and hobbles a bit.”

*4 Counsel for Defendant never disclosed that it possessed video of the Plaintiff. Defendant was under an obligation to disclose any such video as a request for production had been made to that effect. Likewise, Wal-Mart had obtained numerous statements from the Plaintiff prior to her obtaining representation. These statements were requested in requests for production, but not timely disclosed. Counsel for Defendant attributes this failure to the fact that one attorney working this file left the firm and the file was reassigned and the new attorney was unaware of the video’s existence. Although this suggests no “bad faith”, at the time Wal-Mart sent its responses to requests for production and stated that it had no video of the Plaintiff it violated Rule 26(g).

Plaintiff requests that Defendant be sanctioned for failing to disclose that store surveillance video at one point existed and at some point became “lost.” Plaintiff also seeks sanctions because the Wal-Mart manager testified at her deposition that she took multiple photos (including of the pallet) and these photos have never been produced. Likewise, the manager testified that she obtained a statement from the employee who left the pallet unattended and that statement has never been produced. Plaintiff also seeks sanctions because Wal-Mart did not preserve the pallet in question. Finally, Plaintiff requests sanctions generally for Defendant’s failure to honor its discovery obligations. Plaintiff also requests that the Court provide an adverse inference instruction to the jury regarding the missing information. Plaintiff seeks these various sanctions citing generally to Fed. R. Civ. P. 37 and the court’s “inherent
authority.”

Id.

Judge Rodriguez examines the law on sanctions and then considers the ethical Duty of Candor to the Court (Rule 3.3, Model Rules of Professional Conduct) the Duty of Cooperation and Rule 1, FRCP (just, speedy and inexpensive).

D. Duty of Candor, Cooperation and FED. R. CIV. P. 1

Counsel for Defendant wisely opened its Response brief with the following: “Defendant’s counsel
acknowledges and accepts it made mistakes during the discovery of this matter. It accepts that consequences may come from the Court as it considers Plaintiff’s Motions before the Court.”

It is apparent that at the time of the accident, Defendant considered this a low-value or nuisance case. It did not contemplate the severity of the Plaintiff’s injuries and medical treatment. But once Plaintiff placed Defendant on notice that she was going to pursue litigation, reasonable and proportionate preservation obligations were required to be met. Likewise, defense counsel may be on billing constraints, but discovery obligations and adherence to the rules of civil procedure must be met.

*7 Federal Rules of Civil Procedure 1 and 26(f) contemplate that the parties meet in good faith to discuss the case and facilitate resolution of the case and discovery issues because the parties have an obligation “to secure the just, speedy, and inexpensive determination of every action.” Rather than complying with the rules, defense counsel delayed the production of adverse material and the identity of witnesses and the extent of the inappropriate acts only fully became revealed after an inadvertent production of emails was made (after intervention by the Magistrate Judge).

Id. at pgs. 5-6 of 7.

Judge Rodriguez then concludes:

A reading of the file in this case makes apparent that Wal-Mart has known early on that it is responsible for the pallet being left unattended for some period of time in an area frequented by customers. Many counsel for defendants argue that the burden is on a plaintiff to establish all elements of their causes of action. That is true. But if that is going to be the Defendant’s strategy (even when knowing they will likely suffer defeat), this Court is not sympathetic to complaints that litigation is too expensive. In this case, rather than focusing on the extent of Plaintiff’s damages, Wal-Mart has now expended significant time and fees on the liability issue its own claims investigator conceded a long time ago.

Conclusion

Defendant’s Motion to Abate or Strike Plaintiff’s Second Motion for Sanctions (docket no. 49) is DENIED, but as stated above Plaintiff may not use the inadvertently produced documents for any other purpose and counsel must return any documents still in Plaintiff’s possession, if any, to Defendant. Plaintiffs’ Motion for Sanctions (docket no. 50) is GRANTED as stated above. Defendant may not assert any comparative negligence defense in this case, including arguing that the danger was open and obvious.

Id. at pgs. 6-7 of 7.

 


Fraud Is Bad

March 26, 2022

Everybody knows me as an e-Discovery lawyer, but I have other legal-loves, other areas of law I’m returning to now, especially the False Claims Act, aka QUI TAM Whistleblower law. I talked about that recently in Coming Out of the Closet to Share My True Resume. Now that I’m out of Big Law and free to practice in that area again (very few conflicts), I’m building out a new website on Qui Tam. Once a blogger always a blogger. I have a few pages up already, but work is ongoing. Check it out.  It can be reached by all of the following domain names, with Fraud Is Bad the lead. (Yes, I like domain names!)

FraudIsBad.com
FraudIsBad.org
CovidQuiTam.com
CovidQuiTam.law
CovidFalseClaimsAct.com

Please note the all-important Disclaimer Page on my new blog. There are very stringent ethics rules regarding attorney contacts with whistleblowers, primary among them the issue of conflicts. Oddly, I do not see that at other Qui Tam websites, but I am going to err on the side of caution. I am building out an ethical contact system now so all clients are fully protected. Plus, I have a lot more information to assemble and share with everyone. So please check it out and bookmark for later trips.



%d bloggers like this: