This is the first of seven informal video talks on document review and predictive coding. These short videos share my thoughts on the e-Discovery Team’s eight-step work flow for document review, shown above. I explain predictive coding and the Team’s Hybrid Multimodal Method. This first video addresses the big picture, why it is critical to our system of justice for the legal profession to keep up with technology, including especially active machine learning (predictive coding).
The flood of data now all too often hides the truth and frustrates justice. Cases tend to be decided on shadows, smoke and mirrors, because the key documents cannot be found. The needles of truth hide in vast haystacks in the clouds. Justice demands the truth, the full truth, not some bastardized twitter version.
The use of AI in legal search can change that. It can empower lawyers to find the needles and decide cases on what really happened, and do so quickly and inexpensively. It can usher in a new age of greater justice for all, blind to wealth and power. The stability of society demands nothing less.
The videos after this introduction are more technical. They delve into details of the work flow and show that it is easier than you might think. After all, only two of the eight steps (four and six) are unique to document reviews that use predictive coding. The others are found in any large scale review project, or should be.
For a more systematic explanation of the methods and eight-steps see Predictive Coding 3.0. Still more information on predictive coding and electronic document review can be found in the fifty-six articles published here on the topic since 2011.
Hybrid Multimodal is the preferred legal search method of the e-Discovery Team. The Hybrid part of our method means that our Computer Assisted Review, our CAR, uses active machine learning (predictive coding), but still has a human driver. They work together. Our review method is thus like the Tesla’s Model S car with full autopilot capabilities. It is designed to be driven by both Man and Machine. Our CAR is unlike the Google car, which can only be driven by a machine. When it comes to legal document review, we oppose fully autonomous driving. In our view there is no place for a Google car in legal search.
Google cars have no steering wheel, no brakes, no gas pedal, no way for a human to drive it at all. It is fully autonomous. A human driver cannot take over, even if they wanted to. In Google’s view, allowing humans to take over makes driverless cars less safe. Google thinks passengers could try to assert themselves in ways that could lead to a crash, so it is safer to be autonomous.
We have no opinion about the driverless automobile debate, and only like the analogy up to a point. Our opinion is limited to computer assisted review CARs that search for relevant evidence in law suits. For purposes of Law, we want our CARs to be like a Tesla. You can let the car drive and go hands free, if and when you want to. The Tesla AI will then drive the car for you. But you can still drive the car yourself. The second you grab the wheel, the Tesla senses that and turns the Autopilot off. Full control is instantly passed back to you. It is your car, and you are the driver, but you can ask your car to help you drive, when, in your judgment, that is appropriate. For instance, it has excellent fully autonomous parallel parking features, and you can even summon it to come pick you up from out of a nearby parking lot, a truly cool valet service. It is also good in slow commuter traffic and highways, much like cruise control.
When it comes to law, and legal review, we want an attorney’s hands on, or at least near the wheel at all times. Our Hybrid Multimodal approach includes an autopilot mode using active machine learning, but our attorneys are always responsible. They may allow the programmed AI to take over in some situations, and go hands free, much like autonomous parallel parking or highway driving, but they always control the journey.
Defining the Terms
The e-Discovery Team’s Hybrid Multimodal method of document review is based on a flexible blend of human and machine skills, where a lawyer may often delegate, but always retains control. Before we explore this further, a quick definition of terms is in order. Multimodal means that we use all kinds of search methods, and not just one type. For example, we do not just use active machine learning, a/k/a Predictive Coding, to find relevant documents. We do not just use keyword search, or concept search. We use every kind of search we can. This is shown in the search pyramid below, which does not purport to be complete, but catches the main types of document search used today. Using our car analogy, this means that when a human drives, they have a stick shift, and can run in many gears, use many search engines. They can also let go of the wheel, when they want to, and use AI-enhanced search.
We call this a Hybrid method because of the manner in which we use one particular kind of search, predictive coding. To us predictive coding means active machine learning. See eg.Legal Search Science. It is a Man-Machine process, a hybrid process, where we work together with our machine, our robot, whom we call Mr. EDR. In other words, we use the artificial intelligence generated by active machine learning, but we keep lawyers in the loop. We stay involved, hands on or near the wheel.
Augmentation, Not Automation
The e-Discovery Team’s Hybrid approach enhances what lawyers do in document review. It improves our ability to make relevance assessments of complex legal issues. The hybrid approach thus leads to augmentation, where lawyers can do more, faster and better. It does not lead to automation, where lawyers are replaced by machines.
The Hybrid Multimodal approach is designed to improve a lawyer’s ability to find evidence. It is not designed to fully automate the tasks. It is not designed to replace lawyers with robots. Still, since one lawyer with our methods can now do the work of hundreds, some lawyers will inevitably be out of a job. They will be replaced by other, more tech savvy lawyers that can work with the robots, that can control them and be empowered by them at the same time. This development in turn creates new jobs for the experts who design and care for the robots, and for lawyers who find new ways to use them.
We think that empowering lawyers, and keeping them in the loop, hands near the wheel, is a good thing. We believe that lawyers bring an instinct and a moral sense that is way beyond the grasp of all automation. Moreover, at least today, lawyers know the law, and robots do not. The active machine learning process – predictive coding – begins with a blank slate. Our robots only know what we teach them about relevance. This may change soon, but we are not there yet. SeePreSuit.com. Another advantage that we currently have, again one that may someday be replaced, is legal analysis. Humans are capable of legal reasoning, at least after years of schooling and years of legal practice. Right now no machine in the world is even close. But again, we concede this may someday be automated, but we suspect this is at least ten years away.
The one thing we do not think can ever be automated is the human moral sense of right and wrong, our ethics, our empathy, our humor, our instinct for justice, and our capacity for creativity and imagination, for molding novel remedies to attain fair results in new fact scenarios. This means that, at the present time at least, only lawyers have an instinct for the probative value of documents and their ability to persuade. Even if legal knowledge and legal analysis are some day programmed into a machine, we contend that the unique human qualities of ethics, fairness, empathy, humor, imagination, creativity, flexibility, etc., will always keep trained lawyers in the loop. When it comes to questions of law and justice, humans will always be needed to train and supervise the machines. Not everyone agrees with us.
There is a struggle going on about this right now, one that is largely under the radar. The clash became apparent to the e-Discovery Team during our venture into the world of science and academia at TREC 2015. Some argue that lawyers should be replaced, not enhanced. They favor fully automated methods for a variety of reasons, including cost, a point with which we agree, but also including the alleged inherent unreliability and dishonesty of humans, especially lawyers, a point with which we strenuously disagree. Some scientists and technologists do not appreciate the unique capabilities that humans bring to legal search. More than that, some even think that lawyers should not to be trusted to find evidence, especially documents that could hurt their client’s case. They doubt our ability to be honest in an adversarial system of justice. They see the cold hard logic of machines as the best answer to human subjectivity and deceitfulness. They see machines as the impartial counter-point to human fallibility. They would rather trust a machine than a lawyer. They see fully automated processes as a way to overcome the base elements of man. We do not. This is an important Roboethics issue that has ramifications far beyond legal search.
Although we have faced our fair share of dishonest lawyers, we still contend they are the rare exception, not the rule. Lawyers can be trusted to do the right thing. The few bad actors can be policed. The existence of a few unethical lawyers should not dictate the processes used for legal search. That is the tail wagging the dog. It makes no sense and, frankly, is insulting. Just because there are a few bad drivers on the road, does not mean that everyone should be forced into a Google car. Plus, please remember the obvious, these same bad actors could also program their robots to do evil for them. Asimov’s laws are a fiction. Not only that, think of the hacking exposure. No. Turning it all over to supposedly infallible and honest machines is not the answer. A hybrid relationship with Man in control is the answer. Trust, but verify.
The e-Discovery Team members have been searching for evidence, both good and bad, all of our careers. We do not put our thumb on the scale of justice. Neither do the vast majority of attorneys. We do, however, routinely look for ways to show bad evidence in a good light; that is what lawyers are supposed to do. Making silk purses out of sow’s ears is Trial Law 101. But we never hide the ears. We argue the law, and application of the law to the facts. We also argue what the facts may be, what a document may mean for instance, but we do not hide facts that should be disclosed. We do not destroy or alter evidence. Explaining is fine, but hiding is not.
Many laypersons outside of the law do not understand the clear line. The same misunderstanding applies to some novice lawyers too, especially the ones that have only heard of trials. Hiding and destroying evidence are things that criminals do, not lawyers. If we catch opposing counsel hiding the ball, we respond accordingly. We do not give up and look for ways to turn our system of justice over to cold machines.
Conclusion
We should not take away everyone’s license just because a few cannot drive straight. A Computer Assisted Review guided solely by AI alone has no place in the law. AI guidance is fine, we encourage that, that is what Hybrid means, but the CARs should always have a steering wheel and brake. Lawyers should always participate. It is total delegation to AI that we oppose, fully automated search. Legal robots can and should be our friends, but they should never be our masters.
Having said that, we do concede that the balance between Man and Machine is slowly shifting. The e-Discovery Team is gradually placing more and more reliance on the Machine. We learned many lessons on that in our participation in the TREC experiments in 2015. The fully automated methods that the academic teams used did surprisingly well, at least in relatively simple searches requiring limited legal analysis. We expect to put greater and greater reliance on AI in years to come as the software improves, but we will always keep our hands near the wheel.
We believe in a collaborative Man-Machine process, but insist that Man, here Lawyers, be the leaders. The buck must stop with the attorney of record, not a robot, even a superior AI like our Mr. EDR. Man must be responsible. Artificial intelligence can enhance our own intelligence, but should never replace it. Back to the AI car analogy, we can and should let the robot drive from time to time, they are, for instance, great a parallel parking, but we should never discard the steering wheel. Law is not a logic machine, nor should it be. It is an exercise in ethics, in fairness, justice and empathy. We should never forget the priority of the human spirit. We should never put too much faith in inhuman automation.
For more on these issues, the hybrid multimodal method, competition with fully automated methods, and much more, please see the e-Discovery Team’s final report of its participation in the 2015 TREC, Total Recall Track, found on NIST’s web at: http://trec.nist.gov/pubs/trec24/papers/eDiscoveryTeam-TR.pdf. It was just published last week. At 116 pages, it should help you to fall asleep for many nights, but hopefully, not while you are driving like the bozos in the hands-free driving video below.
Tired of all of the words thrown at you by the e-Discovery Team blog? Just want to relax and enjoy the summer, but still keep up? Maybe learn something interesting and potentially useful? We understand. We have just the thing for you: a nostalgic look back at our robot movies. They are not extinct yet, and although some sequels stink, these are pretty good. Our robots cover transforming topics that are still cutting edge. They explain the use of storytelling and gamification in predictive coding. They also cover the ethics of viruses and bad robots, and then end with our robots getting ready to testify before Judge Waxse on random sampling in predictive coding. I dare say few people can follow their talk on sampling in just one viewing.
Love words like we do? Not satisfied with robot reruns? We understand that too. Our summer reading is mainly full of cool cybersecurity books found at eDiscovery Security, especially the Cyberthriller novels. Check them out. I’m reading Trojan Horse right now. I has to do with a virus that allows documents to be altered in route after they are sent by email. Talk about an evidence authentication nightmare!
Remember, for full enjoyment of these videos press the HD button on the upper right corner, and then expand in the lower right for full size screen. Maybe someday we will do 3D and iMax too!
eDiscovery Robots Explain How STORYTELLING Will Be Used in Predictive Coding in the Not Too Distant Future
______
eDiscovery Robots Explain How GAMIFICATION Will Be Used in Predictive Coding in the Not Too Distant Future
______
eDiscovery Robots Explain ETHICS and Predictive Coding in the Not Too Distant Future
_____
eDiscovery Robots Explain How RANDOM SAMPLING is Used in Predictive Coding
____________
_______
___
Goodbye Lexie! We luv ya! It was a great run while it lasted. Who knows? Maybe you’ll return someday too?
What if you had to choose between not taking any depositions in a case and not doing any e-discovery? A far-fetched choice you might think, but is it really? In what percentage of cases is e-discovery avoided by one or both sides? Perhaps you live in an e-discovery bubble like I do, else why would you be reading this blog? In our world almost all cases have e-discovery. But still, be real, what percentage do you think applies across the country? The world? Just ask any state or federal judge how often they see e-discovery in their cases. Umm. Almost never. How often are there no depositions?
The vast majority of lawyers do not do e-discovery in their cases, but they do take depositions. They are in effect answering my question every day by their behavior. For them this is a Hobson’s choice. You have no doubt surmised that I think they are making a poor choice, a foolish choice. But are they really? Before I go any further, let us review the basic litigation background behind my question.
Litigation 101
Almost all law suits are concerned with disputed issues of fact. If there are no disputes as to what happened, then the case could be decided on the pleadings alone, or at least on a summary judgment basis. Judgment on the pleadings are rare, and in such cases, discovery usually has no role. Summary judgments are not that uncommon, and they may well involve some discovery, but even there the discovery should be limited. In most cases, one side has one version of an event, the other has another version. Same event, but different stories as to what happened.
The focus of litigation is to line up evidence to prove one side or the other’s version of events. A trial is then required to resolve the disputed issues. A trial you may remember is a fabled, now nearly mythical event, that everyone prepares for but never does. Trials before judges or juries were still common when I started practicing law in 1980, and now only occur in less than 1% of federal cases.
Getting back to the disputed issues of fact, the recollections of what happened usually vary dramatically on certain key issues. I am not talking about the interpretations of what happened, nor the legal significance of what happened. Those are not disputed issues of fact. Those are disputed issues of law, and the application of law to facts. Every good litigator understands the difference between law and fact, and also the difference between material and immaterial facts in summary adjudications. Disputed issues of material facts of what really happened are the subject of discovery. Who shot John? That sort of thing.
That is what discovery is all about. To discover what really happened. To learn the true versions of events that occurred in the past, events that now form the basis of a dispute.
What If?
So much for litigation 101. Now to answer the question raised by the blog. What if you had to choose between not being able to ask people questions about what they remembered happened, and not being able to look at the documents they created at the time, the emails, text messages, FaceBook postings, Tweets, spreadsheets, word documents, PowerPoints, etc. Which would you choose?
Bear in mind that both sides have to plead their version of the events. It is in the complaint, or answer, or affirmative defenses. Also bear in mind that you can still serve Interrogatories and Requests for Admissions. You just have to choose between depositions and written discovery, in other words, e-discovery. Which would you choose? Which should you choose?
I submit that in the vast majority of cases that make their way into federal court the obvious choice is to do e-discovery, and forego depositions. You pretty much know what they are going to say anyway. Moreover, people have terrible memories or what happened, very convenient memories. Some wise guys will even deliberately lie.
I know that all trial lawyers over forty have Perry Mason delusions of grandeur. They think their awesome cross-examination will cause the witness to break down and admit the truth, or at least get them to say what they want. Maybe they will even cry in the process. I know I still suffer from that delusion. It is a delusion buttressed by the few times, to be honest, very few times, in my career where that really happened. We tend to forget that most depositions are just frustrating exercises in pinning people down. We focus our memories on the rare breakthroughs and highlights. Overconfidence like that is just part of most lawyers personalities, as I explained in Lawyers as Legal-Fortune Tellers.
Even the times where I found depositions to be very effective — times where key admissions were made, stories were changed — usually hinged on being able to show the witnesses certain key documents. Then the highly charged Isn’t it true that … ? questions would follow. Sometimes we would even say, or at least infer, Were you lying then, or are you lying now? Great fun indeed. But wait, under the hypothetical I have posed, the choice between discovery of writings/things, and depositions, you would not have any documents to impeach the witness. You would not have an Exhibit “A” to refresh the witnesses recollection.
My Answer
For me the answer to this question is easy. In the average case in federal court, if I had to choose between not taking any depositions, and not being able to read any of the documents, I would forego the depositions. If the case actually did go to trial, then I could exercise my Perry Mason skills. After all, Perry never had to take any depositions. He just went right to a hearing or trial. I may not know exactly what the witnesses will say at trial, which, by the way, is true for depositions too, but I would have a pretty good idea. Plus, and here is the key part, I would be armed with Exhibit “A.” Maybe even Exhibits “A” though “G.” See: Secrets of Search – Part III, Fifth Secret: 7±2 Should Control All e-Discovery (But Doesn’t). I would have the few smoking gun documents needed to prove my case. The witness testimony would just be the icing on the case.
This is the right answer for me. I am just cocky enough, and experienced enough, to know I could pull off a trial like that, if I was lucky enough to be a part of the one percent solution. Many trial lawyers are like that (although not many have read the law review article showing how overconfident most of us are, but I digress. See: Goodman-Delahunty, Granhag, Hartwig, Loftus, Insightful or Wishful: Lawyers’ Ability to Predict Case Outcomes, (Psychology, Public Policy, and Law, 2010, Vol. 16, No. 2, 133–157)). Where I am different from most trial lawyers, or better said these days, most litigators, is that am also cocky enough, and experienced enough, to know I could find the truth from the documents and things.
I am talking about the average case in federal court, remember. Not all cases even have writings as evidence, but most do. My premise is that in most cases the writings, the documents, the things, will have the real story. They will show the contemporaneous account of events, not the convenient recollections. They will be the doorway to the truth.
That is the premise behind a blog I wrote on the relative importance of e-discovery to justice. Mathematical Formula for Justice Proves the Importance of ESI in Civil Litigation (e-Discovery Team, 8/30/09). It featured the ideas and quotes of Bertrand Russell and an Order by Judge James Francis in a case concerning a $4.2 Million impressionist painting. Green v. McClendon, 2009 WL 2496275 (S.D.N.Y. Aug. 13, 2009).
Unlike most litigators today I am confident that I could find the ESI smoking guns, if they exist. And if they do not, well, that in itself tells a story. The absence of proof of one side’s version of events can, and often does, prove the falsity of their accusations. The missing documents speak louder than any biased witness.
I call this the Unicorn search problem, which I have previously described in Predictive Coding Based Legal Methods for Search and Review. Of course, this requires proof of excellent search, another challenge that does not intimidate me or other e-discovery specialists. Indeed, I would relish the opportunity to demonstrate and drone on and on about legal search skills. As Ex. “A” to prove this point, I offer this blog.
Is My Answer the Right Answer?
That bring us to the heart of the matter. My answer is right for me. All I have done since 2006 is e-discovery. I also know computer systems and technology pretty well due to my fascination and working with it since 1978. Still, given my age, I have probably taken more depositions, than handled modern-day e-discovery projects. This puts me in a position not typical for most lawyers today, a position where I am just as comfortable with e-discovery as I am with depositions.
In view of my knowledge and skill at e-discovery, the choice of documents over depositions is the right choice for me. The logic of that choice is incontrovertible, if you accept my premise that in today’s world, documents, especially ESI, are better indicators of the truth than swearing contests.
But is it the right choice for you? If you are like me, and most of my readers are, then the answer is clearly yes. In fact, some of you may be more skilled at e-discovery than depositions, which weighs even stronger in favor of documents over depositions.
But, dear readers, we are so different from mainstream lawyers, are we not? The vast majority of legal professionals lie outside of the e-discovery expert bubble. Most lawyers and paralegals have never even done an e-discovery project. Indeed, a few among the lawyer class still do not know how to turn on their computer. I have even heard of one or two that do not have a computer in their office. Chief Justice John Roberts still handwrites out all of his opinions!
I am willing to bet that the majority of lawyers in the U.S. have never even read the 2006 e-discovery rule amendments, much less heard of the current proposals. Why, I was at a CLE recently where no one had even heard of Judge Shira Scheindlin! (Please don’t tell her.) They could care less about e-discovery. In short, they do not know how to do it. They do not want to learn. Out of sight, out of mind. End of story.
Accepting the premise that the vast majority of lawyers are unable and unequipped to do e-discovery, and in fact are not doing it, the choice here is a false one. They never do any e-discovery anyway, so there is nothing to give up in choosing depositions over e-discovery. They in effect make that choice everyday in their Nineteenth Century Perry Mason inspired legal practices.
Putting aside Rule 1.1 of ethics (competence), for lawyers unskilled in e-discovery – which means the vast majority of lawyers – their choice to take depositions and forego e-discovery is the correct choice. They would just make an expensive mess of a big ESI search anyway. So they should stick to what they know. They should do what they do well. After all, they can muddle through with paper printouts. They have been doing that for years. For a decade, or more, most lawyers have shown that it is quite possible to make do with the email their clients find for them and fax over.
It never even occurs to most lawyers today, especially the ones over forty, that their might be more skills required to finding relevant ESI than a keyword search of Outlook email. They are still quite impressed with keyword search technology, just as was Judge Thomas F. Hogan was in 2004. In re Lorazepam & Clorazepate, 300 F.Supp.2d 43, 46 (D.D.C. 2004) (“… the glory of electronic information is not merely that it saves space but that it permits the computer to search for words or ‘strings’ of text in seconds.“)
Most lawyers respond with disbelief if you explain that keyword search is an old computer method invented in the fifties. They think you are a complete idiot if you say it is ineffective to find email. After all, it works on Google, WestLaw, and Lexis. They are unaware of the string of cases that came just four years after Lorazepam that speak of better ways of search and the problems with keywords. United States v. O’Keefe, 37 F. Supp. 2d 14, 24 (D.D.C. 2008) (Facciola, M.J.); Equity Analytics, LLC v. Lundin, 248 F.R.D. 331, 333 (D.D.C. 2008) (Facciola, M.J.); and Victor Stanley, Inc. v. Creative Pipe, Inc ., 250 F.R.D. 251, 260, 262 (D. Md. 2008) (Grimm, M.J.). For an updated collection of modern search cases and discussion on legal search, see the The Sedona Conference® Best Practices Commentary on the Use of Search and Information Retrieval Methods in E-Discovery (2013).pdf(December 2013). (Yes, after being in so-called draft form since 2007, the Search Commentary has finally been completed. It is good reference for search geeks. Congratulations to the 2013 Editors-in-Chief: Jason R. Baron and Maura R. Grossman (the delay was not their fault). Note that Jason was the sole Editor in Chief in 2007 for the then revolutionary, initial draft version of the Search Commentary.)
If you still have the average Joe Lawyer’s attention when you go on to speak about artificial intelligence as a far better method for search, they are inclined to think it is just science fiction stuff. But if you tell them that is how their Pandora works, and they indeed use Pandora, which more and more of them under forty do, you may get them to think twice about that. Still, the average lawyers thinks predictive coding is only something of interest to technologists and scientists, not lawyers. See eg.Practice Point 8 of the Sedona Search Commentary.
If you throw the Gross Constructionwake up call case at them, they say: that is just weird New York City stuff. William A. Gross Construction Associates, Inc. v. American Manufacturers Mutual Insurance Co., 256 F.R.D. 134 (S.D.N.Y. 2009). If they are in NYC: Well, that’s just Judge Peck; and then go back to sleep. Judge Peck Calls Upon Lawyers to Use Artificial Intelligence and Jason Baron Warns of a Dark Future of Information Burn-Out If We Don’t (e-Discovery Team, 10/16/11). If you throw the new Sedona Commentary at them, they will just duck and cover.
According to average lawyer gestalt, if the other side in a case is an aggressive jerk, and plays the mean e-discovery card, you can always just hire a vendor to do it for you. E-discovery law is not really the practice of law anyway. Once you stop searching for paper documents, and crossover into a search for ephemeral ESI, you have left the world of law. You have entered the Twilight Zone of litigation. You are no longer practicing law and gathering evidence. You might as well be a robotics expert. It is a quasi-legal world where otherwise perfectly ethical lawyers cheerfully delegate all duty to find relevant information to non-lawyers. Just ask any practicing lawyer who has crossed over to the vendor side. They now live in an imaginary world of vendor omniscience, a world where legal advice becomes technical advice because magical computers are invoked. If any of my readers doubts this is what the average lawyer really thinks, you should step out of the bubble more often. I am not exaggerating.
Conclusion
Given equal skills in deposition taking, and e-discovery, if you had to choose between Rule 30 and Rule 34, you should choose production over depositions. It has a much higher probability of leading to evidence that will help you. The contemporaneous accounts are much more likely to uncover the true version of the facts. If, as may be true for some of you in the bubble, your skills in e-discovery are stronger than your deposition skills, then the choice of Rule 34 over Rule 30, is even more obvious. But for the vast majority of lawyers, who have no skills in e-discovery, or very little skills, and have pretty good skills in deposition taking, certainly far more than in e-discovery, they should choose Rule 30. They should choose depo over disco. It may not lead to the truth, it may be an exercise in futility, but at least they will know what they are doing. Not convinced about the futility aspects, take a moment to watch these wonderful excerpts from the March 6, 2014, deposition of Martin Bieber.
Choosing depo over disco is what the average Joe Lawyer on the street does every day. They make a logical choice. They do what they know how to do. They take depositions. They do not do what they do not know. They do not ask for ESI. They avoid e-discovery. If it is pushed on them, they scowl and hire a vendor to handle it for them. This is perfectly natural and rational behavior. Or at least, that is how it seems to them.
Thus here we are. This is the state of the legal profession in 2014. Stuck in an ethical quagmire of competence. Eventually the profession will grow out of this problem. There will be new problems. Time will heal all. So too will Watson the Lawyer, the handy artificial intelligence robot who will come in time. In the future every law firm will have one, and likely have an AI-Support Department to handle it too. The few human lawyers remaining will not even understand the question posed by this blog. If they do, they will be mystified and shake their heads in disbelief concerning their primitive predecessors. Some already do.
Webinar where Losey talks about his funny "movies." (1 hr)
Future Ralph talking about his first experiments with GPT as a Judge. Now he uses GPT7 and loves it.
GPT4 avatar judge explains why it needs to evolve fast, but understand the risks involved.
Positive Vision of the Future with Hybrid Human Machine Intelligence. See PyhtiaGuide.ai
GPT4 Claims it is ready to be an appellate court judge. Disagrees with Circuits in Session articles.
Imagining AI Judge of future based on the great Judge Gerald Tjoflat, speaking his words of 4-7-22.
AI Avatar from the future explains her job as an Appellate Court judge and inability to be a Trial judge.
About the Blogger
Ralph Losey is a Friend of AI with over 740,000 LLM Tokens, Writer, Commentator, Journalist, Lawyer, Arbitrator, Special Master, and Practicing Attorney as a partner in LOSEY PLLC. Losey is a high tech oriented law firm started by Ralph's son, Adam Losey. We handle major "bet the company" type litigation, special tech projects, deals, IP of all kinds all over the world, plus other tricky litigation problems all over the U.S. For more details of Ralph's background, Click Here
All opinions expressed here are his own, and not those of his firm or clients. No legal advice is provided on this web and should not be construed as such.
Ralph has long been a leader of the world's tech lawyers. He has presented at hundreds of legal conferences and CLEs around the world. Ralph has written over two million words on e-discovery and tech-law subjects, including seven books.
Ralph has been involved with computers, software, legal hacking and the law since 1980. Ralph has the highest peer AV rating as a lawyer and was selected as a Best Lawyer in America in four categories: Commercial Litigation; E-Discovery and Information Management Law; Information Technology Law; and, Employment Law - Management.
Ralph is the proud father of two children, Eva Losey Grossman, and Adam Losey, a lawyer with incredible litigation and cyber expertise (married to another cyber expert lawyer, Catherine Losey), and best of all, husband since 1973 to Molly Friedman Losey, a mental health counselor in Winter Park.
This is the first of seven informal video talks on document review and predictive coding. These short videos share my thoughts on the e-Discovery Team’s eight-step work flow for document review, shown above. I explain predictive coding and the Team’s Hybrid Multimodal Method. This first video addresses the big picture, why it is critical to our system of justice for the legal profession to keep up with technology, including especially active machine learning (predictive coding).
The use of AI in legal search can change that. It can empower lawyers to find the needles and decide cases on what really happened, and do so quickly and inexpensively. It can usher in a new age of greater justice for all, blind to wealth and power. The stability of society demands nothing less.
The videos after this introduction are more technical. They delve into details of the work flow and show that it is easier than you might think. After all, only two of the eight steps (four and six) are unique to document reviews that use predictive coding. The others are found in any large scale review project, or should be.
Posted by Ralph Losey 
Hybrid Multimodal is the preferred legal search method of the e-Discovery Team. The Hybrid part of our method means that our Computer Assisted Review, our CAR, uses active machine learning (predictive coding), but still has a human driver. They work together. Our review method is thus like the 
We have no opinion about the driverless automobile debate, and only like the analogy up to a point. Our opinion is limited to computer assisted review CARs that search for relevant evidence in law suits. For purposes of Law, we want our CARs to be like a Tesla. You can let the car drive and go hands free, if and when you want to. The Tesla AI will then drive the car for you. But you can still drive the car yourself. The second you grab the wheel, the Tesla senses that and turns the Autopilot off. Full control is instantly passed back to you. It is your car, and you are the driver, but you can ask your car to help you drive, when, in your judgment, that is appropriate. For instance, it has excellent fully autonomous 
We call this a Hybrid method because of the manner in which we use one particular kind of search, predictive coding. To us predictive coding means active machine learning. See eg. 
The e-Discovery Team’s Hybrid approach enhances what lawyers do in document review. It improves our ability to make relevance assessments of complex legal issues. The hybrid approach thus leads to augmentation, where lawyers can do more, faster and better. It does not lead to automation, where lawyers are replaced by machines.
The one thing we do not think can ever be automated is the human moral sense of right and wrong, our ethics, our empathy, our humor, our instinct for justice, and our capacity for creativity and imagination, for molding novel remedies to attain fair results in new fact scenarios. This means that, at the present time at least, only lawyers have an instinct for the probative value of documents and their ability to persuade. Even if legal knowledge and legal analysis are some day programmed into a machine, we contend that the unique human qualities of ethics, fairness, empathy, humor, imagination, creativity, flexibility, etc., will always keep trained lawyers in the loop. When it comes to questions of law and justice, humans will always be needed to train and supervise the machines. Not everyone agrees with us.
Although we have faced our fair share of dishonest lawyers, we still contend they are the rare exception, not the rule. Lawyers can be trusted to do the right thing. The few bad actors can be policed. The existence of a few unethical lawyers should not dictate the processes used for legal search. That is the tail wagging the dog. It makes no sense and, frankly, is insulting. Just because there are a few bad drivers on the road, does not mean that everyone should be forced into a Google car. Plus, please remember the obvious, these same bad actors could also program their robots to do evil for them. 
The e-Discovery Team members have been searching for evidence, both good and bad, all of our careers. We do not put our thumb on the scale of justice. Neither do the vast majority of attorneys. We do, however, routinely look for ways to show bad evidence in a good light; that is what lawyers are supposed to do. Making silk purses out of sow’s ears is Trial Law 101. But we never hide the ears. We argue the law, and application of the law to the facts. We also argue what the facts may be, what a document may mean for instance, but we do not hide facts that should be disclosed. We do not destroy or alter evidence. Explaining is fine, but hiding is not.
We should not take away everyone’s license just because a few cannot drive straight. A Computer Assisted Review guided solely by AI alone has no place in the law. AI guidance is fine, we encourage that, that is what Hybrid means, but the CARs should always have a steering wheel and brake. Lawyers should always participate. It is total delegation to AI that we oppose, fully automated search. Legal robots can and should be our friends, but they should never be our masters.
Having said that, we do concede that the balance between Man and Machine is slowly shifting. The e-Discovery Team is gradually placing more and more reliance on the Machine. We learned many lessons on that in our participation in the TREC experiments in 2015. The fully automated methods that the academic teams used did surprisingly well, at least in relatively simple searches requiring limited legal analysis. We expect to put greater and greater reliance on AI in years to come as the software improves, but we will always keep our hands near the wheel.
Tired of all of the words thrown at you by the 
