I’ve escaped the e-Discovery Niche After 15 Years of Super-Specialization
Ralph Losey, January 25, 2022
After fifteen years of writing weekly blogs on e-discovery, I took three years off to focus on implementation of all those words. Now I’m back, back to where I once belonged. Writing again, but writing not just about my Big Law niche, the fun little AI corner that I had painted myself into, but back to writing about ALL of my interests in Law and Technology. That has been my home since I started in legal practice in 1980 and at the same time started coding, mostly games, but also music software, midi creations and law office technology. Proud to recall that I was one of the first computer lawyers in the country. (Also one of the first to get in trouble with the Bar for my Internet Website, FloridaLawFirm.com, which they thought was a television broadcast!)
Ralph in the early 90s
Anyway, when not haggling with the Bar and fellow attorneys who would tease me, the first nerd, and call me a “secretary” (ooh how terrible) for having a keyboard on my desk. I kid you not! I used PCs when they first came out in my law firm as the new associate. I have had them on my desk to try to work smarter ever since. Not PCs necessarily, but all kinds.
So I’m back to where I once belonged, in the great big world of technology law, making deals and giving advice. Oh yeah, I may still consult on e-discovery too, especially the AI parts that so fascinated me ever since my Da Silva Moore breakthrough days. (Thank you Judge Andrew Peck.) For my full story, some of which I had to hide in my Big Law role as a super-specialist, see: https://www.losey.law/our-people/25-uncategorized/108-ralph-losey Not many people know I was a Qui Tam lawyer too; and for both sides.
Wait, there is still more. I’ve left the best for last. I went back home, left Big Law for good, and am now practicing law with my son, Adam Losey, daughter in law, Cat Losey, and thirteen other, crazy tech lawyer types at Losey.law. Yes, that is the real domain name and the name of the firm itself is Losey. So of course I had to go there. Check it out. Practicing law with my son is a dream come true for both of us. I’m loving it. It was lonely being the only tech wiz in a giant firm. Adam knows tech better than me, is much faster in every respect (except maybe doc review with AI) and he and Cat are obviously a lot smarter.
To my long-time readers, thanks for your encouragement. I heard you and got back to my roots of general tech-law, and got back to blogging and home. To quote the Beatles the funny “Get Back” song in their great LET IT BE album:
Rosetta (who are you talking about?) about Sweet Loretta Fart. . . .
Stay tuned, because a new blog is coming at you soon. Feel free to drop me an email at Ralph at Losey dot Law. Humans only please. Robots not welcome (unless you’re from the future and don’t have weapons).
It’s Mueller Time! I predict we will be hearing this call around the world for decades, including boardrooms. Organizations will decide to investigate themselves on sensitive issues before the government does, or before someone sues them and triggers formal discovery. Not always, but sometimes, they will do so by appointing their own independent counsel to check on concerns. The Boards of tomorrow will not look the other way. If Robert Muller himself later showed up at their door, they would be ready. They would thank their G.C. that they had already cleaned house.
Most companies who decide it is Mueller Time, will probably not investigate themselves in the traditional “full calorie” Robert Muller way, as good as that is. Instead, they will order a less expensive, AI based investigation, a Mueller Lite. The “full calorie” traditional legal investigation is very expensive, slow and leaky. It involves many people and linear document review. The AI Assisted alternative, the Mueller Lite, will be more attractive because of its lower cost. It will still be an independent investigation, but will rely primarily on internal data and artificial intelligence, not expensive attorneys.
I call this E-Vestigations, for electronic investigations. It is a new type of legal service made possible by a specialized type of AI called “Predictive Coding” and newly perfected Hybrid Multimodal methods of machine training.
Mueller Lite E-Vestigations Save Money
Robert Mueller investigations typically cost millions and involves large teams of expensive professionals. AI Assisted investigations are cheap by comparison. That is because they emphasize company data and AI search of the data, mostly the communications, and need so few people to carry out. This new kind of investigation allows a company to quietly look into and take care of its own problems. The cost savings from litigation avoidance, and bad publicity, can be compelling. Plus it is the right thing to do..
E-Vestigations will typically be a quarter the cost of a traditional Mueller style, paper investigations. It may even be far less than that. Project fees depend on the data itself (volume and “messiness”) and the “information need” of the client (simple or complex). The competitive pricing of the new service is one reason I predict it will explode in popularity. This kind of dramatic savings is possible because most of the time consuming relevance sorting and document ranking work is delegated to the AI.
The computer “reads” or reviews at nearly the speed of light and is 100% consistent. But it has no knowledge on its own. An idiot savant. The AI cannot do anything without its human handlers and trainers. It is basically a learning machine designed to sort large collections of texts into binary sets, typically relevant or irrelevant.
The human investigators read much slower and sometimes make mistakes (plus they like to get compensated), but they are absolutely indispensable. Someday the team of humans may get even smaller, but we are already down to around seven or fewer people per investigation. Compare that to the hundreds involved in a traditional Muller style document review.
Proactive “Peace of Mind” Investigations
This new legal service allows concerned management to proactively investigate upon the first indications of possible wrong-doing. It allows you to have greater assurance that you really know what is going on in your organization. Management or the Board then retains an independent team of legal experts to conduct the quick E-Vestigation. The team provides subject matter expertise on the suspected problem and uses active machine learning to quickly search and analyze the data. They search for preliminary indications of what happened, if anything. This kind of search is ideal for sensitive legal inquiries. It gives management the information needed without breaking the bank or publicizing the results.
This New Legal Service Is Built Around AI
E-Vestigations are a pre-litigation legal service that relies heavily on artificial intelligence, but not entirely. Investigations like this are very complex. They are nowhere near a fully automated process, and as mentioned the AI is really just a learning machine that knows nothing except how to learn document relevance. The service still needs legal experts, but a much smaller team
AI assisted investigations such as E-Vestigations have five compelling positive traits:
Cost
Speed
Stealthiness
Confidentiality
Accuracy.
This article introduces the new service, discusses these five positive traits and provides background for my prediction that many organizations will order AI assisted investigations in the coming years. In fact, due to the disappearing trial, I predict that E-Vestigations will someday take the lead from Litigation in many law firms. This prediction of the future, like most, requires a preliminary journey into the past, to see the longer causal line of events. That comes next, but feel to skip the next three sections until you come to the heading, What is an E-Vestigation?
King Litigation Is Dead
The glory days of litigation are over. All trial lawyers who, like me, have lived through the last forty years of legal practice, have seen it change dramatically. Litigation has moved from a trial and discovery practice, where we saw each other daily in court, to a discovery, motion and mediation practice where we communicate by email and occasional calls.
Although some “trial dogs” will not admit it, we all know that the role of trials has greatly diminished in modern practice. Everything settles. Ninety-nine percent (99%) of federal court civil cases settle without trial. Although my current firm is a large specialty practice, and so is an exception, in most law firms trials are very rare. A so-called “Trial Practice” of a major firm could go years without having an actual trial. I have seen it happen in many law firms. Good lawyers for sure, but they do not really “do trials,” they do trial preparation.
For example, when I started practicing law in 1980 “dispute resolution” was king in most law firms. It was called the “Litigation Department” and usually attracted the top legal talent. It brought in strong revenue and big clients. Every case in the top firms was either a “Bet the Farm” type, or a little case for kiddie lawyer training, we had no form-practice. Friedmann & Brown, “Bet the Farm” Versus “Law Factory”: Which One Works?(Geeks and Law, 2011).
The opposite, “Commodity Litigation,” was rare; typically just something for some divorce lawyers, PI lawyers, criminal lawyers and bankruptcy lawyers. These were not the desired specialties in the eighties, to put it mildly. Factory like practices like that did not pay that well (honest ones anyway) and were boring to most graduates of decent law schools. This has not changed much until recently, when AI has made certain Commodity practices far more interesting and desirable. SeeJoshua Kubicki, The Emerging Competitive Frontier in Biglaw is Practice Venturing (Medium, 1/24/19).
Aside from the less desirable Commodity practice law firms, most litigators in the eighties would routinely take a case to trial. Fish or Cut Bait was a popular saying. Back then Mediation was virtually unknown. Although a majority of cases did eventually settle, a large minority did not. That meant physically going to court, wearing suits and ties every day, and verbal sparing. Lots of arguments and talk about evidence. Sometimes it meant some bullying and physical pushing too, if truth be told. It was a rough and tumble legal world in the eighties, made up in many parts of the U.S. almost entirely of white men. Many were smokers, including the all-white bench.
Ah, the memories. Some of the Litigation attorneys were real jerks, to put it mildly. But only a few were suspected crooked and could not be trusted. Most were honest and could be. We policed our own and word got around about a bad apple pretty fast. Their careers in town were then soon over, one way or the other. Many would just move away or, if they had roots, become businessmen. There were trials a plenty in both the criminal and civil sides.The trials could be dramatic spectacles. The big cases were intense.
Emergence of Mediation
But the times were a changing. In the nineties and first decade of the 21st Century, trials quickly disappeared. Instead, Mediation started to take over. I know, I was in the first group of lawyers ever to be certified as a Mediator of High Technology disputes in 1989. All types of cases began to settle earlier and with less preparation. I have seen cases settle at Mediation where none of the attorneys knew the facts. They just knew what their clients told them. Even more often, only one side was prepared a knew the facts. The other was just “shooting from the hip.”
At trial the unprepared were quickly demolished by the facts, the evidence. At Mediation you can get away with it. The evidence is often just one side’s contention. Why bother to learn the record when you can just BS your way through a mediation? The truth is what I say it is, nothing more. There is no cross-exam. Mediation is a “liars heaven,” although a good mediator can plow through that.
What happened to all the Trial Lawyers you might ask? Many became Mediators, including several of my good friends. A few started specializing in Mediation advocacy, where psychodrama and math are king (typically division). Mediation has become the everyday “Commodity” practice and trials are now the “Bet the Farm” rarity.
With less than one-percent of federal cases going to trial, it is a complete misnomer to keep calling ourselves Trial Lawyers. I know I have stopped calling myself that. Like it or not, that is reality. Our current system is designed to settle. It has become a relativistic opinion fest. It is not designed to determine final, binding objective truth. It is not designed to make findings of fact. It is instead designed to mediate ever more ingenious ways to split the baby.We no longer focus on the evidence, on the objective truth of what happened. We have lost our way.
Justice without Truth is Destabilizing
Justice without Truth is a mockery of Justice, a Post-Modern mockery at that, one where everything is relative. This is called Subjectivism, where one person’s truth is as good as another’s. All is opinion.
This relativistic kind of thinking was, and still is in most Universities, the dominant belief among academics. Truth is supposed to be relative and subjective, not objective, unless it happens to be science. Hard science is supposed to have a monopoly on objectivity. Unfortunately, this relativistic way of thinking has had some unintended consequences. It has led to the kind of political instability that we see in the U.S. today. That is the basic insight of a new book by Pulitzer Prize winner, Michiko Kakutani. The Death of Truth: Notes on Falsehood in the Age of Trump (Penguin, 2018). Also see Hanlin, Postmodernism didn’t cause Trump. It explains him. (Washington Post, 9/2/18).
Truth is truth. It is not just what the company with the biggest wallet says it is. It is not an opinion. Objective truth, the facts based on hard evidence, is real. It is not just an opinion. This video ad below by CNN was cited by Kakutani in her Death of Truth. It makes the case for objectivity in a simple, common sense manner. The political overtones are obvious.
There is a place for the insights of Post-Modern Subjectivism, especially as it concerns religion. But for now the objective-subjective pendulum has swung too far into the subjective. The pause between directions is over and it is starting to swing back. Facts and truth are becoming important again. This point in legal history will, I predict, be marked by the Mueller investigation. Evidence is once again starting to sing in our justice system. It is singing the body electric. The era of E-Vestigations has begun!
What are E-Vestigations?
E-Vestigations are confidential, internal investigations that focus on search of client data and metadata. They uses Artificial Intelligence to search and retrieve information relative to the client’s requested investigation, their information need. We use an AI machine training method that we call Hybrid Multimodal Predictive Coding 4.0. The basic search method is explained in the open-sourced TAR Course, but the Course does not detail how the method can be used in this kind of investigation.
E-Vestigation is done outside of Litigation and court involvement, usually to try to anticipate and avoid Litigation. Are the rumors true, or are the allegations just a bogus attempt to extort a settlement? E-Vestigations are by nature private, confidential investigations, not responses to formal discovery. AI Assisted investigations rely primarily on what the data says, not the rumors and suspicions, or even what some people say. The analysis of vast volumes of ESI is possible, even with millions of files, because e-Vestigations use Artificial Intelligence, both passive and active machine learning. Otherwise, the search of large volumes of ESI takes too long and is too prone to inaccuracies. That is the main reason this approach is far less expensive than traditional “full calorie” Muller type investigations.
The goal of E-Vestigation is to find quick answers based on the record. Interviews may not be required in many investigations and when they are, they are quick and, to the interviewee, mysterious. The answers to the information needs of a client are sometimes easily found. Sometimes you may just find the record is silent as to the issue at hand, but that silence itself often speaks volumes.
The findings and report made at the end of the E-Vestigation may clear up suspicion, or it may trigger a deeper, more detailed investigation. Sometimes the communications and other information found may require an immediate, more drastic response. One way or another, knowing provides the client with legitimate peace of mind.
The electronic evidence is most cases will be so overwhelming (we know what you said, to whom and when) that testimony will be superfluous, a formality. (We have your communications, we know what you did, we just need you to clear up a few details and help us understand how it ties into guys further up the power chain. That help will earn you a lenient plea deal.) This is what is happening right now, January 2019, with the investigation of Robert Mueller.
Defendants in criminal cases will still plea out, but based on the facts, on truth, not threats. Defendants in civil cases will do the same. So will the plaintiff in civil cases who makes unsubstantiated allegations. Facts and truth protect the innocent. Most of that information will be uncovered in computer systems. In the right hands, E-Vestigations can reveal all. It is a proactive alternative to Litigation with expensive settlements. The AI data review features of E-Vestigations make it far less expensive than a Muller investigations. Is it Mueller Time for your organization?
Robert Mueller never need ask a question of a witness to which he does not already know the answer based on the what the record said. The only real question is whether the witness will further compound their problems by lying. They often do. I have seen that several times in depositions of parties in civil cases. It is sheer joy and satisfaction for the questioner to watch the ethically challenged party sink into the questioner’s hidden traps. The “exaggerating witness” will often smile, just slightly, thinking they have you fooled, just like their own attorney. You smile back knowing their lies are now of record and they have just pounded another nail into their coffin.
E-Vestigations may lead to confrontation, even arrest, if the investigation confirms suspicions. In civil matters it may lead to employee discharge or accusations against a competitor. It may lead to an immediate out-of-court settlement. In criminal matters it may lead to indictment and an informed plea and sentencing. It may also lead to Litigation in civil matters with formal, more comprehensive discovery, but at least the E-Vestigating party will have a big head start. They will know the facts. They will know what specific information to ask for from the opposing side.
Eventually, civil suits will not be filed that often, except to memorialize a party’s agreement, such as a consent to a judgment. It will, instead, be a world where information needs are met in a timely manner and Litigation is thereby avoided. A world where, if management needs to know something, such as whether so and so is a sexual predator, they can find out, fast. A world where AI in the hands of a skilled legal team can mine internal data-banks, such as very large collections of employee emails and texts, and find hidden patterns. It may find what was suspected or may lead to surprise discoveries.
The secret mining of data, otherwise known as “reading other people’s emails without their knowledge” may seem like an egregious breach of privacy, but it is not, at least not in the U.S. under the computer use policies of most groups. Employees typically consent to this search as a condition of employment or computer use. Usually the employer owns all of the equipment searched. The employee has no ownership, nor privacy rights in the business related communications of the employer.
The use of AI assistants in investigations limits the exposure of irrelevant information to humans. First, only a few people are involved in the investigation at all because the AI does the heavy lifting. Second, the human reviewers are outside of the organization. Third, the AI does almost all of the document review. Only the AI reads all of the communications, not the lawyers. The humans look at far less than one percent of the data searched in most projects. They spend most of their time in study of the documents the AI has already identified as likely relevant.
The approach of limited investigations, of going in and out of user data only to search in separate, discreet investigations, provides maximum confidentiality to the users. The alternative, which some organizations have already adopted, is constant surveillance by AI of all communications. You can predict future behavior that way, to a point and within statistical limitations of accuracy. The police in some cities are already using constant AI surveillance to predict crimes and allocate resources accordingly.
I find this kind of constant monitoring to be distasteful. For me, it is too Big Brother and oppressive to have AI looking at my every email. It stifles creativity and, I imagine, if this was in place, would make me overly cautious in my communications. Plus, I would be very concerned about software error. If some baby AI is always on, always looking for suspicious patterns, it could make mistakes. The programming of the software almost certainly contains a number of hidden biases of the programmers, typically young white dudes.
The one-by-one investigation approach advocated here provides for more privacy protection. With E-Vestigations the surveillance is focused and time limited. It is not general and ongoing.
Five Virtues of E-Vestigations
Although I am not going to go into the proprietary details here of our E-Vestigations service (contact me through my law firm if you want to know more), I do want to share what I think are the five most important traits of our AI (robotic) assisted reviews: economics, confidentiality, stealth, speed and accuracy.
Confidentiality:
Complete Secrecy.
Artificial Intelligence means fewer people are required.
Employee Privacy Rights Respected.
Data need never leave corporate premises using specialized tools from our vendor.
Attorney-Client Privilege & Work Product protected.
Stealthiness:
Under the Radar Investigation.
Only some in client IT need know.
Sensitive projects. Discreet.
Stealth forensic copy and review of employee data.
Attorneys review off-site, unseen, via encrypted online connection.
Private interviews; only where appropriate.
Speed:
Techniques designed for quick results, early assessments.
Informal, high-level investigations. Not Litigation Discovery.
High Speed Document Review with AI help.
Example: Study of Clinton’s email server (62,320 files, 30,490 disclosed – 55,000 pgs.) is, at most, a one-week project with a first report after one day.
Accuracy:
Objective Findings and Analysis.
Independent Position.
Specialized Expertise.
Answers provided with probability range limitations.
Known Unknowns (Rumsfeld).
Clients are impressed with the cost of E-Vestigations, as compared to traditional investigations. That is important, of course, but the speed of the work is what impresses many. We produce results, use a flat fee to get there, and do so very FAST.
Certainly we can move much faster than the FBI reviewing email using its traditional methods of expert linear review. The Clinton email investigations took forever by our standards. Yet, Clinton’s email server had only 62,320 files, of which 30,490 were disclosed (around 55,000 pages.) This is, at most, a one-week E-Vestigations project with a first report after one day. Our projects are much larger. They involve review of hundreds of thousands of emails, or hundreds of millions. It does not make a big difference in cost because the AI, who works for free, is doing the heavy lifting of actual studying of all this text.
Most federal agencies, including the FBI, do not have the software, the search knowledge, nor attorney skills for this new type of AI assisted investigation. They also do not have the budget to acquire good AI for assist. Take a look at this selection from the official FBI collection of Clinton email and note that the FBI and US Attorneys office in Alexandra Virginia were communicating by fax in September 2015!
State and federal government agencies are not properly funded and cannot compete with private industry compensation. The NSA may well have an A-Team for advanced search, but not the other agencies. As we know, the NSA has their hands full just trying to keep track of the Russians and other enemies interfering with our elections, not to mention the criminals and terrorists.
Unintended Consequence of Mediation Was to Insert Subjectivism into the Law
As discussed, the rise and commoditization of Mediation over the last twenty years has had unintended consequences. The move from the courtroom to the mediator’s office in turn caused the Law to move from objective to subjective opinion. Discussion of the consequences of mediation, and the subjectivist attitude it brings, complicates my analysis of the death of Litigation, but is necessary. Litigation did not turn into private investigation work. One did not flow into another. Litigation is not changing directly into private Investigations, AI assisted or not. Mediation, and its unexpected consequences, is the intervening stage.
1. Litigation → 2. Mediation → 3. AI Assisted Investigations
Mediation brought down Litigation, at least the all important Trial part of Litigation, not AI or private investigations. There is never a judge making rulings at a mediation. There are only attorneys and assertions of what. Somebody must be lying, but with Mediation you never know who. Lawyers found they could settle cases without all that. They did not need the judge at all. At mediation there are no findings of fact, no rulings of law, just droll agreements as to who will pay how much to whom.
The next stage I predict of AI Assisted Investigations is filling a gap caused by the unintended consequence of Mediation. Mediation was never intended to spawn AI Assisted Investigations, no such thing even existed. It was not possible. We did not have the technology to do something like this. The forces driving the advent of AI Assisted Investigations, which I call E-Vestigations, have little to do with Mediation directly, but are instead the result of rapid advances in technology.
Mediation was intended to encourage settlement and reduce expensive trials. It has been wildly successful at that; exceeded all expectations. But this surprise success has also led to unexpected negative consequences. It has led to a new subjectivistic attitude in Litigation. It has led to the decline of evidence and an over-relativistic attitude where Truth was dethroned.
Most of my Mediator friends strongly disagree, but I have never heard a compelling argument to the contrary. The death of the trial is a stunning development. But mediation has had another impact. One that I have not seen discussed previously. It has not only killed trials, it has killed the whole notion of objective truth. It has led to a mediation mind-set where the “merits” are just a matter of opinion. Where cost of defense and the time value of money are the main items of discussion.
That foreseeable defect has led to the unforeseeable development of an AI Assisted alternative to Litigation. It is led to E-Vestigations. AI can now be used to help lawyers investigate and quickly find out the true facts of a situation.
Many lawyers who litigate today do not care what “really happened.” Very post-modern of them, but come on? A few lawyers just blindly believe whatever damned fool thing their client tells them. Most just say we will never know the absolute truth anyway, so let us just try our best to resolve the dispute and do what’s fair without any test of the evidence. They try to do justice with just a passing nod to the evidence, to the truth of what happened. I am not a fan. It goes against all of my core teachings as a young commercial litigation attorney who prepared and tried cases. It goes against my core values and belief. My opinion is that it is not all just opinion, that there is truth.
I object to that mediation, relativistic approach. After a life in the Law chasing smoking guns and taking depositions, I know for a fact that witnesses lie, that their memories are unreliable, all too human. But I also know that the writings made by and to these same witness often expose the lies, or, more charitably put, expose the errors in human memory. Fraudsters are human and almost always make mistakes. It is an investigator’s job to check the record to find the slip-ups in the con. (I dread the day when I have to try to trace a AI fraudster!)
I have been chasing and exposing con-men most of my adult life. I defended a few too. In my experience the truth has a way of finding its way out.
This is not an idealistic dream in today’s world of information floods. There is so much information, the real difficulty is in finding the important bits, the smoking guns, the needles. The evidence is usually there, but not yet found. The real challenge today is not in gathering the evidence, it is in searching for the key documents, finding the signal in the noise.
Conclusion
Objective accounts of what happened in the past are not only possible, they are probable in today’s Big Data world. Your Alexa or Google speakers may have part of the record. So too may your iWatch or Fitbit. Soon your refrigerator will too. Data is everywhere. Privacy is often an illusion. (Sigh.) The opportunity of liars and other scoundrels to “get away with it” and fool people is growing smaller every day. Fortunately, if lawyers can just learn a few new evidence search skills, they can use AI to help them find the information they need.
Juries and judges, for the most part, believe in objective truth. They are quite capable of sorting through competing versions and getting at the truth. Good judges and lawyers (and jurors) can make sure that happens.
As mentioned, many academics and sophisticates believe otherwise, that there is no such a thing as objective truth. They believe instead in Relativism. They are wrong.
The postmodernist argument that all truths are partial (and a function of one’s perspective) led to the related argument that there are many legitimate ways to understand or represent an event. . . .
Without commonly agreed-upon facts — not Republican facts and Democratic facts; not the alternative facts of today’s silo-world — there can be no rational debate over policies, no substantive means of evaluating candidates for political office, and no way to hold elected officials accountable to the people. Without truth, democracy is hobbled. The founders recognized this, and those seeking democracy’s survival must recognize it today.
It is possible to find the truth, objective truth. All is not just opinion and allegations. Accurate forensic reconstruction is possible today in ways that we could never have imagined before. So is AI assisted search. The record of what is happening grows larger every day. That record written electronically at the time of the events in question is far more reliable than our memories. We can find the truth, but for that need to look primarily to the documents, not the testimony. That is not new. That is wisdom upon which almost all trial lawyers agree.
The truth is attainable, but requires dedication and skilled efforts by everyone on a legal team to find it. It requires knowledge of course, and a proven method, but also impartiality, discipline, intelligence and a sense of empathy. It requires experience with what the AI can do, and just as important, what it cannot do. It requires common sense. Lawyers have that. Jurors have that.
Surely only a weak-minded minority are fooled by today’s televised liars. Most competent trial lawyers could persuade a sequestered jury to convict them. And convict they will, but that still will not cause of rebirth of Litigation. Its’ glory days are over. So too is its killer, Mediation, although its death will take longer (Mediation may not even have peaked yet).
Evidence speaks louder than any skilled mediator. Let the truth be told. Let the chips fall where they may. King Litigation is dead. Long live the new King, confidential, internal AI assisted E-Vestigations.
The term “TAR” – Technology Assisted Review – as we use it means document review enhanced by active machine learning. Active machine learning is an important tool of specialized Artificial Intelligence. It is now widely used in many industries, including Law. The method of AI-enhanced document review we developed is called Hybrid Multimodal Predictive Coding 4.0. Interestingly, reading these words in the new Sans Forgetica font will help you to remember them.
We have developed an online instructional program to teach our TAR methods and AI infused concepts to all kinds of legal professionals. We use words, studies, case-law, science, diagrams, math, statistics, scientific studies, test results and appeals to reason to teach the methods. To balance that out, we also make extensive use of photos and videos. We use right brain tools of all kinds, even subliminals, special fonts, hypnotic images and loads of hyperlinks. We use emotion as another teaching tool. Logic and Emotion. Sorry Spock, but this multimodal, holistic approach is more effective with humans than an all-text, reason-only approach of Vulcan law schools.
We even try to use humor and promote student creativity with our homework assignments. Please remember, however, this is not an accredited law school class, so do not expect professorial interaction. Did we mention the TAR Course is free?
By the end of study of the TAR Course you will know and remember exactly what Hybrid Multimodal means. You will understand the importance of using all varieties of legal search, for instance: keywords, similarity searches, concept searches and AI driven probable relevance document ranking. That is the Multimodal part. We use all of the search tools that our KL Discovery document review software provides.
The Hybrid part refers to the partnership with technology, the reliance of the searcher on the advanced algorithmic tools. It is important than Man and Machine work together, but that Man remain in charge of justice. The predictive coding algorithms and software are used to enhance the lawyers, paralegals and law tech’s abilities, not replace them.
By the end of the TAR Course you will also know what IST means, literally Intelligently Spaced Training. It is our specialty technique of AI training where you keep training the Machine until first pass relevance review is completed. This is a type of Continuous Active Learning, or as Grossman and Cormack call it, CAL. By the end of the TAR Course you should also know what a Stop Decision is. It is a critical point of the document review process. When do you stop the active machine teaching process? When is enough review enough? This involves legal proportionality issues, to be sure, but it also involves technological processes, procedures and measurements. What is good enough Recall under the circumstances with the data at hand? When should you stop the machine training?
We can teach you the concepts, but this kind of deep knowledge of timing requires substantial experience. In fact, refining the Stop Decision was one of the main tasks we set for ourself for the e-Discovery Team experiments in the Total Recall Track of the National Institute of Standards and Technology Text Retrieval Conference in 2015 and 2016. We learned a lot in our two years. I do not think anyone has spent more time studying this in both scientific and commercial projects than we have. Kudos again to KL Discovery for helping to sponsor this kind of important research by the e-Discovery Team.
Working with AI like this for evidence gathering is a newly emerging art. Take the TAR Course and learn the latest methods. We divide the Predictive Coding work flow into eight-steps. Master these steps and related concepts to do TAR the right way.
Pop Quiz: What is one of the most important considerations on when to train again?
One Possible Correct Answer: The schedule of the humans involved. Logistics and project planning is always important for efficiency. Flexibility is easy to attain with the IST method. You can easily accommodate schedule changes and make it as easy as possible for humans and “robots” to work together. We do not literally mean robots, but rather refer to the advanced software and the AI that arises from the machine training as an imiginary robot.
Playing games is a great way to learn. That’s one reason I’ve devised a game concerning the interesting and fairly complex issues involved in trying to determine what e-discovery activities are proportional and appropriate in various sized cases. Specifically, what should you do to prepare for federal court 26(f) conferences in small and medium sized cases, versus large, complicated cases? Small, Medium or Large? is kind of a Goldilocks game of proportionality.
I have had to give these proportionality questions a lot of thought as part of my practice as a lawyer supervising hundreds of e-discovery projects at a time, projects of all different sizes. I could simply give you my answer, but after five books, I’ve already been there and done that. So I thought I’d try something new and make this learning into a game where you consider and vote on what activities you think are appropriate for a Small, Medium or Large case.
The Hive Mind is different from Crowdsourcing, but related. To do properly, a Hive Mind requires Swarming, which we cannot really do properly in this game using polls. Louis Rosenberg, Super-Intelligence and the virtues of a “Hive Mind” (Singularity, 2/10/16). One Silicon Valley startup Unanimous A.I., is developing technologies that enable sophisticated online human swarming and thus better collective intelligence. I might try their free product for social research, UNO, if I further investigate the power of the Hive Mind. Maybe at NY LegalTech? (Email me if you’re interested.) In the meantime we are going to use simple polling for the e-Discovery Hive Mind Game: Small, Medium or Large?
Let’s play the game and see what collective intelligence emerges from many individuals giving their opinion about proportionality and e-discovery. I am playing this game now with all of the litigation associates and paralegals in my law firm, which is a pretty large swarm by itself. Your responses will join in the swarm, the collective intelligence.
In January I’ll share the results of all the polling and opine away as to how well the Hive Mind performed. You can win the game in one of two ways, either by matching the most popular Hive Mind responses or by matching my responses. (I assume there will be a difference because not enough experts will be playing, but who knows, maybe not. In theory, with enough experts swarming, the group, the Hive Mind, will always have the best answer.)
Background To Play the Game
In order to play the Small, Medium or Large? game, you first need to be familiar with the checklist of the Southern District Court of Florida of all of the things that you should do to prepare for e-discovery in a large case. It is a pretty good list and I have written about it before. Good New 33-Point e-Discovery Checklist From Miami (e-Discovery Team, October 1, 2017) (a must read to fully prepare for this game). My article contains comments and explanations about all checklist items, which is the beginning of a kind of swarm interaction, that is, if you take time to ponder the signals. The Court’s checklist incorporates the new provisions in the rules on relevance and proportionality (Rule 26(b)(1)) and on specific objections (Rule 34(b)(2)).
It is not a perfect list, but it is the best one now out there with a court pedigree. It is not too long and complex, like the older lists of some courts that are very detailed, and not too short, like the easy-peasy list that Bill Hamilton and I created for the Middle District Court of Florida many years ago. (Attorneys still complained about how burdensome it was!) In sum, the 33-Point Checklist out of Miami is a good list for legal practitioners all over the country to use to prepare for e-discovery, which means it is a good basis for our Small, Medium or Large?Hive Mind Game. Come play along.
Rules of the Game
The goal of our Hive Mind Game is to determine which of the thirty-three points on the checklist are applicable to big cases only, which are applicable to medium size cases and which to small cases. You are to assume that all thirty-three points apply to large cases, but that they are not all applicable to medium and small size cases. The Hive Mind voting is used to allow the swarm – that’s you – to identify which of the thirty-three only apply to small cases, and which only apply to medium size cases. If a checklist item applies to a medium size case, it automatically also applies to a small size case.
In other words, the game is to sort the thirty-three into three piles, Small, Medium or Large? Simple, eh? Well, maybe not. This is a matter of opinion and things are pretty vague. For instance, I’m not going to define the difference between large, small and medium size case. That is part of the Hive Mind.
The game is important because proportionality in the law is important. You do not prepare for a big case the same way you prepare for a small case. You just don’t. You could, but it would be a waste of your clients money to do so. So the real trick in e-discovery, like in all other aspects of the litigation, is to determine what you should do in any size case to prepare, including even the small cases. For instance, outside of e-discovery, most people would agree that you should take the parties depositions as a minimum to prepare for a trial of any size case, including small ones.
What are the equivalent items in the 33-point checklist? Which of them should be applied to all cases, even the small ones? Which of them are too complicated and expensive to apply in a small case, but not a medium sized case? Which too complicated and expensive to apply in a small or medium sized case, but not a large one? That is where the real skill and knowledge come in. That is the essence of the game.
Assume All 33 Items Apply to Big Cases
This Small, Medium or Large?Hive Game requires you to assume that all thirty-three items on the Court’s checklist apply to big cases, but not to all cases, that there are certain checklist items that only apply to medium size cases, and others, a smaller list, that only apply to small cases. You may question the reality of this assumption. After all, the Court does not say that. It does not say, here’s a checklist we made to guide your e-discovery, but you can ignore many of the items on this list if you have a small case, or even a medium size case. Still, that’s what they mean, but they do not go on to say what’s what. They know that the Bar will figure it out themselves in due time, meaning the next several years. And right they are. But why wait? Let’s figure it out ourselves now with this Small, Medium or Large? Hive Game, and condense years to weeks.
The Hive Game will allow us to fill in the blanks of what the Court did not say. But first, lets focus again on what the Court did say. It said that the checklist may be used by members of the Bar to guide the Rule 26(f) e-discovery conferences and Case Management Reports. It did not say shall be used. It is a suggestion, not a requirement. Still, as every long-term member of the court knows, what they mean is that you damn well should follow the checklist in a big case! Woe unto the lawyers who come before the judges in a big case with an e-discovery issue where they never even bothered to go through the checklist. If your issue is on that list, and chances are it will be, then dear slacker, prepare for a Miami style bench-slap. Ouch! It is going to hurt. Not only you and your reputation, but also your client.
I feel confident in making the game assumption that all thirty-three checklist items apply to big cases. I am also confident they do not all apply to medium and small size cases and that is the real reason for the court’s use of may instead of shall.
With that background we are almost ready to start playing the game and opine away as to which of the 33 are small and medium size only. But, there is still one more thing I have found very helpful when you try to really dig into the checklist and play the game, you need to have the numbers 1-33 added to the list. The one mistake the Court made on this list was in using headings and bullet points instead of numbers. I fix that in the list that follows, so that you can, if nothing else, more easily play the game.
Preservation
The ranges of creation or receipt dates for any ESI to be preserved.
The description of ESI from sources that are not reasonably accessible because of undue burden or cost and that will not be reviewed for responsiveness or produced, but that will be preserved in accordance with Federal Rule of Civil Procedure 26(b)(2)(B).
The description of ESI from sources that: (a) the party believes could contain relevant information; but (b) has determined, under the proportionality factors, is not discoverable and should not be preserved.
Whether to continue any interdiction of any document-destruction program, such as ongoing erasures of e-mails, voicemails, and other electronically recorded material.
The number and names or general job titles or descriptions of custodians for whom ESI will be preserved (e.g., “HR head,” “scientist,” “marketing manager”).
The list of systems, if any, that contain ESI not associated with individual custodians and that will be preserved, such as enterprise databases.
Any disputes related to scope or manner of preservation.
Liaison
The identity of each party’s e-discovery liaison, who will be knowledgeable about and responsible for each party’s ESI.
Informal Discovery About Location and Types of Systems
Identification of systems from which discovery will be prioritized (e.g., e-mail, finance, HR systems).
Descriptions and location of systems in which potentially discoverable information is Stored.
How potentially discoverable information is stored.
How discoverable information can be collected from systems and media in which it is stored.
Proportionality and Costs
The amount and nature of the claims being made by either party.
The nature and scope of burdens associated with the proposed preservation and discovery of ESI.
The likely benefit of the proposed discovery.
Costs that the parties will share to reduce overall discovery expenses, such as the use of a common electronic-discovery vendor or a shared document repository, or other costsaving measures.
Limits on the scope of preservation or other cost-saving measures.
Whether there is relevant ESI that will not be preserved in accordance with Federal Rule of Civil Procedure 26(b)(1), requiring discovery to be proportionate to the needs of the case.
Search
The search method(s), including specific words or phrases or other methodology, that will be used to identify discoverable ESI and filter out ESI that is not subject to discovery.
The quality-control method(s) the producing party will use to evaluate whether a production is missing relevant ESI or contains substantial amounts of irrelevant ESI.
Phasing
Whether it is appropriate to conduct discovery of ESI in phases.
Sources of ESI most likely to contain discoverable information and that will be included in the first phases of Federal Rule of Civil Procedure 34 document discovery.
Sources of ESI less likely to contain discoverable information from which discovery will be postponed or not reviewed.
Custodians (by name or role) most likely to have discoverable information and whose ESI will be included in the first phases of document discovery.
Custodians (by name or role) less likely to have discoverable information from whom
discovery of ESI will be postponed or avoided.
The time period during which discoverable information was most likely to have been
created or received.
Production
The formats in which structured ESI (database, collaboration sites, etc.) will be produced.
The formats in which unstructured ESI (e-mail, presentations, word processing, etc.) will be produced.
The extent, if any, to which metadata will be produced and the fields of metadata to be produced.
The production format(s) that ensure(s) that any inherent searchability of ESI is not degraded when produced.
Privilege
How any production of privileged or work-product protected information will be handled.
Whether the parties can agree on alternative ways to identify documents withheld on the grounds of privilege or work product to reduce the burdens of such identification.
Whether the parties will enter into a Federal Rule of Evidence 502(d) stipulation and order that addresses inadvertent or agreed production.
One Example Before the Games Begin
We are almost ready to play the e-Discovery Small, Medium or Large? Hive Mind Game. We will do so with thirty-two polls that are presented to the player in the same order as the Court’s checklist. To make sure the rules are clear (this is, after all, a game for lawyers, not kids) we start with an example, the first of the thirty-three items on the checklist. The court’s first item is to suggest that you Determine the range of creation or receipt dates for any ESI to be preserved.
The “right answer” to this first item is that this should be done in every case, even the small ones. You should always determine the date range of data to be preserved. In most cases that is very easy to do, and, as every lawyer should know, when in doubt, when it comes to preservation, always err on the side of inclusion. That means you should check the Small Case answer as shown in the “dummy poll” graphic below.
Checklist 1
We have set these polls up so that you cannot see the results, but you can leave private comments. We may do this again later and experiment with what happens when you can see the results. We will share the results (and some comments) when the game ends on January 1, 2019.
Now for the live polls and game proper. Note that several of the checklist items, including number two and three, which are the first two polls shown below, are so long that we had to paraphrase and shorten them to fit in the space allocated in the polling software. To see the original of all thirty-three items on the checklist, go to my prior blog explaining the list (highly recommended) or the court’s page.
Let the Games Begin!
We are now ready to begin playing the e-Discovery Hive Mind Game. So get ready to plug-in. Select an answer to each of the thirty-two polls that follow. After you vote, you also have a chance to leave a private comment to each poll, but that is optional and will not impact your score.
Checklist 2
________________________________________
Checklist 3
________________________________________
Checklist 4
________________________________________
Checklist 5
________________________________________
Checklist 6
________________________________________
Checklist 7
________________________________________
Checklist 8
________________________________________
Checklist 9
________________________________________
Checklist 10
________________________________________
Checklist 11
________________________________________
Checklist 12
________________________________________
Checklist 13
________________________________________
Checklist 14
________________________________________
Checklist 15
________________________________________
Checklist 16
________________________________________
Checklist 17
________________________________________
Checklist 18
________________________________________
Checklist 19
________________________________________
Checklist 20
________________________________________
Checklist 21
________________________________________
Checklist 22
________________________________________
Checklist 23
________________________________________
Checklist 24
________________________________________
Checklist 25
________________________________________
Checklist 26
________________________________________
Checklist 27
________________________________________
Checklist 28
________________________________________
Checklist 29
________________________________________
Checklist 30
________________________________________
Checklist 31
________________________________________
Checklist 32
________________________________________
Checklist 33
________________________________________
________________________________________
Congratulations! You have finished the Game and made your contribution to the e-Discovery Hive Mind. Look for results sometime in early 2018. You can then determine how your answers compared with the collective Hive Mind.
I will also let you know how the Hive Mind answers compared with my own. So you will have two chances to win. Anyone who matches all of my answers wins a free lunch with me in Orlando. Other prizes have yet to be determined. Vendors care to contribute some goodies? Perhaps Elon will donate a free trip to Mars, where I for one hope we don’t run into any Borg cubes, I don’t care how good their Hive Mind is.
In the meantime, please encourage your e-discovery friends and colleagues to join in the game. Teachers and Partners are invited to require their students and associates, paralegals to play too. Resistance is futile! Digging deep into this checklist is a great way to expand your knowledge and expertise of electronic discovery law and practice.
Ralph Losey is an Arbitrator, Special Master, Mediator of Computer Law Disputes and Practicing Attorney, partner in LOSEY PLLC. Losey is a high tech law firm with three Loseys and a bunch of other cool lawyers. We handle projects, deals, IP of all kinds all over the world, plus litigation all over the U.S. For more details of Ralph's background, Click Here
All opinions expressed here are his own, and not those of his firm or clients. No legal advice is provided on this web and should not be construed as such.
Ralph has long been a leader of the world's tech lawyers. He has presented at hundreds of legal conferences and CLEs around the world. Ralph has written over two million words on e-discovery and tech-law subjects, including seven books.
Ralph has been involved with computers, software, legal hacking and the law since 1980. Ralph has the highest peer AV rating as a lawyer and was selected as a Best Lawyer in America in four categories: Commercial Litigation; E-Discovery and Information Management Law; Information Technology Law; and, Employment Law - Management.
Ralph is the proud father of two children, Eva Losey Grossman, and Adam Losey, a lawyer with incredible cyber expertise (married to another cyber expert lawyer, Catherine Losey), and best of all, husband since 1973 to Molly Friedman Losey, a mental health counselor in Winter Park.
1. Electronically stored information is generally subject to the same preservation and discovery requirements as other relevant information.
2. When balancing the cost, burden, and need for electronically stored information, courts and parties should apply the proportionality standard embodied in Fed. R. Civ. P. 26(b)(2)(C) and its state equivalents, which require consideration of importance of the issues at stake in the action, the amount in controversy, the parties’ relative access to relevant information, the parties’ resources, the importance of the discovery in resolving the issues, and whether the burden or expense of the proposed discovery outweighs its likely benefit.
3. As soon as practicable, parties should confer and seek to reach agreement regarding the preservation and production of electronically stored information.
4. Discovery requests for electronically stored information should be as specific as possible; responses and objections to discovery should disclose the scope and limits of the production.
5. The obligation to preserve electronically stored information requires reasonable and good faith efforts to retain information that is expected to be relevant to claims or defenses in reasonably anticipated or pending litigation. However, it is unreasonable to expect parties to take every conceivable step or disproportionate steps to preserve each instance of relevant electronically stored information.
6. Responding parties are best situated to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own electronically stored information.
7. The requesting party has the burden on a motion to compel to show that the responding party’s steps to preserve and produce relevant electronically stored information were inadequate.
8. The primary source of electronically stored information to be preserved and produced should be those readily accessible in the ordinary course. Only when electronically stored information is not available through such primary sources should parties move down a continuum of less accessible sources until the information requested to be preserved or produced is no longer proportional.
9. Absent a showing of special need and relevance, a responding party should not be required to preserve, review, or produce deleted, shadowed, fragmented, or residual electronically stored information.
10. Parties should take reasonable steps to safeguard electronically stored information, the disclosure or dissemination of which is subject to privileges, work product protections, privacy obligations, or other legally enforceable restrictions.
11. A responding party may satisfy its good faith obligation to preserve and produce relevant electronically stored information by using technology and processes, such as data sampling, searching, or the use of selection criteria.
12. The production of electronically stored information should be made in the form or forms in which it is ordinarily maintained or in a that is reasonably usable given the nature of the electronically stored information and the proportional needs of the case.
13. The costs of preserving and producing relevant and proportionate electronically stored information ordinarily should be borne by the responding party.
14. The breach of a duty to preserve electronically stored information may be addressed by remedial measures, sanctions, or both: remedial measures are appropriate to cure prejudice; sanctions are appropriate only if a party acted with intent to deprive another party of the use of relevant electronically stored information.