Announcing the e-Discovery Team’s TAR Training Program: 16 Classes, All Online, All Free – The TAR Course

March 19, 2017

We launch today a sixteen class online training program on Predictive Coding: the e-Discovery Team TAR Course. This is a “how to” course on predictive coding. We have a long descriptive name for our method, Hybrid Multimodal IST Predictive Coding 4.0. By the end of the course you will know exactly what that means. You will also understand the seventeen key things you need to know to do predictive coding properly, shown this diagram.

 hacking of predictive coding document reviews has been my obsession since Da Silva went viral. Da Silva Moore v. Publicis Groupe & MSL Group, 27 F.R.D. 182 (S.D.N.Y. 2012). That is the case where I threw Judge Peck the softball opportunity to approve predictive coding for the first time. See: Judge Peck Calls Upon Lawyers to Use Artificial Intelligence and Jason Baron Warns of a Dark Future of Information Burn-Out If We Don’t

Alas, because of my involvement in Da Silva I could never write about it, but I can tell you that none of the thousands of commentaries on the case have told the whole nasty story, including the outrageous “alternate fact” attacks by plaintiff’s counsel on Judge Andrew Peck and me. I guess I should just take the failed attempts to knock me and the Judge out of the case as flattery, but it still leaves a bad taste in my mouth. A good judge like Andy Peck did not deserve that kind of treatment. 

At the time of Da Silva, 2012, my knowledge of predictive coding was mostly theoretical, informational. But now, after “stepping-in” for five years to actually make the new software work, it is practical. For what “stepping-in” means see the excellent book on artificial intelligence and future employment by Professor Thomas Davenport and Julia Kirby, titled Only Humans Need Apply (HarperBusiness, 2016). Also see: Dean Gonsowski, A Clear View or a Short Distance? AI and the Legal Industry, and, Gonsowski, A Changing World: Ralph Losey on “Stepping In” for e-Discovery (Relativity Blog). 

If you are looking to craft a speciality in the law that rides the new wave of AI innovations, then electronic document review with TAR is a good place to start. See Part Two of my January 22, 2017 blog, Lawyers’ Job Security in a Near Future World of AI. This is where the money will be.


Our TAR Course is designed to teach this practical, stepping-in based knowledge. The link to the course will always be shown on this blog at the top of the page. The TAR page next to it has related information.

Since Da Silva we have learned a lot about the actual methods of predictive coding. This is hands-on learning through actual cases and experiments, including sixty-four test runs at TREC in 2015 and 2016.

We have come to understand very well the technical details, the ins and outs of legal document review enhanced by artificial intelligence, AI-enhanced review. That is what TAR and predictive coding really mean, the use of active machine learning, a type of specialized artificial intelligence, to find the key documents needed in an investigation. In the process I have written over sixty articles on the subject of TAR, predictive coding and document review, most of them focused on what we have learned about methods.

The TAR Course is the first time we have put all of this information together in a systematic training program. In sixteen classes we cover all seventeen topics, and much more. The result is an online instruction program that can be completed in one long weekend. After that it can serve as a reference manual. The goal is to help you to step-in and improve your document review projects.

The TAR Course has sixteen classes listed below. Click on some and check them out. All free. We do not even require registration. No tests either, but someday soon that may change. Stay tuned to the e-Discovery Team. This is just the first step dear readers of my latest hack of the profession. Change we must, and not just gradual, but radical. That is the only way the Law can keep up with the accelerating advances in technology. Taking the TAR Course is a minimum requirement and will get you ready for the next stage.

  1. First Class: Introduction
  2. Second Class: TREC Total Recall Track
  3. Third Class: Introduction to the Nine Insights Concerning the Use of Predictive Coding in Legal Document Review
  4. Fourth Class: 1st of the Nine Insights – Active Machine Learning
  5. Fifth Class: Balanced Hybrid and Intelligently Spaced Training
  6. Sixth Class: Concept and Similarity Searches
  7. Seventh Class: Keyword and Linear Review
  8. Eighth Class: GIGO, QC, SME, Method, Software
  9. Ninth Class: Introduction to the Eight-Step Work Flow
  10. Tenth Class: Step One – ESI Communications
  11. Eleventh Class: Step Two – Multimodal ECA
  12. Twelfth Class: Step Three – Random Prevalence
  13. Thirteenth Class: Steps Four, Five and Six – Iterate
  14. Fourteenth Class: Step Seven – ZEN Quality Assurance Tests
  15. Fifteenth Class: Step Eight – Phased Production
  16. Sixteenth Class: Conclusion

This course is not about the theory or law of predictive coding. You can easily get that elsewhere. It is about learning the latest methods to do predictive coding. It is about learning how to train an AI to find the ESI evidence you want. The future looks bright for attorneys with both legal knowledge and skills and software knowledge and skills. The best and brightest will also be able to work with various kinds of specialized AI to do a variety of tasks, including AI-enhanced document review. If that is your interest, then jump onto the TAR Course and start your training today. Who knows where it may take you?






e-Discovery Team’s Best Practices Education Program

May 8, 2016


EDBP                   Mr.EDR         Predictive Coding 3.0
59 TAR Articles
Doc Review  Videos



e-Discovery Team Training

Information → Knowledge → Wisdom

Ralph_4-25-16Education is the clearest path from Information to Knowledge in all fields of contemporary culture, including electronic discovery. The above links take you to the key components of the best-practices teaching program I have been working on since 2006. It is my hope that these education programs will help move the Law out of the dangerous information flood, where it is now drowning, to a safer refuge of knowledge. Information → Knowledge → Wisdom: Progression of Society in the Age of Computers; and How The 12 Predictions Are Doing That We Made In “Information → Knowledge → Wisdom.” For more of my thoughts on e-discovery education, see the e-Discovery Team School Page.

justice_guage_negligenceThe best practices and general educational curriculum that I have developed over the years focuses on the legal services provided by attorneys. The non-legal, engineering and project management practices of e-discovery vendors are only collaterally mentioned. They are important too, but students have the EDRM and other commercial organizations and certifications for that. Vendors are part of any e-Discovery Team, but the programs I have developed are intended for law firms and corporate law departments.

LIFE_magazine_Losey_acceleratesThe e-Discovery Team program, both general educational and legal best-practices, is online and available 24/7. It uses lots of imagination, creative mixes, symbols, photos, hyperlinks, interactive comments, polls, tweets, posts, news, charts, drawings, videos, video lectures, slide lectures, video skits, video slide shows, music, animations, cartoons, humor, stories, cultural themes and analogies, inside baseball references, rants, opinions, bad jokes, questions, homework assignments, word-clouds, links for further research, a touch of math, and every lawyer’s favorite tools: words (lots of them), logic, arguments, case law and precedent.

All of this to try to take the e-Discovery Team approach from just information to knowledge →. In spite of these efforts, most of the legal community still does not know e-discovery very well. What they do know is often misinformation. Scenes like the following in a law firm lit-support department are all too common.

supervising-tipsThe e-Discovery Team’s education program has an emphasis on document review. That is because the fees for lawyers reviewing documents is by far the most expensive part of e-discovery, even when contract lawyers are used. The lawyer review fees, and review supervision fees, including SME fees, have always been much more costly than all vendor costs and expenses put together. Still, the latest AI technologies, especially active machine learning using our Predictive Coding 3.0 methods, are now making it possible to significantly reduce review fees. We believe this is a critical application of best practices. The three steps we identify for this area in the EDBP chart are shown in green, to signify money. The reference to C.A. Review is to Computer Assisted Review or CAR, using our Hybrid Multimodal methods.



Predictive Coding 3.0 Hybrid Multimodal Document Search and Review

Control-SetsOur new version 3.0 techniques for predictive coding makes it far easier than ever before to include AI in a document review project. The secret control set has been eliminated, so too has the seed set and SMEs wasting their time reviewing random samples of mostly irrelevant junk. It is a much simpler technique now, although we still call it Hybrid Multimodal.

robot-friendHybrid is a reference to the Man/Machine interactive nature of our methods. A skilled attorney uses a type of continuous active learning to train an AI to help them to find the documents they are looking for. This Hybrid method greatly augments the speed and accuracy of the human attorneys in charge. This leads to cost savings and improved recall. A lawyer with an AI helper at their side is far more effective than lawyers working on their own. This means that every e-discovery team today could use a robot like Kroll Ontrack’s Mr. EDR to help them to do document review.

Search_pyramidMultimodal is a reference to the use of a variety of search methods to find target documents, including, but not limited to, predictive coding type ranked searches. We encourage humans in the loop running a variety of searches of their own invention, especially at the beginning of a project. This always makes for a quick start in finding relevant and hot documents. Why the ‘Google Car’ Has No Place in Legal Search. The multimodal approach also makes for precise, efficient reviews with broad scope. The latest active machine learning software when fully integrated with a full suite of other search tools is attaining higher levels of recall than ever before. That is one reason Why I Love Predictive Coding.

Mr_EDRI have found that Kroll Ontrack’s EDR software is ideally suited for these Hybrid, Multimodal techniques. Try using it on your next large project and see for yourself. The Kroll Ontrack consultant specialists in predictive coding, Jim and Tony, have been trained in this method (and many others). They are well qualified to assist you in every step of the way and their rates are reasonable. With you calling the shots on relevancy, they can do most of the search work for you and still save your client’s money. If the matter is big and important enough, then, if I have a time opening, and it clears my firm’s conflicts, I can also be brought in for a full turn-key operation. Whether you want to include extra time for training your best experts is your option, but our preference.



Embrace e-Discovery Team Education to Escape Information Overload


Spoliation Sanctions: The Tide is Turning, Goliath is Smiling

July 29, 2013


Since e-discovery law first began in the nineteen-nineties the problem of spoliation and sanctions has primarily been that of large corporations. In the classic David and Goliath scenario, typical for instance in employment litigation, the little David plaintiffs had a powerful slingshot to win cases: spoliation and sanctions. The large corporate defendants had zillions of bits of ESI, the little guy David requesting data had little or no bits. This imbalance of burden allowed the clever plaintiff’s counsel to win a case, not on the merits of their client’s complaint, but on the failures of the defendant corporation, the failures to lock down and secure all potentially relevant ESI from destruction. If the little guy could prove intentional destruction by one of the key players for the corporate defendant, then it was game over for the defendant. The famous Zubulake saga by Judge Scheindlin is an employment law case that proved the point. After Zubulake very few defendants would even risk going to trial, like UBS Warburg did in Zubulake. Instead, all too often, they just paid too much to settle the case.

Electronic discovery became the great equalizer in the seemingly never-ending battle between the plaintiffs Bar and the defense Bar. Even if a plaintiff could not prove spoliation, there was always an ESI cloud hanging over defense counsel’s head. Did their client preserve everything they needed to? How much was enough?  How were they going to respond to plaintiff’s request to produce. After while, by around 2010, things got so bad that just the threat of a production request was enough to force inflated settlements in some cases. Clever plaintiff’s counsel did not even have to get into spoliation or sanctions motion practice, all they had to do was get an order compelling production. The costs to search and produce might very well exceed the cost to settle the case. Yes, little David was smiling. All he had to do was take his e-discovery slingshot out of  his pocket and the allegedly big bad Goliath might run away.

Continuing Legal Education

Many e-discovery specialists, especially those of us who knew the Goliaths, and knew they were not as bad as the propaganda would suggest, were upset by all the cases we saw settling without regard to merits. Many defense counsel, as I was until 2006, really hated to see claims paid that they knew were bogus. It just did not seem right, even though it made economic sense to insurance adjusters. That was part of the reason that in 2006 I gave up my regular, admittedly defense oriented practice (although I always did some plaintiff’s work too when possible), and only worked on electronic discovery. Like many other tech-oriented attorneys I was focused on trying to level the playing field. I wanted fair trials to come back. I wanted Goliath to clean up his act so that would be possible. I wanted to teach Goliath how to preserve, search, and produce in a fair and proportional manner so that he would not have to shy away from David’s e-discovery slingshot.

Yes. I wanted Goliath to reform, to stop all destruction of ESI, to preserve in a prudent manner. So did many others attorneys moving into the e-discovery field from both sides. We all wanted cases to be decided on the merits. We all wanted important evidence to be preserved, not destroyed, either intentionally or by negligence. The answer then and now seemed to be a combination of education and technology. Most attorneys who had developed knowledge and skills gave of their time to present CLEs that would help other attorneys to preserve and protect, to search and serve . CLEs were a good way to network and improve everyone’s skills.

InnovateThese education efforts have been going on for decades now. I estimate that tens of thousands of e-discovery CLE have been presented by now, most by a relatively small cadre of dedicated judges, lawyers and techs. This effort has now even become a second-generation effort, with my son, Adam Losey, and his non-profit foundation, IT-Lex, putting on its first CLE in Florida on October 17 and 18, 2013. It’s called Innovate, and will have many of the top speakers and educators in a state-of-the-art event. (I suggest you sign up soon, like the Sedona Conference, attendance at Innovate will be capped.)

Why Goliath Is Smiling?

Bruce_Lee_smilingAfter all of these years of e-discovery anguish, Goliath is finally starting to smile. There are two reasons. First, all of this hard work at education and training of the holders of too-much-information is starting to pay off. Goliath is learning how to save evidence. The many thousands of CLEs on e-discovery given over the past fifteen years are starting to bear fruit. Even though corporations have more data than ever before, and more kinds of ESI, they are getting their act together. More and more of them have dedicated internal e-discovery teams, a move I have been pushing in this e-discovery team blog since 2006. Just as important, large organizations have now learned, many of them the hard way, but they learned, that they need to hire lawyers who are competent in e-discovery. They have learned that competence in litigation alone is not adequate. A new class of e-discovery lawyers has emerged, and they are serving their clients well.

The second reason Goliath is smiling is that little David is not so little any more. Frankly, he has become quite pudgy, bloated with ESI. Yes, even the little guys today have more ESI with potential evidence than they know what to do with, much less know how to preserve, search and produce. Also, it turns out that David is not always the honest boy scout we had been led to believe. The little guys are now often the ones out there spoliating away, thinking that they can get away with it, just like Laura Zubulake’s supervisor a decade ago. It is all just people after all, and some people are not that honest. If they are not stopped by assertive counsel, they will try to hide the bad stuff.

The plaintiffs bar is now starting to finds itself in the same position the defense bar was in a few years ago. From my front line position in the employment law practice, where small versus big is the norm, I think the tide has now turned. It is the plaintiffs now who cannot seem to save, find, or produce their ESI. It is the plaintiff’s now who are losing cases, in spite of the merits, based on their intentional or negligent destruction of evidence. It is the plaintiffs who are getting bad or no advice from their attorneys about preservation. It is the plaintiffs who must now settle because they could not or cannot find and produce all of the evidence responsive to defendants requests, all of the Facebook posts, tweets, texts, and email. The tide has turned.

riverview3I offer two recent examples of this, one from my home court in Orlando Florida, and another from Alabama. In the Orlando case a former employee was caught throwing her computer into the river. Simon Property Group, Inc. v. Lauria, 2012 U.S. Dist. LEXIS 184638 (M.D. Fla. 2012). Good defense counsel tactics uncovered this fraud, a fraud that, in my opinion, could have been prevented by better advice to plaintiff from her legal counsel. For details on this case, look at the other blog I edit and write under my law firm’s banner, eDiscoveryLawToday. The article is titled Fishing Expedition Discovers Laptop Cast into Indian River.

BBQ_ComputerIn the other case out of Alabama the employee claiming reverse discrimination decided to get rid of her computer after it looked like a forensic examination would be ordered. Evans v. Mobile Cnty. Health Dep’t, 2012 U.S. Dist. LEXIS 8530 (S.D. Ala. 2012). Rather than throw it in the river, the plaintiff here burned her computer in her backyard. Once again, in my opinion plaintiff’s counsel was asleep at the wheel, or worse. I wrote an article on this one too: Plaintiff’s Backyard Summer BBQ of Her Computer Leads to Sanctions. The truth is always stranger than fiction. Read the articles and cases for yourself to see what is going on in litigation today. The tide is changing. E-discovery is starting to be some defense counsel’s best friend.

Boomerang Effect

boomerangElectronic discovery specialist have explained for years now to attorneys who attend e-discovery CLEs what I call the boomerang effect. That you cannot take action to enforce e-discovery, to compel and to sanction, unless your own house is in order. You may throw the sanctions boomerang at the plaintiff only to have it come back at you. Just ask Apple in the Samsung patent case last years where both sides ended up with spoliation sanctions and countervailing adverse inferences. Apple v. SamsungU.S.D.C., N.D. Ca., Case No.: 11-CV-01846-LHK, Document 1894, Filed 08/21/12.

Awareness of the boomerang effect is one way we motivated and cajoled attorneys to get their clients to take preservation of ESI more seriously. If you do not want the other side to get away with breaking the rules, than you need to be very sure you are following the rules yourself. In the case of large organizations this mean spending money on people, e-discovery teams, and technology. 

The good new is this instruction has finally worked. It has taken over a decade, but the attendees of these many CLEs, including thousands of in-house counsel, have heard and have taken action. Many corporations have now made substantial investments. Even though their ESI is still growing at an exponential rate, and is still disproportional to David’s, they have their act together. Their e-discovery teams can save, find and produce, thanks to the new technologies of the vendors, and the new skills of many lawyers.

Where Are All The Plaintiffs Counsel?

Magritte_detailI have always been willing to teach David’s attorneys too, and so have others from both sides of the bar, but most plaintiffs counsel have not cared about e-discovery. In my view the plaintiffs’ bar has not been trying as hard as the defense bar. Now, as a whole, they have fallen way behind.

Plaintiff’s counsel have always been a no-show at e-discovery education events, despite the fact that several of the country’s top experts are openly plaintiff. You have only to think of Bill Butterfield or  Ariana  J. Tadler. I admit that I was once openly defense, but have tried to go neutral since 2006. Unlike some others in the field who have gone neutral, or attempted to do so, I am aware that I may still be somewhat biased. (Maybe even more than somewhat, but at least I am working on it.) With this perhaps tempering bias in mind, I cannot help but think that the plaintiffs Bar has created this problem, where the pendulum has now begun to swing against them. They have done so by their own laziness and lack of foresight.

Harsh words I know, and of course there are many exceptions (for instance a large plaintiffs firm in my locale, Morgan & Morgan, insisted that one of their new hires take my course at the University of Florida), but how else do you explain that at every single CLE event I have ever attended since 2006, over 90% of the attendees have been defense oriented. The truth is, when I ask the question at CLEs: how many of you primarily do plaintiffs work; usually only one of two hands go up, sometimes none.

I have talked to other CLE presenters and  sponsors about this. They see the same thing. The consensus is that plaintiffs counsel rarely attend e-discovery educational CLEs, even when they are free. Of course, as mentioned, there are exceptions to this general rule; some plaintiff firms understand very well the importance of e-discovery and continued legal education. They are privately chagrined that their colleagues do not seem to be listening to them. The truth is, the vast majority of plaintiffs counsel just do not get it.

Not yet anyway. After they start losing cases because of sanctions, like corporate Goliaths did just a few years ago, the rest of the plaintiff bar will wake up. More personal sanctions of plaintiffs counsel will also help them to wake up. Then they will start insisting that their individual clients do the right thing, that small plaintiffs preserve all of the evidence, not just the ESI that supports their position. I predict that even after a series of cases like this, and even after the plaintiffs bar changes its attitude and starts to take e-discovery education seriously, that it will take some time to improve the skills of that many lawyers. (There are more plaintiffs lawyers than defense lawyers, including the solos and part-timers.) In the meantime, defense counsel may finally have the edge for a while. So too will the few plaintiff’s counsel that already get it. But is this a good thing? Does it serve justice?


Ralph Losey 2012 abstractThe sudden emergence of the common Man as a holder of too-much-information has caught most plaintiffs attorneys by surprise. They are unprepared and vulnerable to defense counsel boomerangs. Some defense counsel may celebrate and gloat about this turn around, but I do not. I am concerned about the big picture. I see danger in this imbalance of legal skills. I see danger to the rights of our citizens to obtain equal justice under the law. I also see danger in allowing destruction of evidence by any party to go unchecked. There is no easy answer.

Sanctions for spoliation is no longer a weapon available only to a few sophisticated counsel in the plaintiffs bar. It is now also available to many, if not most large corporations and their attorneys. There are a growing number of highly trained defense attorneys who are no longer afraid of the boomerang effect. They are free to fight the fraudulent or negligent withholding of evidence on the part of those who sue them. Many large organizations today have their house in order and are preserving all of the evidence that proportionality demands. This is good news for those concerned about e-discovery extortion and settlements forced upon defendants. I certainly celebrate this. It is a goal I have long worked towards. But are we now in danger of excess in the other direction due to the general lack of skills by the plaintiffs bar?

Cases should be decided on the merits, on the evidence.  I do not want plaintiffs to have to settle for too cheap, or lose their meritorious case, just because their attorneys did not know e-discovery, did not know they should make sure their clients preserve evidence, did not know how to make a production. It is not good for our system of justice when this happens to any party, defendants or plaintiffs. That is part of the big picture. But the other part is that the truth must always prevail. Plaintiffs must not be allowed to destroy evidence any more than defendants. If evidence is destroyed by anyone, it is much harder to know the truth. If evidence is destroyed or altered, then the responsible party should be sanctioned. So too should their attorneys, if they are to blame. There is no one-free-bite for spoliation, be they defendant or plaintiff.

sanctionsSanctions should be imposed regardless of whether it is the plaintiff or defendant who is guilty. Sanctions of erring parties, and their attorneys, can have a strong deterrent effect. It can motivate all litigants and their attorneys to take the time and effort needed to marshal and preserve evidence. It can also motivate attorneys to attend CLEs and really learn and practice these new skills. Preservation of evidence is a serious matter for both sides of the “v.” Punishment by sanctions is a strong deterrent of both intentional bad faith actions, and negligent actions. In either event, the truth has been lost, justice has been tainted.

Fear of sanctions is an important motivator. That is one reason I suggest the bar be very careful about any amendment to the rules that weakens a judge’s discretion to sanction a party for deletion or alteration of evidence. The defense bar especially should be careful about what it asks for in the way of rules reform. They should think twice about pushing new rules that make it harder for judges to punish litigants for wrongful conduct, especially in the area of ESI preservation. They may end up hoisted by their own petard.

The Simon Property Group and Evans cases were easy facts, but many other cases of plaintiff spoliation are not. It was clear the plaintiffs in these cases did the deeds themselves and did them in bad faith. The harsh sanctions were appropriate. The court’s discretion in this area  was properly exercised.

I continue to believe that this judicial discretion to punish where appropriate, either the litigant, or their attorney, or both, should be preserved and strengthened, not hindered. I continue to prefer legal education and technology as a solution to the preservation problem, not a weakened judiciary. We have a delicate balance here, one that requires wisdom and good judgment by our judiciary. Too many new rules may upset the balance we now have.

%d bloggers like this: