Welcome to the e-Discovery Team’s training course on TAR (Technology Assisted Review). To help you get through this advanced training we also have a number of articles on predictive coding and you can search here for more.
The term “TAR” as we use it means electronic document review enhanced by active machine learning, a type of specialized Artificial Intelligence. Our method of AI-enhanced document review is called Hybrid Multimodal Predictive Coding 4.0. By the end of the course you will know exactly what this means. You may even grok the above graphic. Reading the below graphic that uses the new Sans Forgetica font should help you to remember.
By the end of the TAR Course you will understand the importance of using all varieties of legal search, for instance: keywords, similarity searches, concept searches and AI driven probable relevance document ranking. That is the Multimodal part. The Hybrid part refers to the partnership with technology, the reliance of the searcher on the advanced algorithmic tools. It is important than Man and Machine work together, but that Man remain in charge of determining relevance. The predictive coding algorithms and software are used to enhance the lawyers, paralegals and law tech’s abilities, not replace them.
Want to hear an AI podcast talking about the TAR Course? Check the audio out below. We accommodate all types of learners and try to keep education interesting, maybe even make you smile.
By course end you will also know what IST means, literally Intelligently Spaced Training, where you keep training until first pass relevance review is completed, a type of Continuous Active Learning, which Grossman and Cormack call CAL.
The course begins with this introductory video by Ralph Losey welcoming you to the TAR Course. Ralph makes multiple video appearances throughout the course. Students are invited to leave comments and questions at the bottom of each class.
________
The TAR Course has Eighteen Classes:
- First Class: Background and History of Predictive Coding
- Second Class: Introduction to the Course
- Third Class: TREC Total Recall Track, 2015 and 2016
- Fourth Class: Introduction to the Nine Insights from TREC Research Concerning the Use of Predictive Coding in Legal Document Review
- Fifth Class: 1st of the Nine Insights – Active Machine Learning
- Sixth Class: 2nd Insight – Balanced Hybrid and Intelligently Spaced Training (IST)
- Seventh Class: 3rd and 4th Insights – Concept and Similarity Searches
- Eighth Class: 5th and 6th Insights – Keyword and Linear Review
- Ninth Class: 7th, 8th and 9th Insights – SME, Method, Software; the Three Pillars of Quality Control
- Tenth Class: Introduction to the Eight-Step Work Flow
- Eleventh Class: Step One – ESI Communications
- Twelfth Class: Step Two – Multimodal ECA
- Thirteenth Class: Step Three – Random Prevalence
- Fourteenth Class: Steps Four, Five and Six – Iterative Machine Training
- Fifteenth Class: Step Seven – ZEN Quality Assurance Tests (Zero Error Numerics)
- Sixteenth Class: Step Eight – Phased Production
- Seventeenth Class: Another “Player’s View” of the Workflow (class added 2018)
- Eighteenth Class: Conclusion
With a lot of hard work you can complete this online training program in a long weekend, but most people take a few weeks. After that, this course can serve as a solid reference to consult during your complex document review projects. It is also recommended that you follow Losey’s personal blog, e-DiscoveryTeam, to stay current.
We call our latest version of AI enhanced document review taught here “Predictive Coding 4.0.” We call it version 4.0 because it substantially improves upon and replaces the methods and insights we announced in our October 2015 publication – Predictive Coding 3.0. In the First Class of the TAR Course we explain the history of predictive coding software and methods in legal review, including versions 1.0 and 2.0. Unfortunately, most vendors are still stuck in these earlier methods. If you have tried predictive coding and did not like it, then the probable reason is that you used the vendors recommended, but wrong method. Either that, or the software was to blame, but it is probably the method. Many lawyers report that they attain better results when they follow their own methods, not the vendors default methods.
Most vendors are still promoting use of random based control sets based on a misunderstanding of statistics and search. The use of control sets is simply wrong and a waste of time. We never saw any of these same vendors at TREC and for good reason. They do not keep up with the latest developments in search science. They are a business. We are not. The e-Discovery Team is a group of lawyers, lead by Ralph Losey, a practicing attorney. We are lawyers sharing what we know with other lawyers (and vendors).
We offer this information for free on this blog to encourage as many people as possible in this industry to get on the AI bandwagon. Predictive coding is based on active machine learning, which is a classic, powerful type of Artificial Intelligence (AI). Our Predictive Coding 4.0 method is designed to harness this power to help attorneys find key evidence in ESI quickly and effectively.
____
PREREQUISITES
Familiarity with these two websites is a prerequisite for this course:
TECHNOLOGY ASSISTED REVIEW (TAR), which is also called Computer Assisted Review (CAR). General Introduction to the e-Discovery Team’s approach to document review using active machine learning, a type of specialized Artificial Intelligence.
LEGAL SEARCH SCIENCE. The Team’s introduction to this new interdisciplinary field. It is concerned with the search, review, and classification of large collections of electronic documents to find information for use as evidence in legal proceedings, for compliance to avoid litigation, or for general business intelligence.
Go on to Class One.
Students are invited to leave a public comment below. Insights that might help other students are especially welcome. Let’s collaborate!
Ralph Losey COPYRIGHT 2018, 2023
ALL RIGHTS RESERVED
Ralph, this course is a wonderful gift to the community. Thank you.
The other gift is your humble apology for the 1.0 protocol. I do not think it necessary as you were first to black letter. Yet an apology for impact is so rare in law or technology that it may persuade those who were turned off by seed set production to try again. I will be recommending your course to our ACEDS community. I’ve heard that EDRM is putting pencil to paper on TAR.
The second word is Then.
Ralph,
Your “four door” photo suggests that CAL uses control sets. Maura and I coined and trademarked the term CAL so that we could control what is described as CAL. No method properly described as CAL uses control sets.
Please correct this misleading image.
thanks,
Gordon
I am relieved to hear that. I have never read you write about control sets. Many used CAL and still use control sets. But I understand your concern and will change right away.
To be pedantic, if they say they “used CAL” without our endorsement, they did not use the CAL method for technology assisted review.