Welcome to the e-Discovery Team’s training course on TAR (Technology Assisted Review).
The term “TAR” as we use it means electronic document review enhanced by active machine learning, a type of specialized Artificial Intelligence. Our method of AI-enhanced document review is called Hybrid Multimodal IST Predictive Coding 4.0. By the end of the course you will know exactly what this means. You may even grok the above graphic.
Here is the introductory video by Ralph Losey welcoming you to the TAR Course. Ralph makes multiple video appearances throughout the course. More videos will be added from time to time to keep the materials current. Students are invited to leave comments and questions at the bottom of each class.
The TAR Course has Seventeen Classes:
- First Class: Background and History of Predictive Coding
- Second Class: Introduction to the Course
- Third Class: TREC Total Recall Track, 2015 and 2016
- Fourth Class: Introduction to the Nine Insights from TREC Research Concerning the Use of Predictive Coding in Legal Document Review
- Fifth Class: 1st of the Nine Insights – Active Machine Learning
- Sixth Class: 2nd Insight – Balanced Hybrid and Intelligently Spaced Training (IST)
- Seventh Class: 3rd and 4th Insights – Concept and Similarity Searches
- Eighth Class: 5th and 6th Insights – Keyword and Linear Review
- Ninth Class: 7th, 8th and 9th Insights – SME, Method, Software; the Three Pillars of Quality Control
- Tenth Class: Introduction to the Eight-Step Work Flow
- Eleventh Class: Step One – ESI Communications
- Twelfth Class: Step Two – Multimodal ECA
- Thirteenth Class: Step Three – Random Prevalence
- Fourteenth Class: Steps Four, Five and Six – Iterative Machine Training
- Fifteenth Class: Step Seven – ZEN Quality Assurance Tests (Zero Error Numerics)
- Sixteenth Class: Step Eight – Phased Production
- Seventeenth Class: Conclusion
With a lot of hard work you can complete this online training program in a long weekend, but most people take a few weeks. After that, this course can serve as a solid reference to consult during your complex document review projects. It is also recommended that you follow Losey’s personal blog, e-DiscoveryTeam, to stay current.
We call our latest version of AI enhanced document review taught here “Predictive Coding 4.0.” We call it version 4.0 because it substantially improves upon and replaces the methods and insights we announced in our October 2015 publication – Predictive Coding 3.0. In the First Class of the TAR Course we explain the history of predictive coding software and methods in legal review, including versions 1.0 and 2.0. Unfortunately, most vendors are still stuck in these earlier methods. If you have tried predictive coding and did not like it, then the probable reason is that you used the vendors recommended, but wrong method. Either that, or the software was to blame, but it is probably the method. Many lawyers report that they attain better results when they follow their own methods, not the vendors default methods.
Most vendors are still promoting use of random based control sets based on a misunderstanding of statistics and search. The use of control sets is simply wrong and a waste of time. We never saw any of these same vendors at TREC and for good reason. They do not keep up with the latest developments in search science. They are a business. We are not. The e-Discovery Team is a group of lawyers, lead by Ralph Losey, a practicing attorney. We are lawyers sharing what we know with other lawyers (and vendors).
We offer this information for free on this blog to encourage as many people as possible in this industry to get on the AI bandwagon. Predictive coding is based on active machine learning, which is a classic, powerful type of Artificial Intelligence (AI). Our Predictive Coding 4.0 method is designed to harness this power to help attorneys find key evidence in ESI quickly and effectively.
We Follow the ‘Hacker Way’ Philosophy
Before the classes begin, we want to share the philosophy of the e-Discovery Team, and the TAR Course, a philosophy that we have in common with Facebook, Google and most other high-tech companies called the Hacker Way. See Eg. HackerWay.org.
The Hacker Way – often called the hacker ethic – has nothing to do with politics or criminal activities. It is the philosophy of the computer age. This credo has influenced many in the tech world, including the great Steve Jobs and Steve’s hacker friend, Steve Wozniak, the laughing Yoda of the Hacker Way. The Hacker approach is primarily known to software developers, but can apply to all kinds of work. Even a few lawyers know about the hacker work ethic and have been influenced by it.
To quote the explanation of this philosophy given by Facebook’s founder, Mark Zuckerberg, in his Letter to Investors for the initial public offering in 2012:
The word `hacker’ has an unfairly negative connotation from being portrayed in the media as people who break into computers. In reality, hacking just means building something quickly or testing the boundaries of what can be done. Like most things, it can be used for good or bad, but the vast majority of hackers I’ve met tend to be idealistic people who want to have a positive impact on the world.
The Hacker Way is an approach to building that involves continuous improvement and iteration. Hackers believe that something can always be better, and that nothing is ever complete. They just have to go fix it — often in the face of people who say it’s impossible or are content with the status quo. . . .
Hackers try to build the best services over the long term by quickly releasing and learning from smaller iterations rather than trying to get everything right all at once. . . . We have the words `Done is better than perfect’ painted on our walls to remind ourselves to always keep shipping. . . .
Hacking is also an inherently hands-on and active discipline. Instead of debating for days whether a new idea is possible or what the best way to build something is, hackers would rather just prototype something and see what works. There’s a hacker mantra that you’ll hear a lot around Facebook offices: ‘Code wins arguments.’
Hacker culture is also extremely open and meritocratic. Hackers believe that the best idea and implementation should always win — not the person who is best at lobbying for an idea or the person who manages the most people.
Zuckerberg, Letter to Investors (1/31/12). The e-Discovery Team has long endorsed the nine basic principles of the Hacker Way set forth in the diagrams below.
See: Losey, “The Hacker Way” – What the e-Discovery Industry Can Learn From Facebook’s Management Ethic (8/18/13); The Solution to Empty-Suits in the Board Room: The “Hacker Way” of Management – Part One and Part Two (8/22/13). Also see: HackerWay.org.
The problem with legal technology is the debates often go on for years, not days. We are against that. We just do it. We have broken many things to get to this point and fixed many more. Our processes are still not perfect, but they keep improving. In the meantime, we keep shipping, we keep making phased productions, we keep improving the TAR Course. Iteration is the essence of both machine learning and creative work. Openness is also part of our core values. Thus we share most of what we learn in the TAR Course and e-Discovery Team Training. On the e-Discovery Team blog we announce from time to time the continuous improvements we make to these programs.
Mark Zuckerberg in his letter to investors for the initial public offering of Facebook divided the Hacker Way into five components:
Focus on Impact
If we want to have the biggest impact, the best way to do this is to make sure we always focus on solving the most important problems. It sounds simple, but we think most companies do this poorly and waste a lot of time. We expect everyone at Facebook to be good at finding the biggest problems to work on.
Moving fast enables us to build more things and learn faster. However, as most companies grow, they slow down too much because they’re more afraid of making mistakes than they are of losing opportunities by moving too slowly. We have a saying: “Move fast and break things.” The idea is that if you never break anything, you’re probably not moving fast enough.
Building great things means taking risks. This can be scary and prevents most companies from doing the bold things they should. However, in a world that’s changing so quickly, you’re guaranteed to fail if you don’t take any risks. We have another saying: “The riskiest thing is to take no risks.” We encourage everyone to make bold decisions, even if that means being wrong some of the time.
We believe that a more open world is a better world because people with more information can make better decisions and have a greater impact. That goes for running our company as well. We work hard to make sure everyone at Facebook has access to as much information as possible about every part of the company so they can make the best decisions and have the greatest impact.
Build Social Value
Once again, Facebook exists to make the world more open and connected, and not just to build a company. We expect everyone at Facebook to focus every day on how to build real value for the world in everything they do.
Here is Losey’s take on Zuckerberg’s five-fold Hacker Wisdom and how it applies to the Law, to Electronic Discovery in general and Predictive Coding in particular.
Focus on Impact
Build Social Value
For a video by Losey on the application of these five principles to life in general, not just e-discovery and TAR, see HackerWay.org.
Familiarity with these two websites is a prerequisite for this course:
TECHNOLOGY ASSISTED REVIEW (TAR), which is also called Computer Assisted Review (CAR). General Introduction to the e-Discovery Team’s approach to document review using active machine learning, a type of specialized Artificial Intelligence.
LEGAL SEARCH SCIENCE. The Team’s introduction to this new interdisciplinary field. It is concerned with the search, review, and classification of large collections of electronic documents to find information for use as evidence in legal proceedings, for compliance to avoid litigation, or for general business intelligence.
Students are invited to leave a public comment below. Insights that might help other students are especially welcome. Let’s collaborate!
e-Discovery Team LLC COPYRIGHT 2017
ALL RIGHTS RESERVED