TAR Course

Welcome to the e-Discovery Team’s training course on TAR (Technology Assisted Review).

The term “TAR” as we use it means electronic document review enhanced by active machine learning, a type of specialized Artificial Intelligence. Our method of AI-enhanced document review is called Hybrid Multimodal IST Predictive Coding 4.0. By the end of the course you will know exactly what this means. You may even grok the above graphic.

Here is the introductory video by Ralph Losey welcoming you to the TAR Course. Ralph makes multiple video appearances throughout the course. More videos will be added from time to time to keep the materials current. Students are invited to leave comments and questions at the bottom of each class.

________

The TAR Course has Seventeen Classes:

  1. First Class: Background and History of Predictive Coding
  2. Second Class: Introduction to the Course
  3. Third Class:  TREC Total Recall Track, 2015 and 2016
  4. Fourth Class: Introduction to the Nine Insights from TREC Research Concerning the Use of Predictive Coding in Legal Document Review
  5. Fifth Class: 1st of the Nine Insights – Active Machine Learning
  6. Sixth Class: 2nd Insight – Balanced Hybrid and Intelligently Spaced Training (IST)
  7. Seventh Class: 3rd and 4th Insights – Concept and Similarity Searches
  8. Eighth Class: 5th and 6th Insights – Keyword and Linear Review
  9. Ninth Class: 7th, 8th and 9th Insights – SME, Method, Software; the Three Pillars of Quality Control
  10. Tenth Class: Introduction to the Eight-Step Work Flow
  11. Eleventh Class: Step One – ESI Communications
  12. Twelfth Class: Step Two – Multimodal ECA
  13. Thirteenth Class: Step Three – Random Prevalence
  14. Fourteenth Class: Steps Four, Five and Six – Iterative Machine Training
  15. Fifteenth Class: Step Seven – ZEN Quality Assurance Tests (Zero Error Numerics)
  16. Sixteenth Class: Step Eight – Phased Production
  17. Seventeenth Class: Conclusion

With a lot of hard work you can complete this online training program in a long weekend, but most people take a few weeks. After that, this course can serve as a solid reference to consult during your complex document review projects. It is also recommended that you follow Losey’s personal blog, e-DiscoveryTeam, to stay current.

predictive_coding_4-0_webWe call our latest version of AI enhanced document review taught here Predictive Coding 4.0.” We call it version 4.0 because it substantially improves upon and replaces the methods and insights we announced in our October 2015 publication – Predictive Coding 3.0. In the First Class of the TAR Course we explain the history of predictive coding software and methods in legal review, including versions 1.0 and 2.0. Unfortunately, most vendors are still stuck in these earlier methods. If you have tried predictive coding and did not like it, then the probable reason is that you used the vendors recommended, but wrong method. Either that, or the software was to blame, but it is probably the method. Many lawyers report that they attain better results when they follow their own methods, not the vendors default methods.

Most vendors are still promoting use of random based control sets based on a misunderstanding of statistics and search. The use of control sets is simply wrong and a waste of time. We never saw any of these same vendors at TREC and for good reason. They do not keep up with the latest developments in search science. They are a business. We are not. The e-Discovery Team is a group of lawyers, lead by Ralph Losey, a practicing attorney. We are lawyers sharing what we know with other lawyers (and vendors).

We offer this information for free on this blog to encourage as many people as possible in this industry to get on the AI bandwagon. Predictive coding is based on active machine learning, which is a classic, powerful type of Artificial Intelligence (AI). Our Predictive Coding 4.0 method is designed to harness this power to help attorneys find key evidence in ESI quickly and effectively.

We Follow the ‘Hacker Way’ Philosophy

Before the classes begin, we want to share the philosophy of the e-Discovery Team, and the TAR Course, a philosophy that we have in common with Facebook, Google and most other high-tech companies called the Hacker Way. See Eg. HackerWay.org.

The Hacker Way – often called the hacker ethic – has nothing to do with politics or criminal activities. It is the philosophy of the computer age. This credo has influenced many in the tech world, including the great Steve Jobs and Steve’s hacker friend, Steve Wozniak, the laughing Yoda of the Hacker Way. The Hacker approach is primarily known to software developers, but can apply to all kinds of work. Even a few lawyers know about the hacker work ethic and have been influenced by it.

We Follow Zuckerberg’s Hacker Way

To quote the explanation of this philosophy given by Facebook’s founder, Mark Zuckerberg, in his Letter to Investors for the initial public offering in 2012:

The word `hacker’ has an unfairly negative connotation from being portrayed in the media as people who break into computers. In reality, hacking just means building something quickly or testing the boundaries of what can be done. Like most things, it can be used for good or bad, but the vast majority of hackers I’ve met tend to be idealistic people who want to have a positive impact on the world.

The Hacker Way is an approach to building that involves continuous improvement and iteration. Hackers believe that something can always be better, and that nothing is ever complete. They just have to go fix it — often in the face of people who say it’s impossible or are content with the status quo. . . .

Hackers try to build the best services over the long term by quickly releasing and learning from smaller iterations rather than trying to get everything right all at once. . . . We have the words `Done is better than perfect’ painted on our walls to remind ourselves to always keep shipping. . . .

Hacking is also an inherently hands-on and active discipline. Instead of debating for days whether a new idea is possible or what the best way to build something is, hackers would rather just prototype something and see what works. There’s a hacker mantra that you’ll hear a lot around Facebook offices: ‘Code wins arguments.’

Hacker culture is also extremely open and meritocratic. Hackers believe that the best idea and implementation should always win — not the person who is best at lobbying for an idea or the person who manages the most people.

Zuckerberg, Letter to Investors (1/31/12). The e-Discovery Team has long endorsed the nine basic principles of the Hacker Way set forth in the diagrams below.

See: Losey, “The Hacker Way” – What the e-Discovery Industry Can Learn From Facebook’s Management Ethic (8/18/13); The Solution to Empty-Suits in the Board Room: The “Hacker Way” of ManagementPart One and Part Two (8/22/13). Also see: HackerWay.org.

The problem with legal technology is the debates often go on for years, not days. We are against that. We just do it. We have broken many things to get to this point and fixed many more. Our processes are still not perfect, but they keep improving. In the meantime, we keep shipping, we keep making phased productions, we keep improving the TAR Course. Iteration is the essence of both machine learning and creative work. Openness is also part of our core values. Thus we share most of what we learn in the TAR Course and e-Discovery Team Training. On the e-Discovery Team blog we announce from time to time the continuous improvements we make to these programs.

Mark Zuckerberg in his letter to investors for the initial public offering of Facebook divided the Hacker Way into five components:

Focus on Impact

If we want to have the biggest impact, the best way to do this is to make sure we always focus on solving the most important problems. It sounds simple, but we think most companies do this poorly and waste a lot of time. We expect everyone at Facebook to be good at finding the biggest problems to work on.

Move Fast

Moving fast enables us to build more things and learn faster. However, as most companies grow, they slow down too much because they’re more afraid of making mistakes than they are of losing opportunities by moving too slowly. We have a saying: “Move fast and break things.” The idea is that if you never break anything, you’re probably not moving fast enough.

Be Bold

Building great things means taking risks. This can be scary and prevents most companies from doing the bold things they should. However, in a world that’s changing so quickly, you’re guaranteed to fail if you don’t take any risks. We have another saying: “The riskiest thing is to take no risks.” We encourage everyone to make bold decisions, even if that means being wrong some of the time.

Be Open

We believe that a more open world is a better world because people with more information can make better decisions and have a greater impact. That goes for running our company as well. We work hard to make sure everyone at Facebook has access to as much information as possible about every part of the company so they can make the best decisions and have the greatest impact.

Build Social Value

Once again, Facebook exists to make the world more open and connected, and not just to build a company. We expect everyone at Facebook to focus every day on how to build real value for the world in everything they do.

Here is Losey’s take on Zuckerberg’s five-fold Hacker Wisdom and how it applies to the Law, to Electronic Discovery in general and Predictive Coding in particular.

Focus on Impact

____

____

Move Fast

____

Be Bold

____

Be Open

____


____

Build Social Value

____

____

For a video by Losey on the application of these five principles to life in general, not just e-discovery and TAR, see HackerWay.org.

____

PREREQUISITES

Familiarity with these two websites is a prerequisite for this course:

TECHNOLOGY ASSISTED REVIEW (TAR), which is also called Computer Assisted Review (CAR). General Introduction to the e-Discovery Team’s approach to document review using active machine learning, a type of specialized Artificial Intelligence.

LEGAL SEARCH SCIENCE. The Team’s introduction to this new interdisciplinary field. It is concerned with the search, review, and classification of large collections of electronic documents to find information for use as evidence in legal proceedings, for compliance to avoid litigation, or for general business intelligence.

____

Go on to Class One.

Students are invited to leave a public comment below. Insights that might help other students are especially welcome. Let’s collaborate!

_

e-Discovery Team LLC COPYRIGHT 2017

ALL RIGHTS RESERVED

13 Responses to TAR Course

  1. […] minute video. It now serves as the core video introduction to the e-Discovery Team’s free TAR Course. It is found in the first of the sixteen classes in the Course. I also revised and improved the […]

    Like

  2. […] new enhancements were made to the TAR Course this weekend, including additions and revisions to the written materials, new graphics, new […]

    Like

  3. […] improved the TAR Course again by adding a video focusing on the three iterated steps in the eight-step workflow of […]

    Like

  4. Mary Mack says:

    Ralph, this course is a wonderful gift to the community. Thank you.

    The other gift is your humble apology for the 1.0 protocol. I do not think it necessary as you were first to black letter. Yet an apology for impact is so rare in law or technology that it may persuade those who were turned off by seed set production to try again. I will be recommending your course to our ACEDS community. I’ve heard that EDRM is putting pencil to paper on TAR.

    Like

  5. Ralph Losey says:

    The second word is Then.

    Like

  6. gvc says:

    Ralph,

    Your “four door” photo suggests that CAL uses control sets. Maura and I coined and trademarked the term CAL so that we could control what is described as CAL. No method properly described as CAL uses control sets.

    Please correct this misleading image.

    thanks,
    Gordon

    Like

  7. Ralph Losey says:

    I am relieved to hear that. I have never read you write about control sets. Many used CAL and still use control sets. But I understand your concern and will change right away.

    Like

    • gvc says:

      To be pedantic, if they say they “used CAL” without our endorsement, they did not use the CAL method for technology assisted review.

      Like

  8. […] This is another new video for the e-Discovery Team’s TAR Course. It is included in the new First Class that we just added to the […]

    Like

  9. […] on AI-enhanced document review, new material has been added to the e-Discovery Team’s TAR Course. The new content includes two video lectures that provide examples of applications of the methods […]

    Like

  10. […] three videos in this blog on the Hacker Way are also included in the First Class of the TAR […]

    Like

  11. […] two videos in this blog on the Hacker Way are also included in the Welcome page of the TAR Course. Other minor improvements were made this week to the Welcome and the First […]

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: