Predictive Coding Articles by Ralph Losey (Not Updated or Current)

Articles by Ralph Losey on Electronic Document Review, Legal Search and Predictive Coding

  1. TAR Course: Eighteen Class Training program on how to do TAR (Technology Assisted Review) properly following the Team’s latest method, AI-enhanced document review called Hybrid Multimodal IST Predictive Coding 4.0.
  2. Elusion Random Sample Test Ordered Under Rule 26(g) in a Keyword Search Based Discovery Plan. Random sample quality control procedure recommended in TAR Course, and ZeroErrorNumerics.com, with case analysis of City of Rockford v. Mallinckrodt ARD Inc. 
  3. The Key Players and Play of an e-Discovery Team in a Complex AI-Enhanced Document Review Project. A new way to describe how machine training and ESI reviews are done circa 2018. Essay provides perspective of players in a team approach designed to be as economical and efficient, yet still attain high recall levels. Supplemented by three video-essays: a) Iterated Four-Step Work Flow for Active Machine Training to Help Attorneys Locate Relevant Evidence, b) The Key Players and Play of an e-Discovery Team in a Complex AI-Enhanced Document Review Project, c) True Confessions: The Importance of Project Managers and Details of Second Pass Review in a Complex ESI Review Project.
  4. Analysis of Important New Case on Predictive Coding by a Rising New Judicial Star: “Winfield v. City of New York”. Opinion by SDNY Judge Katharine H. Parker, a new judicial sTar.
  5. WHY I LOVE PREDICTIVE CODING: Making Document Review Fun Again with Mr. EDR and Predictive Coding 4.0. Here is the PDF version of this article. One of Ralph’s all time favorites updated in 2017.
  6. e-Discovery Team’s 2016 TREC Report: Once Again Proving the Effectiveness of Our Standard Method of Predictive Coding.
  7. Lawyers’ Job Security in a Near Future World of AI, the Law’s “Reasonable Man Myth” and “Bagley Two” – Part One and Part Two.
  8. Five Tips to Avoid Costly Mistakes in Document Review: Five video-essays providing pointers to avoid errors in document reviews. January 2017.
  9. Five Reasons You Should Read the ‘Practical Law’ Article by Maura Grossman and Gordon Cormack called “Continuous Active Learning for TAR” (April 2016).
  10. Document Review and Predictive Coding: an introductory course with 7 videos and 2,982 words. (March 2016).
  11. e-Discovery Team at TREC 2015 Total Recall Track, Final Report (116 pg. PDF), and  (web page with short  summary). Also see – Mr. EDR with background information on the Team’s participation in the TREC 2015 Total Recall Track; and,
    Announcing a New Page to the e-Discovery Team Blog: MR. EDR.
  12. Why the ‘Google Car’ Has No Place in Legal Search. Caution against over reliance on fully automated methods of active machine learning.
  13. Predictive Coding 3.0 article, part one and part two. Part one describes the history and part two describes the method. Superseded by Predictive Coding 4.0 article, but still good background.
  14. Concept Drift and Consistency: Two Keys To Document Review Quality, part one, part two and part three.
  15. AI Enhanced Review. Overview of Computer Assisted Review. This is the frequently updated CAR page of the e-Discovery Team blog.
  16. Legal Search Science. Frequently updated description of this new interdisciplinary field of study combining Law, Science and Software Technology.
  17. License to Kull: Two-Filter Document Culling (25 pg. PDF). Predictive coding should be used as [art of a suite of relevance culling tools.
  18. Electronic Discovery Best Practices pages on predictive coding, frequently updated.
  19. Form Plan of a Predictive Coding Project. Detailed Outline for project planning purposes.
  20. Introducing “ei-Recall” – A New Gold Standard for Recall Calculations in Legal Search – Part One, Part Two and Part Three.
  21. In Legal Search Exact Recall Can Never Be Known.
  22. Visualizing Data in a Predictive Coding Project – Part One, Part Two and Part Three.
  23. Introduction to Guest Blog: Talking Turkey by Maura Grossman and Gordon Cormack where they respond to criticisms of their work on predictive coding.
  24. Latest Grossman and Cormack Study Proves Folly of Using Random Search For Machine Training – Part One,  Part Two,  Part Three, and Part Four.
  25. The “If-Only” Vegas Blues: Predictive Coding Rejected in Las Vegas, But Only Because It Was Chosen Too Late. Part One and Part Two.
  26. IT-Lex Discovers a Previously Unknown Predictive Coding Case: “FHFA v. JP Morgan, et al”
  27. Beware of the TAR Pits! Part One and Part Two.
  28. PreSuit: How Corporate Counsel Could Use “Smart Data” to Predict and Prevent Litigation. Also see PreSuit.com.
  29. Predictive Coding and the Proportionality Doctrine: a Marriage Made in Big Data, 26 Regent U. Law Review 1 (2013-2014).
  30. Less Is More: When it comes to predictive coding training, the “fewer reviewers the better” – Parts One, Two, and Three.
  31. My Basic Plan for Document Reviews: The “Bottom Line Driven” Approach, PDF version suitable for print, or HTML version that combines the blogs published in four parts.
  32. Relevancy Ranking is the Key Feature of Predictive Coding Software.
  33. Why a Receiving Party Would Want to Use Predictive Coding?
  34. Comparative Efficacy of Two Predictive Coding Reviews of 699,082 Enron Documents. (Part Two).
  35. A Modest Contribution to the Science of Search: Report and Analysis of Inconsistent Classifications in Two Predictive Coding Reviews of 699,082 Enron Documents. (Part One).
  36. Introduction to Guest Blog: Quick Peek at the Math Behind the Black Box of Predictive Coding by Jason Baron and Jesse Freeman that pertains to the higher-dimensional geometry that makes predictive coding support vector machines possible.
  37. Keywords and Search Methods Should Be Disclosed, But Not Irrelevant Documents.
  38. Reinventing the Wheel: My Discovery of Scientific Support for “Hybrid Multimodal” Search.
  39. There Can Be No Justice Without Truth, And No Truth Without Search (statement of my core values as a lawyer explaining why I think predictive coding is important).
  40. Three-Cylinder Multimodal Approach To Predictive Coding. (Note the e-Discovery Team’s current methods are now explained as four-cylinder, not three. See Predictive Coding 3.0 article, part two.)
  41. Robots From The Not-Too-Distant Future Explain How They Use Random Sampling For Artificial Intelligence Based Evidence Search. Video Animation.
  42. Borg Challenge: Report of my experimental review of 699,082 Enron documents using a semi-automated monomodal methodology (a five-part written and video series comparing two different kinds of predictive coding search methods).
  43. Predictive Coding Narrative: Searching for Relevance in the Ashes of Enron in PDF form for easy distribution and the blog introducing this 82-page narrative, with second blog regarding an update.
  44. Journey into the Borg Hive: a Predictive Coding Narrative in science fiction form.
  45. The Many Types of Legal Search Software in the CAR Market Today.
  46. Georgetown Part One: Most Advanced Students of e-Discovery Want a New CAR for Christmas.
  47. Escape From Babel: The Grossman-Cormack Glossary.
  48. NEWS FLASH: Surprise Ruling by Delaware Judge Orders Both Sides To Use Predictive Coding.
  49. Does Your CAR (“Computer Assisted Review”) Have a Full Tank of Gas?  (and you can also click here for the alternate PDF version for easy distribution).
  50. Analysis of the Official Report on the 2011 TREC Legal Track – Part One.
  51. Analysis of the Official Report on the 2011 TREC Legal Track – Part Two.
  52. Analysis of the Official Report on the 2011 TREC Legal Track – Part Three
  53. An Elusive Dialogue on Legal Search: Part One where the Search Quadrant is Explained.
  54. An Elusive Dialogue on Legal Search: Part Two – Hunger Games and Hybrid Multimodal Quality Controls.
  55. Random Sample Calculations And My Prediction That 300,000 Lawyers Will Be Using Random Sampling By 2022.
  56. Second Ever Order Entered Approving Predictive Coding.
  57. Predictive Coding Based Legal Methods for Search and Review.
  58. New Methods for Legal Search and Review.
  59. Perspective on Legal Search and Document Review.
  60. The Legal Implications of What Science Says About Recall.
  61. Reply to an Information Scientist’s Critique of My “Secrets of Search” Article.
  62. Secrets of Search – Part I.
  63. Secrets of Search – Part II.
  64. Secrets of Search – Part III. (All three parts consolidated into one PDF document.)
  65. Judge Peck Calls Upon Lawyers to Use Artificial Intelligence and Jason Baron Warns of a Dark Future of Information Burn-Out If We Don’t.
  66. The Information Explosion and a Great Article by Grossman and Cormack on Legal Search.

Please contact Ralph.Losey@gmail dot com if you have any questions.

Comments are closed.

%d bloggers like this: