e-Discovery Team

This is the sixth of seven informal video talks on document review and predictive coding. The first video explained why this is important to the future of the Law. The second talked about step one, ESI Communications. The third about step two, Multimodal Search Review. The fourth about step three, Random Baseline. The fifth about steps four, five and six, the predictive coding steps that iterate during rounds of machine training.

This video talks about step seven, ZEN Quality Assurance Tests, where ZEN stands for Zero Error Numerics. It is the quality control step that includes metrics and a number of other techniques to reduce and catch errors.

The specific test we use on a predictive coding stop decision, one that we have found measures recall the best, is ei-Recall. It is a second random sample of all of the negatives in the dataset, the documents that you presume irrelevant and will not be human reviewed. The ei-Recall formula calculates recall as a range. The sampling test also contains an accept on zero error component, where the presence of even a single Highly Relevant document as a false positive automatically triggers more rounds of machine training. I have explained this in detail in Introducing “ei-Recall” – A New Gold Standard for Recall Calculations in Legal Search – Parts One, Two and Three. The formula and technique is also set forth on one page at ZeroErrorNumericcs.com.

Another important quality control technique, one used throughout a project, is the avoidance of all dual tasking, and learned, focused concentration, a flow-state, like an all-absorbing video game, movie, or a meditation.

Speaking of relaxed, thought free, flow state, did you know that United States Supreme Court Justice Stephen Breyer is a regular meditator? In a CNN reporter interview in 2011 he said:

For 10 or 15 minutes twice a day I sit peacefully. I relax and think about nothing or as little as possible. … And really I started because it’s good for my health. My wife said this would be good for your blood pressure and she was right. It really works. I read once that the practice of law is like attempting to drink water from a fire hose. And if you are under stress, meditation – or whatever you choose to call it – helps. Very often I find myself in circumstances that may be considered stressful, say in oral arguments where I have to concentrate very hard for extended periods. If I come back at lunchtime, I sit for 15 minutes and perhaps another 15 minutes later. Doing this makes me feel more peaceful, focused and better able to do my work.”

Apparently Steve Breyer also sometimes meditates with friends, including legendary Public Interest Lawyer, Professor and meditation promoter, Charles Halpern. Also see Halpern, Making Waves and Riding the Currents (2008) (his interesting autobiography); Charles Halpern on Empathy, Meditation, Barack Obama, Justice and Law (YouTube Interview in 2011 with interesting thoughts on judicial selection).

Document review is not as stressful as a Supreme Court oral argument, but it does go on far longer. Everybody needs to relax with a clear mind, and with focused attention, to attain their peak level of performance. That is the key to all quality control. How you get there is your business. Me, in addition to frequent breaks, I like headphones with music to help me there and help me to stay undistracted, focused. So, sit comfortably, spine erect, and enjoy this moment of ZEN from sunny Florida in February.

_________

____

For details on step seven see ZeroErrorNumericcs.com. For information on all eight steps see Predictive Coding 3.0. More information on document review and predictive coding can be found in the fifty-six articles published here.

This entry was posted on Friday, March 18th, 2016 at 12:49 pm and is filed under Lawyers Duties. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.