Here are my top ten predictions for what will happen in 2014 in our world of electronic discovery. My tenth prediction, which is not necessarily the least important, is tied to Malcolm Gladwell’s new book, David and Goliath: Underdogs, Misfits, and the Art of Battling Giants. This final prediction sets the stage for my next blog, where I will tackle lofty issues of life, justice, and the legal profession. That’s right, next week I will show how e-discovery lawyers are the new Davids of the legal profession. But for now here are my predictions for the new year.
1. More plaintiffs will lose cases as a result of sanctions for ESI spoliation and other e-discovery wrongdoing. The decision of the Fifth Circuit Court of Appeals, Moore v. CITGO Refining & Chemicals Co., LP, is a herald of things to come. There will be many more like this in 2014 by lower courts all over the country. Moore v Citgo shows the defense potential for the proactive use of e-discovery and the importance of proper preservation. Also see: Announcing My Top e-Discovery Case of 2013.
In 2014 we will clearly see that motions for sanctions are no longer just a weapon for plaintiff’s counsel against the dimwitted. Plaintiffs now have too much ESI too. Moreover, the plaintiffs bar as a whole is way behind the defense in e-discovery competence, including ESI preservation.
2. The proposed amendments to the Rules will be accepted by the Supreme Court, even 37(e). All of the proposed revisions to e-discovery related rules of federal civil procedure are set forth on my page listed above, Proposed Rules. They are all acceptable, and after some bickering on 37(e), they will all be adopted by the Supreme Court. Also see the blog I wrote in 2012: Georgetown Part Two: New Rules Are Coming!
The Judicial Conference’s Committee on Rules of Practice and Procedure engineered a series of compromises to come up with these proposed amendments. They seem to think that it is a big improvement, and will make a huge difference. Many doubt that very much, saying the proportionality principles are already in the rules, and the overly complex language of 37(e) is vague and will not make much difference.
Personally, I like the changes to relevance in Rule 26. The impact of proportionality principles is now far clearer than before. I have long advocated for such a change in the rules and am happy that the Committee finally took action. Losey, R., Rethinking Relevancy: A Call to Change the Rules to Narrow the Scope of ESI Relevance (Jan. 21, 2011); Rethinking Relevance: A Call to Modify the Rules of Discovery (February 2011); Call to Modify the Rules chapter in my book, Adventures in Electronic Discovery (West Thomson Reuters, 2011).
I am not as thrilled by the proposed amendment to 37(e). The part of 37(e) that concerns me (and others) the most is subdivision (e)(1)(B)(i):
(B) impose any sanction listed in Rule 37(b)(2)(A) or give an adverse- inference jury instruction, but only if the court finds that the party’s actions: (i) caused substantial prejudice in the litigation and were willful or in bad faith
What does willful mean? This will be litigated for years. Further, were the revisions really intended to immunize a party from sanctions whose preservation efforts were grossly negligent? If so, what in the world was the Committee thinking? For one thing, they did not seem to be thinking about a lawyer’s duty of competence.
They also did not seem to be thinking about the Constitution, especially Separation of Powers. I tend to agree with former Judge Ron Hedges who thinks this proposed court rule crosses too far into substantive law. See Rules Enabling Act, 28 U.S.C. § 2072 (1994) (rules shall not abridge, enlarge or modify any substantive right). Also see eg. Kelleher, Taking Substantive Rights (in the Rules Enabling Act) More Seriously, 74 NDLR 1 (1998).
In fact, I agree with almost all of the criticisms written about 37(e) from both independent bar groups, as well as the one-sided groups. They are all right. Although I think the existence of Rule Committee lobbyists, coupled with all of the political intrigue surrounding these revisions, supports the unconstitutional argument on 37(e).
Moreover, the lobbyists seem to be unaware of the trend behind my first prediction. They do not understand that 37(e) could become a 007 license to kill for witless plaintiffs and their Clouseau-like counsel.
Still, as imperfect as it is, I think 37(e) will be accepted, although it is my hope that the comments section will at least be improved. But that is more of a hope than a prediction.
3. A lawyer will finally get sued for failing to ask for a 502(d) order. In 2013 many judges at CLE events said dozens of times that it was malpractice for a lawyer not to ask for a 502(d) order. I for one applauded these motivational speeches. This year the statements will bear fruit, but not exactly what was intended. Someone will finally sue a lawyer who did not get a 502(d) order. The most interesting part of the case will be the deposition of the trial judge from the first action on whether he would have granted the motion for a 502(d) order, had one been filed. As a sub-prediction, in 2014 we will see more and more judges enter standing orders under 502(d). After all, they don’t want to be deposed.
4. We will finally have a new public search dataset to replace Enron. Super search science nerds will be deliriously happy about having new data to run experiments on, and so, true to nerdy form, will name the new database after a confusing, but delicious fruit. Software demo pros will not be happy. They will have to actually learn something new to show off their latest versions of software.
I also predict that certain sex-obsessed search nerds (you know who you are) will begin by searching the new email collection for porn, exposed body parts and such. You know that such cultural icons have got to be in any real data set of email. I also predict that the first searches for same will use the key words spin the bottle, and they will fail.
The roll-out of a new test set of emails will in turn spawn discovery of other new data sets. Hey, it’s just another asset in bankruptcy. One day we will all be puzzled by the past myopic view of scientific testing, based as it was on one two very old document collections: the dirty old tobacco collection created in the day when faxes were new; and, the other collection of whiny emails from the debacle of a business called Enron. Hey, hasn’t anyone noticed we are in a data explosion? Bring on the new data.
5. More lawyers will figure out that the so-called gold standard of lawyer review of every document was wrong, that it was based on a myth of human infallibility. The delusional thinking still infecting most of the profession in 2103, that the best way to find relevant documents is for humans to read them all, will seriously erode.
Note that I do not say come to an end. Myths like the gold standard of attorney review of each document die very hard. In fact, even some search specialists, who should certainly know better, seem to forget how untrustworthy humans are. They ignore the many studies that prove how inconsistent human reviewers are, even when using search experts. See Less Is More, parts One, Two and Three. They still try to fix the old methods, and try to use human reviewers to measure what final recall and precision testing results were achieved by automation.
This must change. We must devise new, more realistic methods for quality control testing. The methods should not be tainted by false myths of human infallibility. When it comes to scientific result testing, math and probability analysis using comparative analysis – relativity – should be used, not armies of humans sampling inconsistencies.
My experience with search of big data suggests that absolute measures of recall are impossible in large scale legal reviews. So, instead of looking for hard numbers on recall, of F1, we should think probability and relativity. When doing research, testing, and experiments with search we should focus on the relative comparisons of results of differing search methods, different search teams. In practice, we should think of many different kinds of quality control measures, and not just limit testing metrics to recall and precision. They are just probability estimates anyway, not hard cold absolutes. So why treat them as such? In both science experiments and legal practice we must, of course, still have humans involved in the quality control role. But they should all be motivated experts, SMEs, and even then, they should play a supporting role, a training role, not a dispositive one.
Someday, but not yet in 2014, we will all realize that when it comes to legal reviews of large dataset, human reviewers are all too human, especially unpaid volunteers, who often are not even trained search lawyers. To use another of my favorite clichés, you can’t make a silk purse out of a sow’s ear. That is what final human reviews to measure recall of predictive coding amounts to.
Absolute recall and precision measurements are like mysterious elementary particles. If you try to observe them, they will change. I call this the fuzzy lens phenomena of big data in my Secrets of Search essays. Let us all get real and realize relativity. It is all probability now, not Newtonian certainty. It is about the math, not the matter. At least the post-Heisenberg scientists who design the experiments should get that.
6. More lawyers will figure out that predictive coding is for real, not just another marketing gimmick. Another delusion confusing the legal profession will also seriously erode in 2014, that predictive coding is just another marketing hype.
This delusion is easier for me to understand, after all, our friends the vendors have such clever marketing departments. We are used to the inflated promises, the exaggerated hype. Most lawyers do not believe a word of what any vendor says. (Some, those who have taken too many depositions and studied memory, do not believe a word of what anyone says!) After all, we lawyers are practiced at the art of deception.
We were totally burned by the last wave of exaggerated search promises of six years ago; remember, when concept search was heralded as the next big thing? It wasn’t. We were disappointed, and so as a group even techie lawyers just assumed that predictive coding was another false promise. It isn’t. More and more lawyers will get that in 2014.
7. More lawyers will figure out that predictive coding is another feature in search, one that it adds onto all of the other search features, not replaces them. In other words, more lawyers will start using multimodal search. 
Predictive coding, meaning to me artificial intelligence enhanced search (active machine learning), may be on top of the search pyramid, but it is, after all, just another way to search. The value of all of the other methods has not suddenly disappeared! The false notion that if you use predictive coding, then you cannot use any of the other search features, will come to an end.
More lawyers will start to say: Gee, I thought it was an either-or proposition. I had no idea you could use predictive coding AND other search functions. Yes, multimodal search will become more commonly used, and as a result the average project recall and precision number calculations will go way up. Yes, the both-and way of thinking, which is essentially foreign to lawyers, will finally be accepted. Of course, the poor software demos guys, already suffering terribly from having to use another data set, will have to work even harder to try to demo multimodal.
Note that I do not predict that the hybrid aspects of my search techniques will catch on in 2014, much less the proportional bottom line driven parts. I only predict that multimodal will catch on next year. We cannot expect the profession to change that fast.
8. More big name lawyers in e-discovery will jump ship and join new firms or companies. Last year the big name to change jobs was Jason R. Baron, who had been counting the days until his retirement from the federal government for years. There were a number of other prominent attorneys who also made a move in 2013.
This trend will continue in 2014. It might even accelerate as e-discovery experts get tired of being taken for granted by their current firms or corporations. Special expertise in this field is in demand as more and more legal organizations begin to understand the importance of technology in general, and e-discovery in particular. The Davids will finally get their due as the true Goliath killers they are (see prediction ten and the conclusion below).
9. More e-discovery vendors will fail. There are still too many marginal e-discovery vendors. More are likely to end in 2014. Maybe that will include another big name, like FIOS in late 2012. Maybe it will just be regional players, some of whom are merely glorified copy shops with no real expertise. But more than likely, the companies failing in 2014 will also include a few of the more established vendors that are not keeping up with the times. They were built for e-discovery as it existed at the turn of the century. Their expertise is outdated.
The companies that fail next year will either just fade away, or, more likely, get bought up by another company hoping to grab their remaining customers. Some of these will be disguised as a merger. It is unlikely that the companies closing next year will have any technology of value to sell.
10. More of the big Goliath corporations that dominate the law technology scene will realize that they are doomed and try to change, to down-size. The truth is, most of the Goliaths that dominate the legal technology world are already dead, they just don’t know it yet. They were hit in the head in late 2013 by a little shepherd boy in a battle against the big review teams. Big is not better. Less Is More: When it comes to predictive coding training, the “fewer reviewers the better.” The Goliaths can hide for a while, but they can’t run. Eventually the truth will get out. Some understand this and are trying to change, to reinvent themselves. Most do not. They think they are too big to fail.
They do not understand that the little guy, the underdog, has an advantage over the giants. Davids can be creative, innovative, and move fast. Goliaths cannot. They can merely prepare for last generation’s linear battle. See: David and Goliath: Underdogs, Misfits, and the Art of Battling Giants, by Malcolm Gladwell (2013). My next blog will say more about this important story and how it relates to law and e-discovery.
Ralph
I think you are right on track with your predictions and I look forward to next week’s blog.
I am a little surprised, however, that your first prediction regarding Moore v Citgo will have taken so long. I had thought it would have happened much sooner after Pension Committee. Although the buzz on that case involved the written litigation hold, recall that the sanctioned parties there were all Plaintiffs. Moreover, they had all performed the equivalent of a “targeted data preservation” which is really common today in e-discovery cases. In other words the Plaintiffs preserved the documents they thought were relevant. It was not until much later when they received the defendant’s defenses and production requests that they realized there was another group of documents that were also relevant but they no longer had them. Had the Plaintiffs performed Forensic Grade Imaging instead of Targeted Data Preservation they would have likely still had the documents requested by the defendants.
You may also recall an article from November 2010 that was published in the Duke Law Journal, “Sanctions for E-Discovery Violations: By the Numbers”, by Dan Willoughby, Rose Jones and Gregory Antine. As I recall of the approximate 400 cases that they examined about half involved preservation issues and about a quarter of those involved Plaintiffs.
So, I have long thought that it was only a matter of time before defense counsel wised up and attacked the preservation procedures of Plaintiffs more aggressively. Even if they did not pursue sanctions, I have thought that Defense counsel would attack the authenticity of damaging documents when they were preserved using a Targeted Data Acquisition, since in a Targeted Data Acquisition one does not have the necessary system metadata to make sure that the media from which the document was harvested was not a counterfeit—I encounter counterfeit media or at least doctored media quite often. Instead, lawyers tend to authenticate documents with witnesses claiming knowledge of the document and not experts authenticating the media from which the document came.
I hope you are correct about your fifth prediction. It is about time, although the old boots on the ground approach still seems alive and well to me. It will likely take clients to realize what is really happening to them and either complain or file a grievance under one of the professional standards. There is the new comment in the model ABA rule that expects lawyers to be more technology competent and in some states there is greater expectation of lawyers to bill fees that are more reflective of the case’s requirements. In other words, if technology assisted review, even if it was just some form of keyword search, could have been more efficient and effective than the absurdly stupid eyes on every document approach then there could be the basis for a complaint when the lawyer chose the attorney’s eye on every document approach.
I am also very pleased with your seventh comment. I have read a lot of your blogs on predictive coding. I was beginning to think that you were pushing predictive coding as a technology providing the kind of precision that one sees in machining fasteners in the aerospace industry where tolerances can be in the millionths of inch. Even with your push for single SME usage predictive coding will never be a silver bullet for several reasons. First there is always the judgmental issue between SMEs. Second, even an SME’s theory of the case could change from the start of the discovery and the selection of the training sample to what it might be 6 months or a year into discovery where the training set might need to look very different. So, the precision desired with subject matter expertise might not be that great of a benefit if it misses the boat in the end. Thirdly, there is sampling error. The confidence and precisions typically used are only 95 percent plus or minus 2.5 percentage points precision. So, even if all of the technical issues about statistical sampling are handled properly by the litigation team and I have my concerns, there is still a 7.5 percent chance that the training set is not representative of the population or the conclusions being reached about the population.
So, I am very much in agreement that lawyers should see predictive coding as quite a weapon but they should also see it as only one weapon in a vast arsenal of different methodologies that includes other search techniques, analytics, multi-stage discovery or proportionalism, better 26(f) conference planning and many others. Furthermore, each case should likely see implementation of many of these different techniques. Only the mixes and implementation of the different technologies will differ between matters.
Happy New Year.
Ralph, this is a pretty credible list.
Regarding the 502(d) issue, besides it being a risk manager, it saves money. Despite the fact that judges and others on CLE panels are teaching it, we are all blogging or whitepapering it http://www.d4discovery.com/2013/11/whitepaper-the-importance-of-federal-rule-of-evidence-502/ , and the courts are putting out model orders, we see relatively little adoption by litigants in their exchange of discovery protocols. How can we increase adoption? Should they be including a 502(d) agreement and order as part of their exchange of discovery protocols, or should they present it as a separate stipulation and order for the court to sign?
BTW, I think for a #11, there will be at least some action in the courts regarding eDiscovery imperatives, privacy rights and BYOD.
All the best,
–Chuck
Regarding item #9 I wouldn’t characterize FIOS’s acquisition by DTI, an acquisition that was part of a very normal business cycle consolidation, as a business failure. FIOS did not declare bankruptcy.
[…] Top Ten e-Discovery Predictions for 2014 Posted on January 1, 2014 by Ralph Losey […]