For the last nine months I have been working on my firm’s litigation support outsourcing project. As you may have heard, after a grueling Request For Proposal process, Jackson Lewis selected Kroll Ontrack as our firm’s preferred provider of all non-legal, e-discovery services. As a result, my dream of using predictive coding software and related legal arguments in hundreds of my cases, and not just the biggest few, is about to come true. Not only that, but it now makes economic sense for us to use Kroll SAS based software in small cases too.
Under our outsourcing deal our client’s can now use Kroll Ontrack’s well-known computer forensic services, and their advanced software, at prices too low to mention. Suffice it to say, Kroll Ontrack won the RFP on every level, both quality and price. Surprising, I know. But that is one advantage of going all out, and being the first AmLaw 100 firm to do so. We leveraged the size of Jackson Lewis (over 700 attorneys), our high volume of cases (over 2,500 new cases per year), and our growing reputation in the field of e-discovery, to bring our clients Kroll Ontrack quality at discount prices.
My due diligence investigation led to the conclusion that Kroll Ontrack’s Inview software has one of the industry’s strongest predictive coding abilities. They call it Intelligent Review Technology. Also weighing in their favor was Kroll Ontrack’s willingness to customize their procedures and software code somewhat to meet our specifications and practice standards. We can now use their predictive coding enhanced software, Inview, not only in our biggest cases, but also in our medium size cases. We may even be able to use it in some relatively small cases. We can also use their Advanceview software for ECA, review, and small native productions in all size cases, especially the small ones.
The price savings we negotiated inures completely, 100%, to our clients. It is available to all of them, if they use our services and if choose to use Kroll Ontrack. Of course, if in any particular case we do not think Kroll Ontrack is the best vendor for our client in that case, for any reason, we will recommend another vendor. Also, the client is always free to choose the vendor it prefers. Our clients are always our primary concern and our sole duty. Still, in most cases we expect it will be better for them to have us drive a Ferrari at Ford prices, especially when the Ferrari runs on predictive coding type technology that allows us to get the review done faster and cheaper.
In today’s fast-moving technology driven culture, there is a distinct advantage to being first-in. Our new deal with Kroll Ontrack, the first of its kind, and the prices and technology we have attained for our clients by this deal, is another example of this. See eg: “The Hacker Way” – What e-Discovery Can Learn From Facebook’s Culture and Management, and Impactful, Fast, Bold, Open, Values: Guidance of the “Hacker Way.”
E-Discovery Services Are Not Legal Services
The outsourcing deal with Kroll Ontrack allows my firm to provide our client’s with access to the latest Computer Assisted Technology and forensic services, and do so at a good price. But there were many additional reasons for us to go all out with a professional vendor. In fact, there are many good reasons for any law firm to phase out their e-discovery processing and software hosting services. They are not legal services. So why are we selling them?
Vendors cannot provide legal services, neither can so-called technology consultants, even if they are licensed attorneys. Only properly licensed lawyers practicing law can provide legal services. Indeed, the unauthorized practice of law is a crime in most states, albeit usually just a misdemeanor. So the services that e-discovery vendors provide necessarily cannot be legal services. All of them will tell you that, but I suspect a few of them cross the line when a practicing lawyer or judge is not looking.
The one special exception to this involves the use of outside contract lawyers for document review. These are legal services, but they are rendered under the direct supervision of counsel of record, not the vendor. This kind of legal outsourcing is governed by various state rules of ethics and is not what I am referring to here. I am referring to the outsourcing of all of the other non-legal e-discovery services that vendors provide, such as processing, tiffing, software hosting, forensics and the like.
Recent case law on court cost awards is making this dividing line clearer. Standard e-discovery vendor services have been examined in several cases and found to be non-legal services that come within the scope of court costs under 28 USC §1920. See: “Winning isn’t everything, it’s the only thing” – Examining the new trend towards big e-discovery cost awards for winners. For that reason, hundreds of thousands of dollars of vendor costs have been awarded to prevailing parties as costs, namely as exemplification and copying related services. There is a split developing in the Circuits on the point of whether such services come within the scope of 28 USC §1920. Compare: Race Tires America, Inc. v. Hoosier Racing Tire Corp., No. 11-2316, slip op. at *24, *30 (3d Cir. Mar. 16, 2012) with In re Online DVD Rental Antitrust Litig., 2012 WL 1414111 (N.D. Cal. Apr. 20, 2012) ($710,194.23 cost award). But no case has held that the vendor’s e-discovery services are not taxable because they are actually legal services.
Focusing On What Lawyers Do Best
It just did not make sense for our firm to continue to run a highly technical, non-legal business like this as part of our law firm. By going all out, and relying instead on a vendor, Jackson Lewis can now focus exclusively on what we do best, the practice of law. That includes our core services of labor and employment law and e-discovery law.
This move has also eliminated, for us, the many tricky issues facing all law firms who have a litigation support department that charges clients for non-legal e-discovery services. These are complex, highly technical, and expensive services. Many require a great deal of training and education to do correctly. Anyone who thinks it is just a little more difficult than making paper copies is out of touch. Some of these non-legal activities are just as difficult as the practice of law. Some of the computer work involved is very difficult to do correctly. It also requires a very substantial capital investment for equipment and software to run these computer services. Efficient management of these enterprises is also difficult. So is fair billing of these charges to clients.
For all of these reasons, even though my firm is the first AmLaw 100 firm to outsource all of its non-legal e-discovery business activities to a vendor, I expect many others firms will soon follow. So does Kroll Ontrack. So did a whole lot of other vendors who participated in our extensive RFP. They are gearing up now.
Outsourcing Non-Legal Services to e-Discovery Vendors Is Good For Clients and Good For Our System of Justice
Outsourcing makes sense for the non-core functions that are beyond the mission and special expertise of an organization. What could be more non-core for a law firm than non-legal services? This is especially true for complex computer processing, computer forensics and software services. Putting clients’ interests first requires this kind of team work.
Kroll Ontrack responded favorably to my constant public harping in this blog and elsewhere, for inexpensive, advanced software. The volume justified the deal, as did the precedential value. We can now use Advanceview and Inview software in all size cases, not just the biggest. I contend that the significant price move will allow Kroll Ontrack to make up in volume what they have lost in margin. It was a smart business move and win-win for everyone. I suggest that all other vendors be open to similar deals with their customers.
Our negotiations with Kroll Ontrack took nearly nine months, with my primary contact being Chris Wall, an attorney and tech expert. I met with many Kroll Ontrack executives to make this deal, including their President and CEO, Dean Hager. I met with even more of their front line professionals to set up a custom work flow, iron out details, and set up extensive training. I cannot begin to name all of the people I have worked with to make this happen, but I did notice that one of Kroll Ontrack’s leaders, Joel Vogel, has a YouTube video online. So I thought it would be fun to showcase him for special embarrassment. After all, he did put up with several of my outbusts when I was fighting to get even more favorable terms for our clients. Joel is currently Sr. Vice President, Worldwide Legal Technologies, but back in 2010 when this video was shot at Kroll Ontrack studios, he was known as the Chief Service Evangelist.
I have no doubt that other law firms will soon follow our lead with Kroll Ontrack and other vendors. It just makes sense for law firms to focus on the practice of law and leverage the volume of their litigation for the benefit of all their clients.
It is true that some clients do not need this, the few sophisticated mega-corporations that have already made vendor deals leveraging the volume of their e-discovery needs. But most companies have not done so, at least not yet.
Some companies, since they are a business to begin with, and not a law firm, have set up and run internal e-discovery business to serve their own needs. Some companies do this well, but many do not, because again, like law firms, the e-discovery business is not their core competency.
Many of these companies are now realizing their limitations, and the complexity of running e-discovery services, and are also turning to outsourcing. In fact, my due diligence of outsourcing in general showed that in-house law departments are ahead of law firms, and many have already turned to outsourcing. So although we are the first large law firm to outsource all non-legal e-discovery, several corporations have already gone that route, but not many. I predict that will change too, and change fast. Outsourcing non-legal e-discovery services to professional vendors is inevitable for both law firms and corporations. The trend will accelerate with the growing complexity of the technology and amount of data.
Electronic discovery vendors need to help themselves by making it easier for this shift to happen. All full service e-discovery vendors should consider making similar deals with other law firms and corporations for the general benefit of clients everywhere and our system of justice. We need to be able to get to the relevant facts in large volumes of data quickly and inexpensively. We need the new search and review technologies to be affordable for all size cases.
No Slight Intended Against Other Vendors
Please do not construe my comments, nor my firm’s selection of Kroll Ontrack over others, as a slight against all of the other e-discovery vendors and software manufacturers. The RFP had very stringent requirements, not just software excellence. We demanded not only the best software, but also national one-stop-shopping for all non-legal e-discovery services, not just search and review software. This included an area that is very important for us, and our clients, namely forensic analysis and collection services. Very few national vendors could provide all of the non-legal services we needed.
Further, although many software manufacturers now partner with full service vendors, wherein their software is also bundled with other non-legal services, a few do not. One such software vendor is Recommind who makes one of our favorite predictive coding products, Axcelerate. A few other software programs were eliminated for similar reasons.
So although I think Inview is a good product, and am especially impressed with the input that information scientist David D. Lewis has had on the programing design, that does not mean that I think all other competing programs are not good, or not at least as good.
I will say, however, without naming names, that not all software claiming to have predictive coding capacities are created equal. In fact, I did not like some of them. But please remember, as an avid software user and computer hobbyist since 1978, I have strong opinions about all software, not just e-discovery software, and I can spot bs a mile away. Plus, as you may have noticed, I tend to have strong opinions about things, when I have any opinion at all.
Predictive Coding Type Software Features Are Now Critical to the Practice of Law in the Information Explosion Age
There are many fine software programs out there with predictive coding type features. These advanced features, which I call artificial intelligence type features, are now critical to moving the law forward. The recent Rand Report agrees. Where The Money Goes: Understanding Litigant Expenditures for Producing Electronic Discovery (2012); also see my blog on this report.
Just as important are the legal methods now being developed to apply this new technology. These days, that is what I spend most of my time working on. Here is one of the charts that I created to summarize the new legal methods of review.
Last week I spent three days in Eden Prairie, Minnesota, at Kroll Ontrack’s headquarters teaching with several charts like this. With the help of several Kroll Ontrack specialists, I taught forty of my liaisons how to use Inview and its Intelligent Review Technologies (predictive coding). My training always includes discussion of how predictive coding capacities fit into our firm’s core e-discovery legal arguments of proportionality and cooperation.
I had previously had similar sessions with Kroll’s experts, including Joe White, and an outside expert that we retained, Cliff Dutton. We have worked together for several months now to develop very specific firm standards for search and review. In case you don’t know Cliff Dutton, he is one of the original founders of EDRM whom we retained to provide us with independent, non-legal, technical advice on outsourcing, costs, and e-discovery standards. Cliff is the author of eOPS 2010: Electronic Discovery Operational Parameters Survey.
One of the most exciting features of the new technology is that it allows for intelligent document ranking for the first time. This is a key feature of predictive coding that greatly facilitates proportional review. See eg. a few of my recent articles on the topic of proportional review and on predictive coding:
- Bottom Line Driven Proportional Review;
- Predictive Coding Based Legal Methods for Search and Review;
- New Methods for Legal Search and Review;
- Perspective on Legal Search and Document Review;
- LegalTech Interview of Dean Gonsowski on Predictive Coding and My Mission to Make Predictive Coding Software More Affordable;
- My Impromptu Video Interview at NY LegalTech on Predictive Coding and Some Hopeful Thoughts for the Future;
- The Legal Implications of What Science Says About Recall;
- Reply to an Information Scientist’s Critique of My “Secrets of Search” Article;
- Secrets of Search: Parts One, Two, and Three;
- Information Scientist William Webber Posts Good Comment on the Secrets of Search Blog;
- Judge Peck Calls Upon Lawyers to Use Artificial Intelligence and Jason Baron Warns of a Dark Future of Information Burn-Out If We Don’t;
- We Are at the Dawn of a Golden Age of Justice.
As discussed at length in the above articles, predictive coding is far more accurate than the alternatives of manual review or keyword search. It captures far more relevant documents, in other words, has better recall, and does so with much greater precision. Better precision means lawyers waste less time and money in final review of likely relevant documents before production. This means significant costs savings in document review. The Rand Corporation study understood this well. The conclusion of the report states at pages 97-99:
The most promising alternative available today for large-scale reviews is the use of predictive coding and other computerized categorization strategies that can rank electronic documents by the likelihood that they are relevant, responsive, or privileged. Eyes-on review is still required but only for a much smaller set of documents determined to be the most-likely candidates for production. Empirical research suggests that predictive coding is at least as accurate as humans in traditional large-scale review. Moreover, there is evidence that the number of hours of attorney time that would be required in a large-scale review could be reduced by as much as three-fourths, depending on the nature of the documents and other factors, which would make predictive coding one answer to the critical need of significantly reducing review costs. …
Conclusion
Better quality and lower costs. The long elusive promise of technology as applied to e-discovery is finally coming true. That is why predictive coding is so important and why every lawyer would be well advised to get on the bandwagon as soon as possible.
Clients are not stupid, especially sophisticated corporations with intelligent, cost-conscious in-house counsel. Predictive coding technology, when used correctly, saves money on review, perhaps as much as 75% according to the Rand Report. It also fits hand-in-glove with proportionality arguments to counter would-be e-discovery extortionists.
Yet, in spite of this, most litigation attorneys have, until very recently, been afraid to use the new technology for the stated reason that it had never been approved by a court. When I first heard of this popular excuse, I thought back to the good old days of the 1980s and a famous trial lawyer I have always admired, Brendan V. Sullivan. When representing Oliver North who was testifying before Congress about his role in the Iran-Contra affair, Sullivan responded to criticism from Senator Inouye for his aggressive, bold objections with the famous line: “Well, sir, I’m not a potted plant. I’m here as the lawyer.” That line was watched on tv by millions and was famous for years. All lawyers in the 80s admired him for it, so did everyone else. We need more gutsy lawyers like that today in e-discovery, not more e-discovery businessmen and techies.
The excuse of no-judicial approval for predictive coding is now gone. The main excuse still remaining for lawyers to refrain from the adoption of predictive coding methods is the perfectly valid and understandable one that they do not know how to do it. But that can be rectified by time, training and effort. There are many vendors and consultants that can help. I have been writing thousands of words to do my part to help the profession. More brass-tack descriptions will be coming on this blog, plus just read the e-discovery newspapers on popular cases.
We can do it. We are lawyers, not potted plants. Besides, we really have no choice if we want to keep practicing law in the Twenty-First Century. All sophisticated litigants will very soon rise up and demand that their lawyers use these new processes. Either that, or they will get new lawyers. That is the way of the world.
[…] Going “All Out” for Predictive Coding and Vendor Cost Savings […]
Might the term “proportionate review” be better than “proportional review”? The latter suggest that some fixed proportion of the document set is being reviewed; or, when used (as here) in reference to manual review and ranked retrieval, that the probability of a document being sampled for review is some (inverse) proportion of its rank.
[…] multimodal approach is also encouraged by KO, which is one reason we selected KO as our preferred vendor. But not all software vendors and experts agree with the multimodal approach. Some advocate use of […]
[…] employ legal skills), and therefore that using non-lawyers to perform it should not be considered unauthorized practice of the law, but I’ll offer a bottle of wine to the lawyer brave enough to argue that in court on the […]
Ralph,
In your extensive review of vendor offerings did you find that all vendors base their machine learning (i.e., training) on LSI or SVM? I know predictive coding is a process, rather than a technology, but I am just looking for some clarity as to whether there are other approaches to digesting and categorizing the characters on a given page of text. I have read a fair amount about how those algorithms work and I’d like to read up on anything else that is out there.
Also, one common misconception seems to be that the training portion of the predictive coding process involves the consideration of system and application meta-data. Did you find this to be the case in any of the offerings you reviewed? Everything I have read suggests the training process does not involve system or application metadata. Rather, those nuggets of information must be considered separately elsewhere in the process. Thanks.
I’ll try to anser your two Ques:
1. Both, and more. All kinds of systems, which are well proven, just new to legal search. At least that is what I have been told by several academic types with no skin in the game. It is not new unproven technology
2. I think all of the ones I looked at included the basic metadata fields. Not sure where you heard that. Could you send references to the readings you mentioned? Maybe there are some limitations with some software, but I’m not aware of that.
Caveat FYI – I did not look at all vendors or all predictive coding software. Not even close, as I had many other criteria to consider, not just predictive coding. So calling my review “extensive” is an overstatement.
[…] employ legal skills), and therefore that using non-lawyers to perform it should not be considered unauthorized practice of the law, but Ill offer a bottle of wine to the lawyer brave enough to argue that in court on the basis of […]
[…] “All Out” for Predictive Coding and Vendor Cost Savings – http://bit.ly/LyAtIb (Ralph […]
[…] Support (LTN, Nov. 2, 2012); WRECK-IT RALPH: Things in e-discovery that I want to destroy!; Going “All Out” for Predictive Coding and Vendor Cost Savings. Data hosting is a completely different line of work, and is very hazardous in today’s world […]
[…] Support (LTN, Nov. 2, 2012); WRECK-IT RALPH: Things in e-discovery that I want to destroy!; Going “All Out” for Predictive Coding and Vendor Cost Savings. Data hosting is a completely different line of work, and is very hazardous in today’s world of […]
[…] them this month. Losey, R., Five Reasons to Outsource Litigation Support (LTN, Nov. 2, 2012); Going “All Out” for Predictive Coding and Vendor Cost Savings. The five reasons I wrote about in 2012 for outsourcing remain the same for our renewed in 2014, […]
[…] Support (LTN, Nov. 2, 2012); WRECK-IT RALPH: Things in e-discovery that I want to destroy!; Going “All Out” for Predictive Coding and Vendor Cost Savings. Data hosting is a completely different line of work, and is very hazardous in today’s world of […]