e-Disco News, Knowledge and Humor: What’s Happening Today and Likely to Happen Tomorrow

June 7, 2015

Spock_smilingMy monthly blogs seems to be getting too heavy, even for me, so this month I am going to try to change. This month I will resort to e-discovery gossip and cheap laughs. I’m hoping that even Spock himself would smile.

But first, a little introspective musing. In February this year, after nine years of writing a weekly blog, I switched to monthly. Since then my blogs have not only been long, complex and difficult, which I warned you would happen, but have also been a tad serious and intellectual. That was never my intent, but it just turned out that way. For instance, my first monthly blog in March, where I started harmlessly enough with a fantasy about time travel and a hack of the NSA, the blog morphed into a detailed outline and slide show on how to do a predictive coding project. Heavy, some might even say boring, well, at least the second half. My next blog was my all time deepest writing ever, where I explained my new intellectual paradigm, Information → Knowledge → Wisdom. I really do hope as many people as possible will read this. It is intended to be insightful, not necessarily entertaining, and certainly not light reading. It went beyond just e-discovery, and law, and ventured into general social commentary.

ZENumericsIn last month’s blog I shared a moment of ZEN, but the moment was filled with math and metrics, not bliss. That’s because in my bizarro world ZEN now means Zero Error Numerics and is designed for quality control in legal search and document review, not Enlightenment. The focus in that blog was on seventeen skills that must be learned to master the ZEN of document review, including concentration. If it were not bad enough to share deep knowledge, instead of fun facts, I even included links to wisdom words with quotes of Zen Masters, old and new. I also mentioned the new trend in corporate America, especially Silicon Valley, of meditation and mindfulness. That was a heavy blog indeed, even the name was way long: Introducing a New Website, a New Legal Service, and a New Way of Life / Work; Plus a Postscript on Software Visualization and Thanks to Kroll Ontrack.

zen_garden_kyotoThe response from most of you, my dear readers, to last month’s blog reminded me of the sound of one hand clapping, or, as I will explain latter, the pauses after Craig Ball’s jokes at his keynote in London last month. Still, last month’s blog did at least provoke an enthusiastic response from all Krollites. I have to concede, however, that this could be a result of my mention and sincere thanks to Kroll Ontrack in the Postscript to Data Visualization at the end of the blog, rather than any great fascination on Kroll’s part with ZEN. Still, I may go with KO next year to teach predictive coding in Tokyo, and even visit Kyoto, so their interest in stages beyond mere information may well be sincere. See: Information → Knowledge → Wisdom: Progression of Society in the Age of Computers.

This month, with my goal to amuse and make even Spock smile, my blog will focus on information, name dropping and insider references. Some knowledge will be thrown in too, of course, because, after all, that is the whole point of information. Information is never an end in itself, or at least should not be. A dash of wisdom may also be thrown in, but, I promise, I will wrap it in humor and sneak it by with vague allusions. No more Zen Master quotes, not even Steve Jobs. Hopefully you will not even notice the wise guy comments, and may even suspect, falsely of course, that you are none the wiser for reading all this bull.

LTN Finalist for Innovative CIO of the Year

ralphlosey_cartoon_smallI will start this newsy blog off on a personal note about my surprise nomination for an honest to God award. No. It has nothing to do with ZEN or document review competitions (ahem – never did get an award for that). It has to do with innovation. Me and new ideas. Imagine that. Unlike former government guru award laden Jason R. Baron, now IG champion of the World after his recent trashing of me in London, I have never won an award (I don’t count my third grade spelling bee) (imagine very small violins playing now). I still have not won an award, mind you, but I have at least now been nominated and qualified as one of three finalists in the Legaltech News Innovation Award 2015. For losers like me just getting a third place mention is a big deal. Sad huh? The award is supposed to recognize “outstanding achievement by legal professionals in their use of technology.”


Thank you dear readers for nominating and voting for me to receive this award. The award category I am in is a bit odd (for me at least), Chief Information Officer, but apparently that is the only one that someone like me could be crammed into. The three finalists in each Innovation category are determined by open voting by LTN magazine subscribers and through LTN’s website. So again, thanks for all of you who voted, especially my family and paid voters in Eastern Europe (they work cheap). The final winner among the finalists in each category are, according to LTN, chosen by “a panel of judges comprised of members of Legaltech News’ editorial staff.” Uh-oh.

Congratulations to all who made it as a finalists and good luck to one and all. There were many vendor categories too, aside from the law firm ones listed in the chart. I list all the vendor categories and finalists below. I have heard of most of them, and know a few very well. But to be honest, I had never heard of many of these vendors, which, no doubt, is what most law firm CIOs are now saying about me. This is an informative list, so I suggest you take time to read it. Again, congrats to all finalists.

Vendor Finalists/Winners

New Product of the Year
Avvo Inc., Advisor
Catalyst Repository Systems, Predict
Diligence Engine
Lex Machina
Best Marketing Services Providers
JD Supra
Best Knowledge Management Software
Motivation Group’s Easy Data Maps
MDLegalApps’ Not Guilty App
ZL Technologies
Best Mobile Device Tool or Service
Abacus Data Systems
Logik Systems’ Logikcull
Best Trial Support Software
Indata Corp.’s TrialDirector
LexisNexis CaseMap
Thomson Reuters’ Case Notebook
Best Case/Matter Management System
Bridgeway Software
Mitratech Holdings’ TeamConnect 4
Best Records Management Software
Hewlett-Packard Co., HP Records Manager (formerly TRIM)
IBM Records Manager
ZL Technologies
Best Risk Management Software
Compliance Science Inc.
IBM OpenPages Operational Risk Management
Best Time and Billing Software
Abacus Data Systems
Tabs3 Software
Tikit North America
Best Collaboration Tool
Accellion kiteworks
Litera IDS
Mitratech’s Lawtrac Self-Service Portal
Opus2 Magnum
Best Document Automation/Management
HotDocs Market
Leaflet Corp.
Best E-Discovery Managed Service Provider
Clutch Group
FTI Consulting, FTI Technology
Iris Data Services’ Arc
Best E-Discovery Processing
Exterro Inc.
iPro Tech
ZL Technologies
Best E-Discovery Review Platform
FTI Technology’s Ringtail software
iConect Developement
kCura Corp.’s Relativity
Recommind Inc.’s Axcelerate 5
Best E-Discovery Legal Hold
Exterro Legal Hold
Legal Hold Pro
Best E-Discovery Hosting Provider
Iris Data Services
Nextpoint Inc.
Best E-Discovery OEM Technology Partner
Content Analyst
Best Research Product
Docket Alarm Inc.
Handshake Corp.
Best Research Platform
Bloomberg Law
LexisNexis’ Lexis Advance
Thomson Reuters’ Westlaw Next
Best Practice Management Software
LexisNexis Legal & Professional’s Firm Manager
Thomson Reuters’ Firm Central

The other two finalists in “my” category, CIO (if hell freezes over and I win, you know I’ll add that title to my card), are Dan Nottke and Harry Shipley. Again, good luck to them and please excuse my pathetic attempts at humor. I Googled them both and will share what I know about them and then make a prediction as to how I will do in this event (hint – it’s not good).

Nottke_danDan Nottke is currently the Chief Information Officer for Kirkland & Ellis LLP, a law firm that always seems to begin descriptions of itself by saying it is 100-years old. (In just a few more years I may be able to say that too.) Most of us know Kirkland, not as old, but as one of the largest, most powerful law firms, having over 1,800 lawyers in key cities around the world. The IT challenges of a firm like that must be huge. Dan is obviously a serious player in the law firm CIO world.

I have never met Dan, but a quick Google shows he is on LTN’s Law Firm Chief Information & Technology Officers Board. With the exception of Monica Bay, who has now left LTN and this CIO Board, I have never met, or even heard of, any of the LTN CIO board members. They all appear to be great people, we just do not travel in the same circles. They are, after all, real life law firm CIOs. They are engineers, not lawyers, but for lawyers. Googling Dan shows that he is usually described with the following defining accomplishment:

Since joining the Firm in 2008, Dan has led the transformation of the Information Technology department from a decentralized team to a fully centralized Information Technology Infrastructure Library (ITIL) based on a high performing organization.

Since his expertise is so different from mine (to be honest I had to look up ITIL on Wikipedia, since I had never heard of it), it is no surprise that we have never met. About the only thing we have in common is high performing law firms, although his firm is more than twice the size of mine. The same goes for the other finalist, Harry Shipley. He has yet another completely different skill set and list of accomplishments.

Harry Shipley is the Assistant Executive Director and CFO of the Iowa State Bar Association. According to his Linked-In profile he is a graduate of Grand View College and the top skill listed is legal research, but otherwise he does not disclose much. Further research shows that he is an expert in document automation, an area he has been working on for over 15 years. I also see that Harry received an award last year from the Iowa State Bar Association in 2014, the Patriot Award, in recognition of his support of Iowa Bar employees serving in the National Guard and Reserve. Seems like a very nice guy, but I could not find out much more about him. Obviously he has many LTN fans or he would not have made the finals.

If I were the LTN Editors and had to pick a winner from these three for the most Innovative CIO of the year, I would pick Dan Nottke (no offense Harry). After all, Dan is the only real life CIO, and taking a “decentralized team to a fully centralized Information Technology Infrastructure Library” seems pretty innovative to me. But, what do I know, I’m just a lawyer, which appears to be the only advantage I have at this point over Dan and Harry. So, congratulations in advance to Mr. Nottke. In the very unlikely event that Harry or I win instead, Dan can at least console himself in knowing that the most Innovative CIO this year did not go to a competitor CIO, it went to a Bar guy or hacker lawyer instead.

LTN-innovation-awardsThe Legaltech News announcement of the award finalists said: “The winners will be recognized at a special event on July 14 at the close of Legaltech West at the City Club of San Francisco.” Well, I never go to Legaltech West, just East. So, even if I did not already have another very important conflict, flying from Orlando to San Francisco is a tad too far to travel for a maybe award dinner. So, Dan, please do not be insulted if I do not show to applaud your acceptance. I admit I did ask Legaltech about any possible advance notice, and they said no way, come to the dinner to find out just like everyone else (I respect that, but had to try). Apparently, this is the first time LTN has ever had an awards dinner for this with all the super-secrecy stuff. (I understand they used to just make an announcement in LTN and mail you something.) But now they have a dinner and are looking for a good turn-out. I don’t blame them for that, having put on a few events myself during my nearly 100 years.

Oscar_AwardAnyway, LTN told me that I had to be there, at the awards dinner, in order to physically receive the award (not sure exactly that means). So, even though I cannot come due to the long distance, and an expected very big and important conflict, namely my playing a new role of Grandfather at about that time, my firm, Jackson Lewis, does have a nice office in San Francisco. So, I am hoping to persuade one or two of my e-discovery attorneys in that office to show up at the dinner for me, to clap a million times where appropriate as awards are doled out, and, in case lightning strikes, to accept the award for me in absentia. In fact, I hope to make them Ralph face-masks to wear, just in case, so, if I win, they can make a convincing showing and quickly grab the hardware before any of the LTN editors figure out that I’m not really there, much less not a real CIO.

e-Discovery in Switzerland

Alice_Down_the_Rabbit_HoleWe used to think of e-discovery as a unique U.S. legal obsession, but that is not true anymore. Our little preoccupation of following evidence down the rabbit hole of technology is now a worldwide phenomena. This was very evident at a couple of events I attended last month in Zurich and London. I’ll start off with Zurich, which has got to be one of the most beautiful cities in the world. The city seemed like a kind of Disney World, super clean, nice and expensive, but without the annoying characters or tourists, and, incredibly quiet. Zurich is all about pristine water, the Swiss alps, and environmentally conscious, healthy, smart people.

I knew all that coming in, but what I did not know until I got there was how sharp and interested the Swiss Bar would be about e-discovery, especially with an advanced topic like predictive coding. I now know why half the world’s money is stashed in Switzerland. They are a very secure bunch, and all carry Swiss army knives and ride around on bikes. Their only vice in Zurich appears to be chocolate, which they eat constantly, and even drink. The only negative thing I can say about Zurich is that it shuts down at 9:00 and it is thereafter impossible to find a good restaurant.

Taylor_HoffmanI was invited to Zurich by Swiss Re e-discovery department to be on a panel that followed the premiere in Switzerland of Joe Looby’s documentary, The Decade of Discovery. Our primary host was Taylor Hoffman, SVP, Head of eDiscovery at Swiss Re. What a dream job Taylor has. He primarily works in New York, but spends a lot of time in Zurich. Jason Baron and his wife, Robin, and I had a seven-course lunch at the private dining facility as Swiss Re’s headquarters overlooking Lake Zurich. We were joined by other members of Swiss Re’s legal department, plus some e-discovery lawyers who came in from Germany and elsewhere to meet and greet. We discussed e-discovery between various wine pairings and ever-changing dishes.

The focus on e-discovery in the EU is all about government investigations, a fact later confirmed by my discussions in London. They also focus on privacy and cross-border issues, and seem to think we are barbarians when it comes to privacy. Since I do not really disagree with them on their privacy criticisms (See: Losey, Are We the Barbarians at the Gate? (e-Discovery Team, Sept. 1, 2008)), a position that seemed to surprise them even more than my being a blogger in a suit, I was able to dodge the daggers very politely thrown at Jason and me.

hide the ballInstead, being the accomplished diplomat that I am (I even have my own email server, rather than blind copying the Chinese on everything), and used to arguing with lawyers everywhere, just as a matter of professional courtesy, it did not take long (one glass) for me to bring up the whole pesky notion of truth and justice. Namely, how can you have justice when both sides in litigation are permitted to hide any documents that they want? They explained to me, an obviously naive and hopelessly idealistic American, that in civilized society, namely Europe, all you are required to disclose are the documents, the ESI, that happen to support your case. In civil litigation you only produce the documents that support your side of the story of what happened.

JusticeThey have virtually no conception of a duty in private litigation to disclose to opposing parties the documents that you have found that show your witnesses are “misremembering” the facts, i.e.- lying. You can imagine how diplomatic I was, and how squirmy and quiet Jason soon became, but it did all end well. We agreed that no one should lie to a judge. Apparently judges everywhere get tired of all of the contradicting allegations and may force both sides to disclose the truth, the whole truth, and nothing but. Apparently, however, that is rare in non-criminal litigation. The primary focus of the kind of disclosure that we know, involving both good and bad documents, is in criminal cases, government investigations, and private, internal investigations.

I asked the non-Swiss Re attorneys attending the lunch how much of their time they spend doing e-discovery work, as opposed to other types of legal services. The answer was it depends, of course, but upon close cross-examination (yeah, I was popular), I learned that the percentage was from 10% to 25%. Remember, these are the outside counsel with special expertise in e-discovery. To me that made it all the more impressive to see how quickly the Zurich attorneys got it who attended the The Decade of Discovery movie. They paid attention, and most importantly, they laughed at all of the right places and seemed to understand. Their questions were good too. They were an unusually astute group, considering that no one outside of Swiss Re and the sponsoring vendor, Consilio, actually do much of this work.

Michael-BeckerConsilio sponsored Joe Looby’s movie showing in Zurich, and then again in London. Consilio’s Managing Directors also presented at both panels following the show, Michael Becker (shown here) in Zurich and Drew Macaulay in London. My thanks for Concilio’s gracious sponsorship and well-run events. Also presenting at these events were Joe Looby, Jason Baron, and Taylor Hoffman.

The main draw was not the panel discussions, as interesting as I think they were, but rather the movie itself, Looby’s Decade of DiscoveryEveryone in Zurich assumed I had seen the documentary many times, but in truth that was only the second time I had seen it. The first was the major showing in DC where everyone who is anyone attended, and most of us wondered how we ended up on the cutting room floor. Still in DC it was a standing ovation, and very emotional, as the star of the movie, Richard Braman, had recently passed away. This movie is a fitting tribute to his work.


Notice how the movie poster says “Justice … is the right to learn the truth from your adversary.” Who knew that is not a popular sentiment in Europe and the UK? We need to learn about privacy from them and they need to learn from us about the importance of full disclosure.

The Decade of Discovery movie prominently features the award-winning Mr. Baron, as the journeyman to Sedona. It makes for a good story, and in the process explains predictive coding pretty well.

I made a movie with Jason myself many years ago, Did You Know: e-Discovery? Apparently our short little slide-show type video is now hard to find, so, even though it is not in the same league as Looby’s real movie, I reproduce it here again so all can easily find it. I can brag that all of our predictions have, so far, come true and the exponential increase in data continues. Feel free to share it by using the share button in the upper left. I would reproduce the Decade of Discovery movie instead, but it is not available online.

Unlike my little slide show video with Jason, Joe Looby’s Decade of Discovery is a real movie. Now that I have seen it twice, I appreciate it much more. I urge you to take time to see it if it ever comes to your town. Check out Joe’s Facebook page for his movie company 10th Mountain Films.

Joe_LoobyOne of the surprise treats from my European trip was to learn what a great guy Joe Looby is. I did not really know Joe. What a pleasure to learn there is no b.s. in Joe, and no big ego either. I did not make any money, nor get any new clients from this trip, but I did make a new friend in Joe Looby. Skeptics may think I’m just kissing up in the hopes of getting a part in an e-discovery sequel, but that’s not true. Joe’s next documentary will concern how emergency decisions are made in the oval office, think Cuban missile crisis. I for one cannot wait to see it. Joe is a true scholar and artist and is evolving beyond his roots in law. Unlike Jason and I, he will surely go on to bigger and better movies. It would not surprise me to see him at the Oscars some day.

e-Disclosure in London

Lord Chief JusticeWe showed the movie in London and had a panel, where, surprisingly the lawyers in attendance did not seem as engaged as the Swiss. We even served popcorn at this event, so go figure. Maybe it was because it was raining (but isn’t it always in London), or maybe it was that whole truth for justice approach that us yanks have. Anyway, Jason, Joe and I had a good time. By the way, they do not call it e-discovery in the UK, they call it e-disclosure. Also, and this amazes me, they do not take depositions over there, or least it is very rare. They just serve prepared statements on each other. That and produce the documents that they want you to see, and hide the rest. The Barristers must be very skilled at cross examination to earn their wigs.

The day after the London movie Jason and I were a keynote at the IQPC event at the Waldorf Astoria in London. We were billed as the great debate on Information Governance. Jason was pro, of course, and I was sort of against, as per my old blog post, Information Governance v Search: The Battle Lines Are Redrawn.  Our keynote was entitled: Let’s Have A Debate About Information Governance — Are We at the End or At the Beginning?

BaronsThe event was the IQPC 10th Anniversary of Information Governance and eDiscovery. Everyone there was either already an IG specialist or hoped to be one. In other words, I was there to argue to the audience that they were all wrong, that IG was dead. Needless to say, my presentation did not go over that well, and Jason soundly won. Even though the deck was stacked against me going in, Jason pulled out all the plugs to make sure he won decisively. I found out why he is banned by his family from playing Monopoly because he is over-competitive, a story he tells whenever he talks about cooperation. His beautiful wife Robin, shown right with Jason in Zurich, confirmed that story for me later. And much more, but I am sworn to secrecy.

british flagSo anyway, just to be sure that he beat me at the great debate, Jason changed the rules at the last moment to have some strange formal debate structure that I’d never heard of involving stop-watch timing, which he controlled. Then, at the closing he surprised me with a carefully scripted speech that he must have stayed up all night writing. He evoked Winston Churchill’s War Room, that was just a few blocks away, and then finished with a rousing quote from the end of Churchill’s most famous speech, We Shall Fight on the Beaches. The only thing missing was a Union Jack draped around his shoulders. The crowd went crazy with patriotic fervor and go-team IG enthusiasm. They will never surrender! It was the only time I saw London lawyers express any emotion. They were real quiet after I followed Churchill, I mean Baron, with my closing statement. Since Baron was Churchill urging all good British citizens to fight on for Information Governance, it was not hard to figure out who they thought I was. I was lucky to be able to goose step out of there alive.

Alison_NorthWell, at least I made some friends by my attack of the London IG establishment, including Alison North, another presenter at the event who is an IG expert herself. She was very nice, protected me from the flying umbrellas that came my way, and politely said she agreed with me. It was more of a whisper really. We sat together for most of the event after that. I was glad to meet such an obviously sophisticated, anti-establishment thinker. We even tried to build a structure out of toothpicks together to hold a marshmallow up in the air as high as possible. That is apparently what lawyers in London do for team building at CLEs. We were at the table with Craig Ball, who was very keen on winning this event. We spent a good fifteen minutes arguing with Craig the ethics of his interpretation of the contest rules. Even though I won that debate, I got called away, so as a team-builder, it was a another loss for me.

Ball_London_15Craig Ball gave the keynote presentation to kick-off the event the first thing in the morning. That is a difficult time slot and I thought he did a good job. As you can see from the photo I took, they had crazy disco type lighting. On stage it was hard for a speaker to see the audience over the bright lights. Craig made many attempts to humor and entertain the London IG professionals. I smiled and laughed a few times, but was alone. Most of Craig’s witty remarks did not even draw a smile, much less a laugh. Only when he made an off-color reference to Fifty Shades of Grey (who better than Craig to do that) did he get a laugh.

I learned a lesson from his start and did not even try for humor. Apparently it does not translate well into whatever language it is they speak over there. In fact the only speaker that was able to get the audience riled up was the Baron of IG himself with his Churchill impression. You know when Craig speaks here again he will surely quote Churchill at length.

Judge_LaporteOther presentations at the event included, U.S. magistrate Judge Elizabeth Laporte (shown right), whom I always enjoy hearing. She did very well with the British Judges on her panel, pointing out that if you are in her court, you have to follow U.S. rules requiring mutual full disclosure, like it or not. The rules of UK and other foreign courts are not what govern. Also presenting and moderating at many of the panels was the reporter, blogger, and retired Solicitor, Chris Dale, whom at the time I thought was a colleague and friend.

Also keynoting at the IQPC were Jeffrey Ritter, Professor of Law, Georgetown University; Jamie Brown, Global eDiscovery Counsel, UBS; Karen Watson, Digital Forensic Investigations, Betfair; Greg O’Connor, Global Head of Corporate, Policy and Regulation, Man Group; Anwar Mirza, Financial Systems Director, TNT Express; and, Jan-Johan Balkema, Global Master Data Manager, Akzo Nobel.

_Balazs_BucsayIn addition to debating Baron on IG, I presented with a reformed black-hat hacker, Balazs Bucsay, who now works for Vodaphone, and Judge Michael Hopmeier, Kingston-on-Thames Crown Court. We had a very short 35 minute panel presentation on cybersecurity. Hacker Bucsay, who is one scary guy, gave a demonstration where a volunteer came on stage and had his password hacked. Impressive. Judge Hopmeier – who was a great guy by the way, tech savvy, frank and outspoken – told everyone how many cybersecurity crimes he sees, and shared a story of a brilliant teenage hacker charged with a serious crimes, even though no money was taken. The kid did it for fun, much like Bucsay used to do. But often it is done by hardened criminals or terrorists. Judge Hopmeier well understands the problem. I hope he is invited to speak in the U.S. soon. We need to hear from him.

Data_Breach_Cost_2014I emphasized Judge Hopmeier’s points on the enormity of the problem, and the Billions of dollars now lost each year by cyber crimes. The average cost of a data breach last year was $3.5 Million. Then I closed with twelve pointers on what a lawyer can do about cyber crime to try to protect their legal practice and their client’s data:

  1. Invest in your company or law firm’s Cybersecurity.
  2. Think like a Hacker and allocate resources accordingly.
  3. Most Law Firms should Outsource primary targets.
  4. Keep Virus Protection in place and updated.
  5. Harden your IT Systems and Websites; $$ and people.
  6. Intrusion Response Focus (Hackers will get in).
  7. Penetration Testing and Vulnerability Scans
  8. Train and Test Employees on Phishing and Social Engineering; Reward/Discipline to prove you are serious.
  9. Be Careful with Cloud Providers and their Agreements.
  10. Buy as much Insurance as possible (insurer guessing game).
  11. Change Laws to make Software Cos Accountable for Errors.
  12. Update Anti-Hacking Laws.

Chinese-cyber-warIt was the only panel on cybersecurity at the IG CLE, which, as far as I am concerned, is a huge mistake. It was late in the day and not well attended. The IG crowd does not seem to grasp the importance of the problem. The Chinese Army applauds their apathy. Let me be very clear using a recent event as an example where they hacked the U.S. government employee database and email. If you are one of the four million past and present federal government employees impacted, the Chinese military not only knows where you live, and has your social security number, user names and passwords, they also know pretty much everything about your personal and professional life. Experts Say China Is Hacking Federal Employees’ Info to Create a Database of Government Workers.

If you are a federal employee who has been a bad boy or girl, say you had an affair, or took a bribe, or maybe you are paying brides to former high school students you molested years ago like Dennis Hastert, they probably know about that too. They read your emails, texts, and Facebook posts. If you have any kind of security clearance, they will have a couple of paid hackers monitoring your every move on the Net. If you were bad, or otherwise have something to hide, they will try will try to extort you. That is what spies do. The FBI is taking this seriously. The four million plus federal employees whose email was hacked should too.

Diner at the Savoy

SavoyI do not usually mention CLE speaker dinners, but the one hosted by Recommind at the IQPC deserves an exception. It was held at a private dining room in the Gordon Ramsay’s Savoy Grill, in The Savoy hotel. I stayed at the Savoy in Zurich and wish I had in London too. But do not waste your time eating at the other famous restaurant at the Savoy, Simpson’s-In-The-Strand. The atmosphere at Simpson’s was good, but not the food. Ramsay’s Savoy Grill, on the other hand, was so good that we went back there the next night. It was by far the best food we had in London, even though some of the waiters spoke with a fake French accent that sounded just like Steve Martin’s Inspector Clouseau. No. Hamburger was not on the menu.

Sherlock Holmes in the Twenty First CenturyWhat made the Recommind dinner special was the group of people they brought together as guests. This was primarily a group of young UK attorneys, the ones who specialize in e-disclosure. Many of them were not able to attend the IQPC event, but they did accept an invite from Recommind for dinner at the Savoy. Aside from the famous Chris Dale, there were only a couple of other speakers there. Most of the dinner guests were true London lawyers, with a couple of Americans lawyers thrown in, those who were lucky enough to be transferred to London. It was a sophisticated group of very smart creatives, all with lovely accents. I felt right at home will all of them and found we had much in common, including my London favorite, Sherlock Holmes.

RALPHCaricatureThis was not my first trip as a speaker to London. Last year I spoke about predictive coding at the famous Lincoln Inn, and also had a dinner with a small group of specialists and judges. That was sponsored by Kroll. I look forward to an opportunity to speak in London again. It is very important to both of our countries that we maintain a close relationship. Next time, however, I just want to speak about predictive coding and cybersecurity. I will leave IG to Jason. You know, old man, it is not really my cup of tea.

My Hack of the NSA and Discovery of a Heretofore Unknown Plan to Use Teams of AI-Enhanced Lawyers and Search Experts to Find Critical Evidence

March 1, 2015

NSA_logoNow that my blog has changed from weekly to monthly I have more time for my hobbies, like trying to hack into NSA computers. I made a breakthrough with that recently, thanks primarily to exuberant disclosures by Snowden after the Oscars. I was able to get into one of the NSA’s top-secret systems. Not only that, my hack led to discovery of a convert operation that will blow your mind. (Hey, if the NSA can brag about their exploits, then so can I.) And if that were not enough, I was able to get away with downloading two documents from their system. I will share what I borrowed with you here (and, of course, on Wikileaks). The documents are:

  • A perviously unknown Plan to use sophisticated e-Discovery Teams with AI-enhancements to find evidence for use in investigations and courtrooms around the world.
  • A slide show in movie and PDF form that tells you how these teams operate.

nsa-spying-logoI can disclose my findings and stolen documents here without fear of becoming Citizen Five because what I found out is so incredible that the NSA will disavow all knowledge. They will be forced to claim that I made up the whole story. Besides, I am not going to explain how I hacked the NSA. Moreover, unlike some weasels, I will never knowingly give aid and comfort to foreign governments. This is something many Hollywood types and script kiddies fail to grasp. All I will say is that I discovered a critical zero-day type error in two lines of code, out of billions, in a software program used by the NSA. In accord with standard white hat protocol, if the NSA admits my story here is true, I will tell them the error. Otherwise, I am keeping this code mistake secret.

Time_SpiralThe hack allowed me to access a Top Secret project coded-named Gibson. It is a Cyberspace Time Machine. This heretofore secret device allows you to travel in time, but, here’s the catch, only on the Internet. Since it is an Internet based device the NSA has to keep it plugged in. That is why I was not faced with the nearly insoluble air gap defense protecting the NSA’s other computer systems.

From what I have been able to figure out, the time travel takes place on a subatomic cyber-level and requires access to the Hadron Collider. The Gibson somehow uses entangled electrons, Higgs bosons, and quantum flux probability. The new technology is based on Hawking’s latest theories, the speed of light, gravity, quantum computers, and, can you believe it, imaginary numbers, you know, the square root of negative numbers. It all seems so obvious after you read the NSA executive summary, that other groups with Hadron Collider access and quantum computers are likely to come up with the same invention soon. But for now the NSA has a huge advantage and head start. Maybe someday they will even share some of that info with POTUS.


The NSA Internet Time Machine allows you to peer into the past content of the Internet, which, I know, is not all that new or exciting. But, here is the really cool part that makes this invention truly disruptive, you can also look into the future. With the Gibson and special web browsers you can travel to and capture future webpages and content that have not been created yet, at least not in our time. You can Goggle the future! Just think of the possibilities. No wonder the NSA never has any funding problems.

Apple_buildingThis kind of breakthrough invention is so huge, and so incredible, that NSA must deny all knowledge. If people discover this is even possible, other groups will race to catch up and build their own Internet Time Machines. That is probably why Apple is hoarding so much cash. Will there be a secret collider built off the books under their new headquarters? It kind of looks like it. Google is probably working on this too. The government cannot risk anyone else knowing about this discovery. That would encourage a dangerous time machine race that would make the nuclear race looks like child’s play. Can you imagine what Iran would do with information from the future? The government simply cannot allow that to happen.

minority-report_Cruse_LoseyFor that reason alone my hack and disclosures are untouchable. The NSA cannot admit this is true, or even might be true. Besides, having seen the future, I already know that I will not be prosecuted for these intrusions. In fact, no one but a few hard-core e-Discovery Team players will even believe this story. I can also share the information I have stolen from the future without fear of CFAA prosecution. Technically speaking my unauthorized access of web pages in the future has not happened yet. Despite my PreCrimelike proposals in PreSuit.com, you cannot (yet) be prosecuted for future crimes. You can probably be fired for what you may do, but that is another story.

nsa_eye_blueStill, the hack itself is not really what is important here, not even the existence of the NSA’s Time Machine, as great as that is. The two documents that I brought back from the future are what really matters. That is the real point of this blog, just in case you were wondering. I have been able to locate and download from the future Internet a detailed outline of a Plan for AI-Enhanced search and review.

The Plan is apparently in common use by future lawyers. I am not sure of the document’s exact date, but it looks like circa 2025. It is obviously from the future, as nobody has any plans like this now. I also found a video and PDF of a PowerPoint of some kind. It shows how lawyers and other investigators in the future use artificial intelligence to enhance all kinds of ESI search projects, including overt litigation and covert investigations. It appears to be a detailed presentation of how to use what is still called Predictive Coding. (Well, at least they do not call it TAR anymore.) Nobody in our time has seen this presentation yet. I am sure of that. You will have the first glimpse now.

The Plan for AI-Enhanced search and review is in the form of a detailed 1,500 word outline. It looks like this Plan is commonly used in the future to obtain client and insurer approval of e-discovery review projects. I think that this review Plan of the future is part of a standardized approval process that is eventually set up for client protection. Obviously we have nothing like that now. The plan might even be shared with opposing counsel and the courts, but I cannot be sure of that. I had to make a quick exit from the NSA system before my intrusion was detected.

I include a full copy of this Plan below, and the PowerPoint slides in video form. See if thee documents are comprehensible to you. If my blog is brought down by denial of service attacks, you can also find it on Wikileaks servers around the world. The Plan can also be found here as a standalone document, and the PDF of the slides can be found here. I hope that this disclosure is not too disruptive to existing time lines, but, from what I have seen of the future of law, temporal paradox be damned, some disruption is needed!

Time_MachineAlthough I had to make a quick exit, I did leave a back door. I can seize root of the NSA Gibson Cyberspace Time Machine anytime I want. I may share more of what I find in upcoming monthly blogs. It is futuristic, but as part of the remaining elite who still follow this blog, I’m sure you will be able to understand. I may even start incorporating this information into my legal practice, consults, and training. You’ll read about it in the future. I know. I’ve been there.

If you have any suggestions on this hacking endeavor, or the below Plan, send me an encrypted email. But please only use this secure email address: HackerLaw@HushMail.com. Otherwise the NSA is likely to read it, and you may not enjoy the same level of journalistic sci-fi protection that I do.


Outline of 12-Step Plan for Predictive Coding Review

1. Basic Numerics of the Project

a. Number and type of documents to be reviewed

b. Time to complete review

c. Software to be used for review

(1) Active Machine Learning features

(A) General description

(B) Document ranking system (ie- Kroll ranks documents by percentage probability, .01% – 99.9%)

(2) Vendor expert assistance to be provided

d. Budget Range (supported by separate document with detailed estimates and projections)

2. Basic Goals of the Project, including analysis of impact of Proportionality Doctrine and Document Ranking. Here are some possible examples:

a. High recall and production of responsive documents within budget proportionality constraints and time limits.

b. Top 25% probable relevant, and all probable (50%+) highly relevant is a metric goal proportional and reasonable in this particular case for this kind of ESI. (Note – these numbers are often used in high-end, large scale projects where there is a premium on quality.)

c. All probable relevant and highly relevant within a specified range or set of ranges.

d. Zero Errors in document review screening for attorney client privileged communications.

e. Evaluation of large production received by client.

f. Time sensitive preparations for specific hearings, mediation, depositions, or 3rd party subpoenas.

g. Private internal corporate investigations as part of quality control, business information, compliance and dispute avoidance..

h. Compliance with government requests for information, state criminal investigations and private civil litigation.

3. General Cooperation Strategy

a. Disclosures planned

(1) Transparent

(2) Translucent

(3) Brick Wall

b. Treatment of Irrelevant Documents

c. Relevancy Discussions

d. Sedona Principle Six

4. Team Members for Project

Penrose_triangle_Expertisea. Predictive Coding Chief. Experienced searcher in charge of the Predictive Coding aspects of the document review

1. Experienced ESI Searcher

2. Same person in charge of non-PC aspects, if not, explain

3. Authority and Responsibilities

4. List qualifications and experience

b. Subject Matter Experts (SME)

(1) Senior SME

A. Final Decision Maker – usually partner in charge of case

B. Determines what is relevant or responsive

(i) Based on experience with the type of case at issue

(ii) Predicts how judge will rule on relevance and production issues

C. Formulates specific rules when faced with particular document types

D. Controls communications with requesting parties senior counsel (usually)

E. List qualifications and experience

(2) Junior SME(s)

A. Lead Document Review expert(s)

B. Usually Sr. Associate working directly with partner in charge

C. Seeks input from final decision maker on grey area documents (Undetermined Category)

D. Responsible for Relevancy Rule articulations and communications

E. List qualifications and experience

(3) Amount of estimated time in budget for the work by Sr and Jr SMEs.

A. Assurances of adequate time commitments, availability

B. Reference time estimates in budget

C. Time should exclude training

(4) Response times guaranties to questions, requests from Predictive Coding Chief

c. Vendor Personnel

(1) Anticipated roles

(2) List qualifications and experience

d. Power Users of particular software and predictive coding features to be used

(1) Law Firm and Vendor

(2) List qualifications and experience

e. Outside Consultants or other experts

(1) Anticipated roles

(2) List qualifications and experience

f. Contract Lawyers

(1) Price list for reviewers and reviewer management

A. $500-$750 per hr is typical (Editors Note: Is this widespread inflation, or new respect?)

B. Competing bids requested? Why or why not.

(2) Conflict check procedures

(3) Licensed attorneys only or paralegals also

(4) Size of team planned

A. Rationale for more than 5 contract reviewers

B. “Less is More” plan

(5) Contract Reviewer Selection criteria

g. Plan to properly train and supervise contract lawyers

5. One or Two-Pass Review

a. Two pass is standard, with first pass selecting relevance and privilege using Predictive Coding, and second pass by reviewers with eyes-on review to confirm relevance prediction and code for confidentiality, and create priv log.

b. If one pass proposed (aka Quick Peek), has client approved risks of inadvertent disclosures after written notice of these risks?

6. Clawback and Confidentiality agreements and orders

a. Rule 502(d) Order

b. Confidentiality Agreement: Confidential, AEO, Redactions

c. Privilege and Logging

(1) Contract lawyers

(2) Automated prep

7. Categories for Review Coding and Training

a. Irrelevant – this should be a training category

b. Relevant – this should be a training category

(1) Relevance Manual for contract lawyers (see form)

(2) Email family relevance rules

A. Parents automatically relevant is child (attachment) relevant

B. Attachments automatically relevant if email is?

C. All attachments automatically relevant if one attachment is?

c. Highly Relevant – this should be a training category

d. Undetermined – temporary until final adjudication

e. No or Very Few Sub-Issues of Relevant, usually just Highly Relevant

f. Privilege – this should be a training category

g. Confidential

(1) AEO

(2) Redaction Required

(3) Redaction Completed

i. Second Pass Completed

8. Search Methods to find documents for training and production

a. ID persons responsible and qualifications

CULLING.2-Filters.3-lakes-ProductionLb. Methods to cull-out documents before Predictive Coding training begins to avoid selection of inappropriate documents for training and to improve efficiency

(1) Eg – any non-text document; overly long documents

(2) Plan to review by alternate methods

(3) ID general methods for this first stage culling; both legal and technical

c. ID general methods for Predictive Coding, ie – Machine selected only, or multimodal

d. Describe machine selection methods.

(1) Random – should be used sparingly, and never as sole method

(2) Uncertainty – documents that machine is currently unsure of ranking, usually in 40%-60% range

(3) High Probability – documents as yet un-coded that machine considers likely relevant

(4) All or some of the above in combination

Multimodal Search Pyramide. Describe other human based multimodal methods

(1) Expert manual

(2) Parametric Boolean Keyword

(3) Similarity and Near Duplication

(4) Concept Search (passive machine learning, such as latent semantic indexing)

(5) Various Ranking methods based on probability strata selected by expert in charge

f. Describe whether a Continuous Active Learning (CAL) process for review will be used, or two-stage process (train, then review), and if later, rationale

9. Describe Quality Control procedures, including, where applicable, any features built into the software, to accomplish following QC goals

quality_trianglea. Three areas of focus to maximize quality of predictive coding

(1) Quality of the AI trainers work to select documents for instruction in the active machine learning process

(2) Quality of the SME work to properly classify documents, especially Highly Relevant and grey area documents, in accord with true probative value and court opinions

(3) Quality of the software algorithms that apply the training input to create a mathematical model that accurately separates the document cloud into probability polar groupings

b. Supervise all reviewers, including contract reviewers who usually do the bulk of the document review work.

(1) ID persons responsible

(2) ID general methods

c. Avoid incorrect conceptions and understanding of relevance and responsiveness, iw – what are you searching for and what will you produce.

(1) Target matches legal obligations

(2) Relevance scope dialogues with requesting party

(3) 26(f) conferences and 16(b) hearings

(4) Motion practice with Court for early resolution of disputes

(5) ID persons responsible

d. Minimize human errors in document coding. Zero Error Numerics.

(1) Mistakes in relevance rule applications to particular documents

(2) Physical mistakes in clicking wrong code buttons

(3) Inconsistencies in coding of same or similar documents

(4) Inconsistencies in coding of same or similar document types

(5) ID persons responsible

e. Facilitate horizontal and vertical communications in team

(1) ID persons responsible

(2) ID general methods

f. Corrections for Concept Drift inherent in any large review project where understanding of relevance changes over time

(1) ID persons responsible

(2) ID general methods

g. Detection of inconsistencies between predictive document ranking and coding

(1) ID persons responsible

(2) ID general methods

h. Avoid incomplete, inadequate selection of documents for training

(1) ID persons responsible

(2) ID general methods

i. Avoid premature termination of training

(1) ID persons responsible

(2) ID general methods

j. Avoid omission of any Highly Relevant documents, or new types of strong relevant documents

(1) ID persons responsible

(2) ID general methods

k. Avoid inadvertent production of privileged documents

(1) List of attorneys names and email domains

(2) Active multimodal search supplement to predictive coding

(3) Dual pass review

(4) ID persons responsible

(5) ID general methods

l. Avoid inadvertent production of confidential documents without proper labeling and redactions

(1) ID persons responsible

(2) ID general methods

m. Avoid incomplete, inaccurate privilege logs

(1) ID persons responsible

(2) ID general methods

n. Avoid errors in final media production to requesting party

(1) ID persons responsible

(2) ID general methods

UpSide_down_champagne_glass10. Decision to Stop Training for Predictive Coding

a. ID persons responsible

b. Criteria to make the decision

(1) Probability distribution

(2) Separation of documents into two poles

(3) Ideal of upside down champagne glass visualization

(4) Few new relevant documents found in last rounds of training

(5) Few new strong relevant types found

(6) No new Highly Relevant documents found

11. Quality Assurance Procedures to Validate Reasonability of Decision to Stop

ei-Recall_smalla. Random Sample Tests to validate the decision

(1) ei-Recall method used, if not, describe

(2) accept on zero error for any Highly Relevant found in elusion test, or new strong relevant type.

(3) Recall and Precision goals

b. Judgmental sampling

12. Procedures to Document the Work Performed and Reasonability of Efforts

a. Clear identification of efforts on the review platform itself with screen shots before project closure

b. Memorandums to file or opposing counsel

(1) Basic metrics for possible disclosure

(2) Detail for internal use only and possible testimony

c. Availability of expert testimony if court challenges arise



What follows is another file I stole from the NSA, a video of PowerPoint slides (no voiceover) for a future presentation called:

Predictive Coding: An Introduction and Real World Example.

The PDF of the slides can be found here.



%d bloggers like this: