Browning Marean: The Life and Death of a Great Lawyer

August 24, 2014

Browning-in-2013Browning Marean passed away on August 22, 2014. His death is a tremendous loss to the e-discovery community. For details on his life, career, and final days of struggle, I suggest you read the blog by his long time close friend, Craig Ball, and also see Browning Marean: A Remembrance by Tom O’Connor. I grieve his passing and feel compelled to share some personal insights, if nothing else to help me to cope with this loss. Browning was always so encouraging and helpful. Such a good friend and colleague. Everyone who knows him understands what I mean. To those who did not have that chance, let me share a few tales of this wise and funny, yet very serious man.

I first met Browning Marean at a Kroll Ontrack sponsored CLE in Atlanta in 2006. Craig Ball and Browning were traveling the country that year spreading the word of electronic discovery at Kroll events. What a powerful and persuasive team they were. I loved Marean’s wit and humor immediately and, like so many others that Browning met, we quickly became friends. I am happy to say that my son, Adam, also had the chance to become Browning’s friend. He was always so encouraging of the next generation, and of all newcomers to e-discovery of any age who were willing to spend the time to seriously study this area of law.

Browning MareanBrowning, by his words, his personal encouragement, and his example, helped inspire me to put aside my litigation practice in 2006 to devote myself full time to e-discovery. I wanted to be like Marean. He was so smart, yet so deft of touch, so full of wit and charm. He was successful, yet unlike others who have enjoyed the pinnacle of the legal profession, he was not full of himself. He was full of fun and life. Above all, Browning loved to laugh. That is how I will remember him. Browning made me, and everyone else laugh too. What a gift he had.

Browning taught me, and many others, so much about so many things. He not only taught me about the finer points of e-discovery, but also how to handle senior status by specializing in e-discovery in a big firm. He showed it could be done, that the firm would benefit immensely, and that you could have a good time closing out your career that way.

Browning MareanBrowning Marian also taught me the ins and outs of what he called the rubber chicken circuit of CLEs. I had a chance to work with him on several occasions. I saw how masterfully he handled every event he chaired, and how he kept everything on time and everyone in their place. He ran a tight ship, which, as an old Navy man, is an expression that Browning would approve of. The man with the funny name kept the ship sailing on time and on course, but he did so with a light touch, and a smile, that I have never seen anyone else equal. Browning Marian was a great role model, with shoes too big for anyone to fill.

Browning was the best ambassador of electronic discovery that I have ever met. His dedication to e-discovery law and teaching is unrivaled by anyone. Browning travelled the globe for over ten years teaching tens of thousands of lawyers, judges, and technologists. He personally touched thousands. Browning Marean was truly one of the great men of the law in the Twenty-First Century.

In closing, I offer a blog I did five years ago featuring Browning and his good friend, Tom O’Connor. Browning was always so encouraging of my blog, and especially liked it when I dared to be controversial, attack the powerful, and still do so with a bit of humor and satire. That is the kind of thing that Browning liked. He was a charming rascal at heart, proponent of the little man, and tireless champion of the cause of justice.


 The New Tonight Show Starring Browning Marean and Tom O’Connor

June 6, 2009

The e-discovery version of the Tonight Show with dueling hosts Browning Marean and Thomas J. O’Connor is the visual theme for my blog this week. Browning, the head of DLA Piper’s e-discovery practice group, plays the role of Jay Leno, and, of course, O’Connor, the Director of the Legal Electronic Document Institute, plays Conan O’Brien. My role is the stammering stand by guest and sometimes also the Office Space employee, Milton Waddams.

The Tonight Show starring Browning and Tom

Yes, this means I have submitted to yet another e-discovery talk show interview. Who knew there were so many? Browning and Tom are well known experts in keeping e-discovery entertaining. Since they have both run out of things to say on their own, they now go around interviewing all of the e-discovery nerds in the known universe. Having by now already talked to all of the really important people, or been turned down (the vast majority), they finally got around to me, something of a low point. But not to worry, next week they have a really good show lined up – an interview with Laura Zubulake’s cat!

They call these audio webcasts the e-Discovery Zone, no doubt because their guests feel like they’ve wandered into the Twilight Zone. This questionable enterprise is sponsored by Techlaw Solutions, although I have no idea why. We had a good time talking, so this interview went on for more than a hour. If you are a real glutton for punishment, go here to listen to the full audio, streamed or downloaded. Alternatively, read on for what I think are the best parts, which, of course, means the drastically shortened and edited parts that make me look good. Also, as I have done before in such interviews, most famously in the brutal Mark Mack interview, I once again share a few of my <Secret Thoughts> to try to make the reading slightly less boring.

The Ratio of People to Cake is Too Big

O’CONNOR: Hello and welcome to the latest edition of the E-Discovery Zone. This is Tom O’Connor along with Browning Marean of DLA Piper and a very special guest today, Ralph Losey of Akerman Senterfitt. Many of you probably read Ralph’s blog or have seen him speaking so we’re very, very pleased to have him today. Welcome, Ralph.

Ralph in Milton in the great movie Office Space who never seems to get his piece of cake. Of course, he who laughs last, laughs best.LOSEY: My pleasure to be here. You two are my favorite guys, so this should be a lot of fun. <Secret Thought: I was promised a piece of cake, and “the last time I didn’t receive a piece. And I was told…”>

O’CONNOR: Great. Well, I’d like to start talking right away about your blog. As we were recounting offline before we started here, I just came out of a hearing that involved someone who didn’t seem to have a great grasp of obligations under the federal Rules, specifically regarding litigation holds in preserving data. And Ralph, you just had a post that you wrote earlier this week specifically about the ethical obligation to know about e-discovery. So I’d really like to have you talk about that a little bit because it’s just so fresh in my mind.

LOSEY: Some people think maybe I’m exaggerating when I say that the problem of competence is reaching a near ethical crisis level. But what you were describing earlier Tom, which we can’t really talk about because it’s a pending matter, just confirms it. Those of us who are in the field dealing with these issues every day know all too well that there’s just a huge lack of information and training by many of the attorneys that are specializing in trial work and dispute resolution. They are still pretending like they’re living in a paper world and they’re not getting any training in law school on this except for a very few schools, maybe 5%. I’m proud to be part of that 5% that is teaching it in law school. <Go Gators!> But in most schools they’re not getting the training. If they’re learning anything about e-discovery, it’s from their own law firms.  Most law firms are not the size of Browning’s and mine and they don’t really have the resources or training to teach it. <Blind leading the blind.> So, it’s a matter of lawyers learning on the job or maybe by catching a one-hour CLE.

The bottom line is, the training is insufficient. For this reason attorneys are not doing a competent job and not fulfilling that very important dictate – an ethical dictate – to perform their job with reasonable competence. They are also failing in their duty to be diligent because they really don’t know what to do to be diligent. I think it’s become, at this point, a serious problem. …

Browning's head on the body of Jay Leno - bad deal for Jay, but at least his chin is smaller.

MAREAN: You know, Ralph, I hearken back to Legal Tech and Judge Facciola’s extraordinary keynote on the third day, and was struck by the passion that he demonstrated in that keynote, again, talking about attorney competence. I was thinking somewhat about, you know, how do we effect change in the legal profession? And sometimes it’s a carrot and sometimes it’s a stick. What I’m wondering is, whether or not malpractice insurers are going to perhaps use a stick of premium and really start to do underwriting due diligence on a law firms’ ability to do electronic discovery. It seems to me that there is going to be the possibility of malpractice suits arising where outside counsel or inside counsel – but again from our standpoint, outside counsel – are held to malpractice cases. I don’t know whether that is something that’s going to change the profession or not. …

No Shortcuts To Competence

O’CONNOR: How do we solve this? I mean, I’m always astonished. It seems like there is just a plethora of webinars and conferences and articles. It astonishes me that people don’t know some of the basics about e-discovery because it seems like there’s educational opportunities everywhere. So how do we solve this problem?

LOSEY: Well I’ve been thinking about this a lot and talking about it. I know I haven’t been talking about it for as long as you two. <Who has? I mean you two guys are really old!> But I’ve been writing about it quite a bit lately and the answer is education, but a different kind of education. We’ve got to do things differently than we’ve been doing it because it’s not working. <The flood of technology and information is moving far faster than the current lame CLEs being offered.> There’s the ever-increasing volume of information and ever-increasing complexity of the systems and information, so that, you know, a year ago we weren’t worried about Twitter – that wasn’t part of the scene. Now it’s taken off. Two years ago you really weren’t worried so much about social networking. Now that’s really exploding, such that every housewife pretty much has it and every employee has it. The systems just keep getting more and more complicated – mergers and acquisitions. Your average company now is just a patchwork of IT systems that are hard for the specialist to understand, much less a general attorney to come in and understand. So, I think we’re losing the battle.

We’ve got to start thinking out of the box <Oh brother, did I really just say that?> and come up with different solutions to what we have been offering so far, the CLE for an hour, or even the day-long CLE. I think Georgetown is an example of taking the lead to go into the one week intensive program, which I had the opportunity to participate in. I think it was just February of this year where it was eight hours a day, every day, and then the fifth day I had the opportunity to be a tester. So I spent all day long – me and Ann Kershaw and Sherry Harris – we divided up into three groups, and we tested these folks to see how much they learned. They did pretty good, really.

Conan Obrien with Tom Oconnor's face

O’CONNOR: Not to, you know, cast any aspersions on what they were doing, but at 50 people at a time, we’re going to need 10 of those a year, right?

LOSEY: Well, yeah. <There goes your invitation to Georgetown.>

O’CONNOR: The law schools seem to be the answer.

LOSEY: It wasn’t cheap and it was limited. It was deliberately capped at I think 40 or 50, as are the classes that Bill Hamilton and I teach at the University of Florida. We capped ‘em at 40 and we turned away students, and it filled up within an hour because the students get it. They see an opportunity here to use their skills, and in today’s marketplace, any edge you can get to help you to get a job, or get ahead if you have a job, is something they’re looking for.

Milton Waddams in his basement office in Office Space: "Mr. Lumbergh told me to talk to payroll and then payroll told me to talk to Mr. Lumbergh and I still haven't received my paycheck and he took my stapler and he never brought it back and then they moved my desk to storage room B and there was garbage on it... "Maybe a fringe benefit of this recession/depression we’re suffering through is that people are now going to be more motivated to take the time to really dig deep and start learning this. Frankly, some students don’t have a job, some attorneys are out of work. <We are all turning into Milton Waddams, the character in the Office Space movie, fearing another downsize move to the office in the basement.> They’re going to have the time to do it, time that they might not have had in a better economy.

Testing Competence

O’CONNOR: Browning, you mentioned Judge Facciola, I know during that – I attended that same presentation, and he gave a not so thinly veiled reference to perhaps we need to have some sort of testing requirement. He seemed to, as I recall, say that he didn’t feel the law schools were really picking up the slack the way they should.

MAREAN: Well, you know, I wonder – Ralph and I had the privilege of attending the Second International Litigation Support Conference, I think, in Washington a week or so ago, and one of the things that struck me there is that law firms, to the extent that they have litigation support groups within the law firm, that in fact that is the source of the most practical knowledge for dealing with e-discovery issues. They, in fact, get it and can be of tremendous help in guiding the attorneys, if the attorneys will turn to them in a timely fashion, to deal with such things that lawyers aren’t very well equipped to deal with – such as form or forms of production. …

O’CONNOR: And so given all that – and I guess this comes back to the point I raised earlier – we seem to have a number of resources out there. Why do we still have, as Ralph said, this critical mass of folks who are ignorant? And as I recall Judge Facciola saying, it is not because they’re not intelligent, it’s because they’re – I believe the word he used was – obstinate. They’re simply not availing themselves of these resources.

LOSEY: A lot of it has to do with who does the law attract, what kind of person is screened in the LSATs, the admissions. We’re not attracting people that are oriented to computers. Math and science majors typically don’t go to law school. They go to med school or they go to engineering school. That’s part of it. Law schools need to change their admission and, number one, they need to start teaching it. I mean, University of Florida, Georgetown – these are rare exceptions. Even Georgetown only teaches it once a year. University of Florida, at least, we’re teaching it every semester now.

MAREAN: Ralph, tell us a little bit – I know you and Bill Hamilton are involved down in Florida – tell us a little bit about that curriculum, what kind of a curriculum have you put together and the like.

tests and examsLOSEY: … There is competency testing in law school. That’s the beauty of it. The final exam I gave them was pretty darn hard. As a matter of fact, it was only slightly simpler than the exam I dreamed up for the Georgetown experts who were, you know, some of them 10, 20, 30 year lawyers. It was basically the same test, a little less complicated, and they had just three hours to write the answer out. We tested the full EDRM model, one through nine. They actually started on two, identification, preservation, collection – those first three, and then our last question was on what we’ve learned from this fact scenario. How would you recommend that the IT and Information Management Systems be redesigned? These were challenging questions, that I am sure 95% of the litigation attorneys in America wouldn’t know how to answer correctly. I can tell you that all of the student answers were good. Some of the answers were fantastic! <The Book Award for best student this semester was awarded to two students: Jason Pill and Johann Van Lierop. Congratulations!> …

These are all smart people. They respond to training. But this is intensive training – I estimate it would take 200-250 hours over our 4½ month semester of study and work to get to this point. Two hundred and fifty hours, okay, in a four month period. This is not happening in the CLE programs. We’re not getting that kind of commitment and intensity, and so we’re getting superficial learning. And to be honest – because, you know, I’m not connected with any vendor so I can be a little controversial – most of the CLEs I see that are vendor-sponsored, they’re “scare you into hiring us” type CLEs. <The “pay to play” type CLEs are even worse. No bona fide subject matter expert ever pays to teach. The ones you see at these events are mostly just salesman trying to hustle in-house counsel. They know enough to be dangerous and make a boring speech.>  Lawyers are getting sick of that. Lawyers tell me, “I’m tired of these e-discovery CLEs. I don’t learn anything practical. I just learn that I don’t know what the hell I’m doing and I should be scared.” Of course, what they would like is a magic pill to easily learn all of the practical stuff. That’s the first problem. There is no shortcut. It takes time and effort and practice and more practice.

MAREAN: Well your comment reminds me of Malcolm Gladwell’s Outliers book which I’ve picked up and I’ve only read some reviews of, but where he talks about what does it take for some people to be successful. <He’s talking about my article.> I think he was using Joe Flom at the Skadden Arps firm that, you know, how many hours does it take to become an expert? And I do agree, I think that we absolutely need to be spending a lot more time. But Ralph, to your earlier point, I think this is a wonderful opportunity for somebody coming into a firm to really spend the time and become the go-to person in this area. Talk about making yourself valuable to the firm even at a young age, to me it’s one of the most obvious routes open today.

LOSEY: It really is. It’s a great opportunity. I’m finding that the young people get it much quicker. They already know all the basics that you and I know because we’ve been doing it for years, but that a lot of  people our age don’t. So, it’s a quicker learn for them. It still takes 200-250 hours to get the basics. Malcolm Gladwell cited the scientific studies that weren’t about getting the basics, they were about attaining a level of mastery where you really could teach this stuff. They found it takes 10,000 hours. That’s five years, maybe 10 years, depending on how much time you devote to it. So it takes a lot of time. How many masters of discovery are there really that can teach this stuff? And so, that’s the problem. We’ve got to – everybody’s got to raise their game up. Those of us that know something need to be sure that we’re doing legitimate education and we’re really helping the rest of The Bar, our brothers and sisters that are struggling with this, to really understand it. That’s the solution. It’s not, well, you need to understand enough to know you’ll never learn how to do this crap so you better hire us, which unfortunately is a lot of what goes on. We don’t do that, but we’ve all seen it done. …

Judges Are Smart

O’CONNOR: That does make an interesting point, thought, Browning which is, if we don’t think the attorneys are being educated, are the judges being educated?

MAREAN: Well I pick up on Ralph’s point and Ralph, I think that Bray & Gillespie case was out of the federal Court in Orlando, and I was struck by the thoughtfulness of the magistrate judge in that opinion which, by the way, really does loop back to issues of competence and sort of getting with the program of discovery, but I found her opinion – and I was really not familiar with that magistrate judge before, but I thought she put out a very thoughtful opinion.

LOSEY: Well yes, I know Judge Karla Spaulding pretty well. I have been practicing here my whole career and Judge Spaulding has been here a long time. She is not a computer hobbyist like me, she’s not a techie, but she’s a smart person – all the federal judges are smart. And she is very diligent. She just dug in there and worked very hard, had two evidentiary hearings to get to the bottom of things when she saw the smokescreens and the lawyers saying different things. She really worked hard. And it shows that if you’re diligent and put in an enormous – I have no idea how many hours that she and her team of law clerks put into it, but I’m sure it was very substantial. Not many judges will take the time to do that.

We can’t expect to find hero judges like that willing to do it all the time. But it does show that people of above average intelligence, which all of our federal judges are like that, can sort through it and figure it out. They can hear expert testimony on both sides and figure it out. But the truth is, most judges don’t have the time necessary to dig into it like that, or maybe they just don’t have the inclination to do it, in which case I think the solution is a special master. I really think that’s part of the answer, if the parties are in a difficult situation and the magistrate may not be willing or able to take the time to do that, or it may take them a year to do that, then the parties ought to consider agreeing upon a special master that has particular training and expertise in the area of e-discovery and come up with a quicker, possibly more just ruling for them. …

Crystal Ball Gazing Five Years Into The Future

ESCHER famous etching of a man gazing into a crystal ball ruined by putting Losey's face into itMAREAN: Well Ralph, let’s assume that we’re now five years hence, it’s May 21st, 2014, what do you see will have changed in the next five years?

LOSEY: I think I’ve lost some weight and am in better condition.

MAREAN: A consummation devoutly to be wished.

LOSEY: Yeah but I’m an optimist. No, I think what’s going to happen is, we’re going to see some big players come in. Somebody’s going to step up to the plate and we’re going to get some real intensive training. We’re going to get competency certification, and I doubt it will be the local Bars because they just take too long. Most state Bars do have certification programs in different areas, but it’s going to take, I think, longer than five years. … I don’t think we’ll be there yet with certification from the state Bars. What do you think Marean?

MAREAN: I think they’re going to view it as too narrow and not really pick up on Judge Scheindlin’s comment that it’s not just  “e-discovery,” it’s now just “discovery.”

LOSEY: I think Judge Scheindlin’s remark has been fairly criticized. I don’t know, maybe it was you Tom who pointed out that it is not “discovery” because they’re still depositions, there’s interrogatories, you know, there’s some stuff. But it is, I think, document discovery or better said discovery of writings, which has always been critical to the outcome of most civil cases. What were the parties writing? E-discovery is really discovery of writings because there’s so few paper writings – original paper writings – nowadays that you might as well discount them as of marginal importance. The discovery of writings today is what e-discovery is all about. Any lawyer who has a case where what the people wrote is important needs to know e-discovery.

They need to know on a couple of levels: there’s a base competency level, that they need to learn to handle the small cases; and, then there’s more advanced for the bigger and more complicated cases. I think we’re going to have to see training, better training and certification on these two levels. I don’t think it will come from the state Bars, and I also don’t think the law schools will move that quick, although we’re going to  see some leaders and schools like Georgetown that are figuring out that this is very good. We’re going to get more schools, but will Harvard be offering e-discovery in five years? Maybe that will be the time they first offer it, I guess, Harvard and Yale, and the other top 25. It will probably take them that long before they hold their noses and deal with something so practical and narrow, but that’s the attitude you’re getting from academia. So the solution is not from the Bar, it’s not from academia, I don’t think, with all due respect to Kroll, that their two-day certification program is really what we’re talking about here either, so I’m not sure a vendor is going to do it, but I think somebody needs to. …

So that’s the training part of it. The other thing we’re going to see in five years, though, to change the subject a little bit, is we’re going to see improved software. I don’t think we’ll have the magic button yet, but I’ve been talking to vendors a lot, I know you guys talk to vendors a lot too, and I said, you know, I want to see the random button on your software henceforth. It’s not there yet. But in five years every program is going to have a random button where you do random sampling. Random sampling is going to be commonplace instead of “Wow, that’s such an exotic idea,” which, believe it or not, is the reaction today of most vendors. In five years it’s going to be common. Also, in five years it’s going to be the exception, rather than the rule, to negotiate key words in the blind and then run them. I think in five years we can hope the Bar will get away from that and we will have testing and sampling as part of everybody’s normal way of search. …

O’CONNOR: Right. Well we’re starting to run out of time here. Any last thoughts or topics that either of you would like to cover?

LOSEY: There’s one thing we haven’t mentioned and it goes to Browning’s excellent five-year question, and that is, I’m hoping that in five years the Sedona Cooperation Proclamation is going to be not only widely known, but followed. Again, it’s going to come out of money concerns, cost driven; the clients are going to stop paying lawyers just to jerk around with hide the ball. They’re only going to pay them to argue over the merits of the case and argue over the legal implication of the facts, and not argue over whether they should get the facts or not. I think we’re going to see, out of necessity, lawyers being more cooperative in the area of discovery,including e-discovery. So this may be a hope more than a prediction but, you know, I’ll go out on a limb. It’s a prediction. It’s not just a hope because we cannot afford business as usual in discovery.

MAREAN: Well absolutely. I’m in to that and I think getting attorneys to read 26(G) and most states courts or most states have similar type rules about what it means when we sign our name to a pleading and what that certification carries with it. I think putting teeth into 26(G) – the tool is there. The interesting thing will be whether or not the judiciary decides to give it the emphasis that I think it needs.

LOSEY: You’re right. We’re full circle back to the ethics, which is so important. It’s competence and also cooperation – just following the rules. Federal Procedure Rule number one: quick, speedy, inexpensive, just adjudication. We’ve got to get back to that, otherwise we’re going to have people fleeing the public justice system into the private system of arbitration. Judge Facciola is very concerned about that. I am too. I like public adjudication. I would rather not see everything go into private arbitration, although there’s a place for that, and that means we’ve got to have discovery of all kinds be affordable.

O’CONNOR: Right. Well as always, Ralph, it’s great speaking with you and hearing your thoughts. Once again, I want to recommend to everybody that they take a look at Ralph’s blog, the e-Discovery Team …

Bates stampLOSEY: I hate to interrupt somebody endorsing me <Boy am I stupid or what?>, but while you’re at it, check out the HASH article because I know that’s something Tom and Browning – the three of us are really big advocates of, using the HASH algorithm instead of Bates numbers. We can’t go through an interview together, guys, without at least saying the HASH word once.

O’CONNOR: Stop using your Bates stamper. Browning, do you still have the old Bates stamp machine in your desk?

MAREAN: I do indeed, and people will come in and say, “What’s that?”

Are You The Lit Support Tech?

August 16, 2014

Ralph_2013_beard_frownThe video below shows what it is like on a Friday afternoon in the Litigation Support Departments of most law firms in the U.S. and U.K.. That’s because most lawyers are still clueless about technology in general, much less the intricacies of electronic discovery work and ESI processing. There are some exceptions of course. A few firms have it all together and everyone gets it. They are called sole proprietorships. Then there are also some large law firms like mine where this does not occur. Why? What’s our secret you may wonder? We outsourced our whole litigation support to a top vendor. I’m sure you know who they are. Now as the partner in charge of our litigation support department, my Friday afternoons are relatively stress free. I just approve KO doing it, and they do it well. It is their problem, and their core competence.

I no longer have to supervise a litigation support department. Instead I manage a relationship with a vendor. It is much more pleasant, believe me. When not working on projects and serving clients, I focus my internal e-discovery firm management time on the training and education of my firm’s lawyers and paralegals. IMO this is the way it should be. Law firms should stick to their core competency, practicing law and teaching law, and should not try to run little vendor corps in their midst.

With a smoothly running outsourced deal my Friday afternoons are much nicer. I can now look at this video and laugh. My cringing at the same time has substantially diminished. The anguish and bitter ironies are almost all gone. I know that it is getting better every day, that real learning is happening, and, for me, these scenes have disappeared, well, at least they’ve moved to a better venue. To those of you still living the old dream, you have my sympathies and condolences. Know that there is a way out.


Jones Day Attorney Misconduct Shows Rotten State of Obstructionist Discovery in America

August 10, 2014

U.S. District Court Judge Mark Bennett

You have probably heard about the sanctions order against a Jones Day attorney entered by U.S. District Court Judge Mark W. Bennett. But have you read the order? Now you can. It is reproduced here in full. I have not changed a thing, except to embolden the text that I think is of special interest to e-discovery practitioners. The misconduct described in here pertains to deposition defense, but the general comments apply to all types of discovery, including electronic.


THE SECURITY NATIONAL BANK OF SIOUX CITY, IOWA, as Conservator for J.M.K., a Minor, Plaintiff,






I. PROCEDURAL HISTORY ………………………………………………………. 5

II. ANALYSIS ……………………………………………………………………………. 7

A. Standards for Deposition Sanctions ……………………………………… 7

B. Deposition Conduct ……………………………………………………………. 10

1. “Form” Objections ………………………………………………………………. 10

2. Witness Coaching ………………………………………………………………. 18

3. Excessive Interruptions …………………………………………………….. 30

C. Appropriate Sanction ………………………………………………………… 31

III. CONCLUSION …………………………………………………………………. 34

Something is rotten, but contrary to Marcellus’s suggestion to Horatio, it’s not in Denmark.1 Rather, it’s in discovery in modern federal civil litigation right here in the United States. Over two decades ago, Griffin Bell—a former United States Attorney General, United States appeals court judge, and private practitioner—observed: “The criticism of the civil justice system has reached a crescendo in recent years. Because



Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 1 of 34


much of the cost of litigation is incurred in discovery, the discovery process has been the focal point of considerable criticism.”2 How little things have changed.

Discovery—a process intended to facilitate the free flow of information between parties—is now too often mired in obstructionism. Today’s “litigators” are quick to dispute discovery requests, slow to produce information, and all-too-eager to object at every stage of the process. They often object using boilerplate language containing every objection imaginable, despite the fact that courts have resoundingly disapproved of such boilerplate objections.3 Some litigators do this to grandstand for their client, to intentionally obstruct the flow of clearly discoverable information, to try and win a war of attrition, or to intimidate and harass the opposing party. Others do it simply because it’s how they were taught. As my distinguished colleague and renowned expert on civil procedure Judge Paul Grimm of the District of Maryland has written: “It would appear that there is something in the DNA of the American civil justice system that resists cooperation during discovery.”4 Whatever the reason, obstructionist discovery conduct is born of a warped view of zealous advocacy, often formed by insecurities and fear of the truth. This conduct fuels the astronomically costly litigation industry at the expense of “the just, speedy, and inexpensive determination of every action and proceeding.”


2 Griffin B. Bell et al., Automatic Disclosure in Discovery—The Rush to Reform, 27 GA. L. REV. 1, 1 (1992).

3 See Matthew L. Jarvey, Note, Boilerplate Discovery Objections: How They Are Used, Why They Are Wrong, and What We Can Do About Them, 61 DRAKE L. REV. 913, 917 n.20 (2013) (collecting cases disapproving of boilerplate objections); St. Paul Reinsurance Co., Ltd. v. Commercial Fin. Corp., 198 F.R.D. 508, 513 (N.D. Iowa 2000) (same).

4 Hon. Paul W. Grimm & David S. Yellin, A Pragmatic Approach to Discovery Reform:How Small Changes Can Make a Big Difference in Civil Discovery, 64 S.C. L. REV. 495,530 (2013).

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 2 of 34


Fed. R. Civ. P. 1. It persists because most litigators and a few real trial lawyers even very good ones, like the lawyers in this case—have come to accept it as part of the routine chicanery of federal discovery practice.But the litigators and trial lawyers do not deserve all the blame for obstructionist discovery conduct because judges so often ignore this conduct,6 and by doing so we 


5 Judge Grimm and David Yellin aptly describe some of the misplaced motivations behind obstructionist tactics:

The truth is that lawyers and clients avoid cooperating with their adversary during discovery—despite the fact that it is in their clear interest to do so—for a variety of inadequate and unconvincing reasons. They do not cooperate because they want to make the discovery process as expensive and punitive as possible for their adversary, in order to force a settlement to end the costs rather than having the case decided on the merits. They do not cooperate because they wrongly assume that cooperation requires them to compromise the legitimate legal positions that they have a good faith basis to hold.

Lawyers do not cooperate because they have a misguided sense that they have an ethical duty to be oppositional during the discovery process—to “protect” their client’s interests—often even at the substantial economic expense of the client.

Clients do not cooperate during discovery because they want to retaliate against their adversary, or “get back” at them for the events that led to the litigation. But the least persuasive of the reasons for not cooperating during the discovery process is the entirely misplaced notion that the “adversary system” somehow prohibits it.

Id. at 525-26 (footnotes omitted). Amen Brother Grimm and Mr. Yellin for being so insightful and refreshingly candid.

6 Cf. Daniel C. Girard & Todd I. Espinosa, Limiting Evasive Discovery: A Proposal for Three Cost-Saving Amendments to the Federal Rules, 87 DENV. U. L. REV. 473, 475 (2010) (“The Federal Rules prohibit evasive responses . . . . In practice, however, these rules are not enforced. Service of evasive discovery responses has become a routine—and rewarding—litigation tactic.”)

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 3 of 34


reinforce—even incentivize—obstructionist tactics.7 Most litigators, while often inept in jury trials (only because they so seldom experience them), are both smart and savvy and will continue to do what has worked for them in the past. Obstructionist litigators, like Ivan Pavlov’s dogs, salivate when they see discovery requests and are conditioned to unleash their treasure chest of obstructive weaponry. Unlike Pavlov’s dogs, their rewards are not food but successfully blocking or impeding the flow of discoverable information.

Unless judges impose serious adverse consequences, like court-imposed sanctions, litigators’ conditional reflexes will persist. The point of court-imposed sanctions is to stop reinforcing winning through obstruction.

While obstructionist tactics pervade all aspects of pretrial discovery, this case involves discovery abuse perpetrated during depositions. Earlier this year, in preparation for a hard-fought product liability jury trial, I was called upon by the parties to rule on numerous objections to deposition transcripts that the parties intended to use at trial. I noticed that the deposition transcripts were littered with what I perceived to be meritless objections made by one of the defendant’s lawyers, whom I refer to here as “Counsel.” I was shocked by what I read. Thus, for the reasons discussed below, I find that Counsel’s deposition conduct warrants sanctions.

I do not come to this decision lightly. Counsel’s partner, who advocated for Counsel during the sanctions hearing related to this case (and who is one of the best trial lawyers I have ever encountered), urged that sanctions by a federal judge, especially on


7 Cf. id. at 483 (“The reluctance of courts to impose sanctions under Rule 37 has encouraged the use of evasive and dilatory behavior in response to discovery requests. Such behavior serves no purpose other than to increase the cost and delays of litigation.”).

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 4 of 34


a lawyer with an outstanding career, like Counsel, should be imposed, if at all, with great hesitation and a full appreciation for how a serious sanction could affect that lawyer’s career. I wholeheartedly agree. I am still able to count each of the sanctions I have imposed on lawyers in my twenty years as a district court judge on less than all the fingers of one hand. Virtually all of those sanctions have been imposed on (or threatened to be imposed on) lawyers from out-of-state law firms.8


This matter arises out of a product liability case tried to a jury in January of 2014. Plaintiff Security National Bank (SNB), acting as conservator for a minor child, J.M.K., sued Defendant Abbott Laboratories (Abbott), claiming that J.M.K. suffered permanent brain damage after consuming baby formula, produced by Abbott, that allegedly contained a dangerous bacteria called enterobacter sakazakii. SNB went to trial against Abbott on design defect, manufacturing defect, and warning defect claims. On January 17, 2014, a jury found in favor of Abbott on SNB’s product liability claims. The Clerk entered judgment in favor of Abbott on January 21, 2014.

During the trial, I addressed Counsel’s conduct in defending depositions related to this case. Specifically, I filed a sua sponte order to show cause as to why I should not


8 Iowa trial lawyers have a long and storied tradition and culture of civility that is first taught at the state’s two law schools, the University of Iowa College of Law and the Drake University Law School. I know this because I have taught and lectured at both of these outstanding law schools that produce the bulk of Iowa lawyers. Civility is then taken very seriously, nourished and lead by the Iowa Supreme Court, and continually reinforced by the Iowa State Bar Association, the Iowa Academy of Trial Lawyers, and all of the other legal organizations in the state, as well as senior members of the bar, law firm partners from large to small firms, and solo practitioners across the state. There is great pride in being an Iowa lawyer, and describing someone as an Iowa lawyer almost always connotes that lawyer’s high commitment to civility and professionalism. Of course, there are stinkers in the Iowa bar, but they are few and far between.

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 5 of 34


sanction Counsel for the “serious pattern of obstructive conduct” that Counsel exhibited during depositions by making hundreds of “form” objections that ostensibly lacked a valid basis. Because I did not want to burden Counsel with the distraction of a sanctions hearing during trial, I suggested we table any discussion of sanctions until after the trial was over. Thus, the same day the judgment was filed, I entered a supplemental order to show cause, ordering Counsel to address three issues that potentially warrant sanctions: (1) Counsel’s excessive use of “form” objections; (2) Counsel’s numerous attempts to coach witnesses; and (3) Counsel’s ubiquitous interruptions and attempts to clarify questions posed by opposing counsel. My supplemental order focused on Counsel’s conduct in defending two particular depositions—those of Bridget Barrett-Reis and Sharon Bottock—but I noted that I would consider any relevant depositions in deciding whether to impose sanctions. On January 24, 2014, Counsel requested a substantial extension of time to respond to my supplemental order, which I granted. On April 21, 2014, Counsel responded to my supplemental order to show cause. My chambers later contacted Counsel to set this matter for telephonic hearing. Counsel requested another one-month delay, which I granted. Counsel filed an additional brief on July 9, 2014, and the hearing was finally held on July 17, 2014. During the hearing, I requested that Counsel follow up with an e-mail suggesting an appropriate sanction, should I decide to impose one. On July 21, 2014, Counsel’s partner sent an e-mail to me declining to suggest a sanction, and urging me not to impose sanctions.

After reviewing Counsel’s submissions, I find that Counsel’s conduct during depositions warrants sanctions. I discuss below the basis for imposing sanctions and the particular sanction that I deem appropriate in this case.

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 6 of 34



A. Standards for Deposition Sanctions

“It is well established that a federal court may consider collateral issues [like sanctions] after an action is no longer pending.” Cooter & Gell v. Hartmarx Corp., 496 U.S. 384, 395 (1990). Because Counsel’s deposition conduct is at issue here, Federal Rule of Civil Procedure 30 applies. Rule 30(d)(2) provides: “The court may impose an appropriate sanction—including the reasonable expenses and attorney’s fees incurred by any party—on a person who impedes, delays, or frustrates the fair examination of the deponent.” Rule 30(d)(2) does not limit the types of sanctions available; it only requires that the sanctions be “appropriate.” See Francisco v. Verizon S., Inc., 756 F. Supp. 2d 705, 712 (E.D. Va. 2010), aff’d, 442 F. App’x 752 (4th Cir. 2011) (“Although Rule 30(d)(2) does not define the phrase ‘appropriate sanction,’ the imposition of discovery sanctions is generally within the sound discretion of the trial court.” (citations omitted)).

District courts also have a “‘well-acknowledged’ inherent power . . . to levy sanctions in response to abusive litigation practices.” Roadway Exp., Inc. v. Piper, 447 U.S. 752, 765 (1980). “A primary aspect of that [power] is the ability to fashion an appropriate sanction for conduct which abuses the judicial process.” Chambers v. NASCO, Inc., 501 U.S. 32, 44-45 (1991). “[T]he inherent power of a court can be invoked even if procedural rules exist which sanction the same conduct.” Id. at 49.

Counsel incorrectly argues—without citing to any dispositive authority—that I may not impose sanctions sua sponte under Rule 30(d)(2). Because SNB’s lawyers did not file a motion for sanctions, Counsel argues that I am without power to impose them under the Federal Rules.9 Rule 30(d)(2)’s text, however, imposes no such limitation on a


9 The fact that SNB’s lawyers did not move for sanctions further suggests that lawyers have simply become numb to obstructionist discovery tactics, either because they are used to them, they choose to take the high ground, or perhaps because they use such tactics themselves. (After observing SNB’s lead lawyer at trial, I seriously doubt the latter.) Based on my 39 years as a member of the federal bar, I surmise that SNB’s lawyer did not move for sanctions because he has other enterobacter sakazakii cases against Counsel and did not want to undermine his ongoing relationship with Counsel by seeking sanctions. This rationale makes particular sense in a case like this where all of the information SNB’s lawyer needed to prove SNB’s manufacturing and product defect claim resided with Abbott and Counsel, and where there was no other avenue to obtaining case-critical information.

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 7 of 34


court’s authority to sanction deposition conduct. The rule contains no motion-related preconditions whatsoever; it simply provides that “[t]he court may impose an appropriate sanction” on a person who obstructs a deposition. The advisory committee notes further suggest that courts may issue Rule 30(d)(2) sanctions without a motion from a party. The notes provide that sanctions under Rule 30(d) are congruent to those under Rule 26(g):

The rule also explicitly authorizes the court to impose the cost resulting from obstructive tactics that unreasonably prolong a deposition on the person engaged in such obstruction. This sanction may be imposed on a non-party witness as well as a party or attorney, but is otherwise congruent with Rule 26(g).

Fed. R. Civ. P. 30, advisory committee notes (1993 amendments). Under Rule 26(g), courts may issue sanctions sua sponte: “If a certification violates this rule without substantial justification, the court, on motion or on its own, must impose an appropriate sanction on the signer, the party on whose behalf the signer was acting, or both.” Fed. R. Civ. P. 26(g)(3) (emphasis added). In addition to Rule 30(d)’s text and the advisory committee notes, the United States Supreme Court has noted that “court[s] generally may act sua sponte in imposing sanctions under the Rules.” Chambers, 501 U.S. at 43 n.8; see also Jurczenko v. Fast Prop. Solutions, Inc., No. 1:09 CV 1127, 2010 WL 2891584, at *2-4 (N.D. Ohio July 20, 2010) (imposing sanctions under Rule 30(d)(2) where party moved for sanctions only under Rule 37(d)). And even if I lacked the power to issue

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 8 of 34


sanctions under Rule 30(d), I would retain the authority to sanction Counsel under my inherent power. See In re Itel Sec. Litig., 791 F.2d 672, 675 (9th Cir. 1986) (“Sanctions may also be awarded sua sponte under the court’s inherent power.” (citing Roadway Exp., 447 U.S. at 765)).

Counsel also claims to have acted in good faith during the depositions related to this case. Even if that is true, it is inapposite. In imposing sanctions under either Rule 30(d)(2) or my inherent power, I need not find that Counsel acted in bad faith. “[T]he imposition of sanctions under Federal Rule[] of Civil Procedure 30(d)(2) . . . does not require a finding of bad faith.” GMAC Bank v. HTFC Corp., 248 F.R.D. 182, 196 (E.D. Pa. 2008). Rather, the person sanctioned need only have “impede[d], delay[ed], or frustrate[d] the fair examination of the deponent.” Fed. R. Civ. P. 30(d)(2). And only the most extreme sanctions under a court’s inherent power—like assessing attorney’s fees or dismissing with prejudice—require a bad-faith finding. See Chambers, 501 U.S. at 45-46 (noting that “a court may assess attorney’s fees when a party has acted in bad faith, vexatiously, wantonly, or for oppressive reasons” (citations and internal quotation marks omitted)); Stevenson v. Union Pac. R. Co., 354 F.3d 739, 751 (8th Cir. 2004) (“A bad faith finding is specifically required in order to assess attorneys’ fees.” (citations omitted)); Lorin Corp. v. Goto & Co., Ltd., 700 F.2d 1202, 1207 (8th Cir. 1983) (“Dismissal with prejudice is an extreme sanction and should not be imposed unless the default was wilful or in bad faith.”). For less extreme sanctions, like those at issue here, “a finding of bad faith is not always necessary to the court’s exercise of its inherent power to impose sanctions.” Stevenson, 354 F.3d at 745 (citations omitted); see also Harlan v. Lewis, 982 F.2d 1255, 1260 (8th Cir. 1993) (“We do not believe Roadway extends the ‘bad faith’ requirement to every possible disciplinary exercise of the court’s inherent power, especially because such an extension would apply the requirement to even the most routine exercises of the inherent power. We find no statement in Roadway,

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 9 of 34


Chambers, or any other decision cited by the parties, that the Supreme Court intended this ‘bad faith’ requirement to limit the application of monetary sanctions under the inherent power.” (internal citations and footnote omitted)). Still, while I need not find bad faith before imposing sanctions, I find it difficult to believe that Counsel could, in good faith, engage in the conduct outlined in this opinion.

The Eighth Circuit Court of Appeals “review[s] the imposition of discovery sanctions for an abuse of discretion.” Craig v. St. Anthony’s Med. Ctr., 384 F. App’x 531, 532 (8th Cir. 2010).

B. Deposition Conduct

In defending depositions related to this case, Counsel proliferated hundreds of unnecessary objections and interruptions during the examiner’s questioning. Most of these objections completely lacked merit and often ended up influencing how the witnesses responded to questions. In particular, Counsel engaged in three broad categories of improper conduct. First, Counsel interposed an astounding number of “form” objections, many of which stated no recognized basis for objection. Second, Counsel repeatedly objected and interjected in ways that coached the witness to give a particular answer or to unnecessarily quibble with the examiner. Finally, Counsel excessively interrupted the depositions that Counsel defended, frustrating and delaying the fair examination of witnesses. I will address each category of conduct in turn.

1. “Form” Objections

In the two depositions I asked Counsel to review in my order to show cause, Counsel objected to the “form” of the examiner’s question at least 115 times. That means that Counsel’s “form” objections can be found on roughly 50% of the pages10 of both the Barrett-Reis and Bottock depositions. Counsel made “form” objections with similar


10 I calculated this number based on the number of deposition pages that actually contained testimony, excluding pages like the title page, etc.

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 10 of 34


frequency while defending other depositions, too. Sometimes Counsel followed these “form” objections with a particular basis for objection, like “speculation” or “narrative.” Other times, Counsel simply objected to “form,” requiring the reader (and, presumably, the examiner) to guess as to the objection’s basis.

In addition to the sheer number of “form” objections Counsel interposed, Counsel also demonstrated the “form” objection’s considerable range, using it for a number of purposes. For example, Counsel used “form” objections to quibble with the questioner’s word choice (for no apparent reason, other than, perhaps, to coach the witness to give a desired answer):

Q. Would it be fair to say that in your career, work with human milk fortifier has been a significant part of your job?

COUNSEL: Object to the form of the question. “Significant,” it’s vague and ambiguous. You can answer it.

A. Yeah, I can’t really say it’s been a significant part. It’s been a part of my job, but “significant” is rather difficult because I have a wide range of things that I do there.

(Barrett-Reis Depo. 56:19 to 57:4).11 Counsel used “form” objections to voice absurdly hyper-technical truths:

Q. Are there certain levels that one can get, that have catwalks or some similar apparatus so I can get to the dryer?

A. The dryer is totally enclosed. You cannot get into the dryer from any of the levels.

Q. Can I get on the outside of the dryer?


11 In reproducing portions of the deposition transcripts for this opinion, I occasionally change the notation identifying the speaker for reasons of anonymity, consistency, and ease of reading. For example, I do not use Counsel’s name, which appears in the transcripts. I also use “A.” to indicate a witness’s answer, whereas some of the transcripts use the phrase “the witness.” The words used by the speakers, however, remain unaltered.

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 11 of 34


COUNSEL: Object to the form of the question; outside of the dryer? Everything is—I mean, outside of the dryer is a huge expanse of space; anything that’s not inside the dryer is outside the dryer, so I object to it as vague and ambiguous. Object to the form of the question.

A. Rephrase the question.

(Bottock Depo. 130:3-15). Counsel also used “form” objections to break new ground, inventing novel objections not grounded in the rules of evidence or common law:

Q. Are you familiar with the term “immunocompromised”?

A. Yes.

Q. And that would include premature babies?

COUNSEL: Object to the form of the question, “that would include premature babies?” It’s a non sequitur.12

(Barrett-Reis Depo. 54:15-21). (In case there is any doubt, non sequitur is not a proper objection.) But, whatever their purpose, Counsel’s “form” objections rarely, if ever, followed a truly objectionable question.

In my view, objecting to “form” is like objecting to “improper”—it does no more than vaguely suggest that the objector takes issue with the question. It is not itself a


12 In response to my order to show cause, Counsel claims that “the question was misleading, confusing, vague and ambiguous[,]” and that it “call[ed] for a medical opinion or conclusion” (docket no. 193, at 13). None of these reasons relate to Counsel’s original claim that the question was a non sequitur. But, in any event, there is absolutely nothing confusing about the question, nor does it call for a medical conclusion (the witness held a PhD in nutritional science, though). This litany of adjectives— “misleading, confusing, vague and ambiguous”—are all too common in federal depositions and roll too easily and too frequently off the lips of lawyers who engage in repeated obstructionist conduct. Multiple objections like this are often a harbinger of obstructionist lawyers. That Counsel would cite those objections in “defense” of Counsel’s conduct suggests very strongly that Counsel just doesn’t get it, and further undermines Counsel’s claim of good faith. That these objections are part of an oft-used litigation strategy does not suggest that Counsel made them in good faith.

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 12 of 34


ground for objection, nor does it preserve any objection. Instead, “form” objections refer to a category of objections, which includes objections to “leading questions, lack of foundation, assuming facts not in evidence, mischaracterization or misleading question, non-responsive answer, lack of personal knowledge, testimony by counsel, speculation, asked and answered, argumentative question, and witness’ answers that were beyond the scope of the question.” NGM Ins. Co. v. Walker Const. & Dev., LLC, No. 1:11-CV-146, 2012 WL 6553272, at *2 (E.D. Tenn. Dec. 13, 2012). At trial, when I asked Counsel to define what “form” objections entail, Counsel gave an even broader definition. Counsel first stated simply, “I know it when I hear it.” Counsel then settled on the barely narrower definition that “form” objections include “anything that can be remedied at the time of the deposition so that you do not waive the objection if the deposition is used at a hearing or trial.” Given that “form” may refer to any number of objections, saying “form” to challenge a leading question is as useful as saying “exception” to admit an excited utterance.

Yet, many lawyers—and courts for that matter—assume that uttering the word “form” is sufficient to state a valid objection. This assumption presumably comes from the terminology used in the Federal Rules. Rule 30(c)(2) governs deposition objections and provides in part:

An objection at the time of the examination—whether to evidence, to a party’s conduct, to the officer’s qualifications, to the manner of taking the deposition, or to any other aspect of the deposition—must be noted on the record, but the examination still proceeds; the testimony is taken subject to any objection. An objection must be stated concisely in a nonargumentative and nonsuggestive manner.

The advisory committee notes clarify the types of objections that must be noted on a deposition record:

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 13 of 34


While objections may, under the revised rule, be made during a deposition, they ordinarily should be limited to those that under Rule 32(d)(3) might be waived if not made at that time, i.e., objections on grounds that might be immediately obviated, removed, or cured, such as to the form of a question or the responsiveness of an answer.

Fed. R. Civ. P. 30, advisory committee notes (1993 amendments) (emphasis added).

These notes refer to Rule 32(d)(3), which provides that certain objections are waived if not made during a deposition:

An objection to an error or irregularity at an oral examination is waived if:

(i) it relates to the manner of taking the deposition, the form of a question or answer, the oath or affirmation, a party’s conduct, or other matters that might have been corrected at that time; and

(ii) it is not timely made during the deposition.

Fed. R. Civ. P. 32(d)(3)(B) (emphasis added). Together, these rules provide that any objection to the form of a question must be made on the record at a deposition, or that objection is waived.

But these rules do not endorse the notion that “form” is a freestanding objection. They simply describe categories of objections—like those to the form of a question—that must be noted during a deposition. Nothing about the text of Rules 30 or 32 suggests that a lawyer preserves the universe of “form” objections simply by objecting to “form.”

I agree with my colleague, Magistrate Judge Scoles, in his analysis of this issue:

[Some] contend that the objection should be limited to the words “I object to the form of the question.” The Rule, however, is not so restrictive. Rather, it simply provides that the objection must be “stated concisely in a nonargumentative and nonsuggestive manner.” . . . [T]he general practice in Iowa permits an objector to state in a few words the manner in which the question is defective as to form (e.g., compound,

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 14 of 34


vague as to time, misstates the record, etc.). This process alerts the questioner to the alleged defect, and affords an opportunity to cure the objection.

Rakes v. Life Investors Ins. Co. of Am., No. C06-0099, 2008 WL 429060, at *5 (N.D. Iowa Feb. 14, 2008); see also Cincinnati Ins. Co. v. Serrano, No. 11-2075-JAR, 2012 WL 28071, at *5 (D. Kan. Jan. 5, 2012) (“Although the [rules] talk about objections based on the ‘form’ of the question (or responsiveness of the answer), this does not mean that an objection may not briefly specify the nature of the form objection (e.g. ‘compound,’ ‘leading,’ ‘assumes facts not in evidence’).”). I would go further, however, and note that lawyers are required, not just permitted, to state the basis for their objections.

Moreover, “form” objections are inefficient and frustrate the goals underlying the Federal Rules. The Rules contemplate that objections should be concise and afford the examiner the opportunity to cure the objection. See Fed. R. Civ. P. 30(c)(2) (noting that “objection[s] must be stated concisely”); id., advisory committee notes (1993 amendments) (noting that “[d]epositions frequently have been unduly prolonged . . . by lengthy objections and colloquy” and that objections “ordinarily should be limited to those . . . grounds that might be immediately obviated, removed, or cured, such as to the form of a question”). While unspecified “form” objections are certainly concise, they do nothing to alert the examiner to a question’s alleged defect. Because they lack specificity, “form” objections do not allow the examiner to immediately cure the objection. Instead, the examiner must ask the objector to clarify, which takes more time and increases the amount of objection banter between the lawyers. Briefly stating the particular ground for the objection, on the other hand, is no less concise and allows the examiner to ask a remedial question without further clarification.

Additionally, it is difficult, if not impossible, for courts to judge the validity of unspecified “form” objections:

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 15 of 34


[U]nless an objector states with some specificity the nature of his objection, rather than mimicking the general language of the rule, i.e., “objection to the form of the question,” it is impossible to determine, based upon the transcript of the deposition itself, whether the objection was proper when made or merely frivolous.

Mayor & City Council of Baltimore v. Theiss, 729 A.2d 965, 976 (Md. 1999). When called upon to rule on an unspecified “form” objection, a judge either must be clairvoyant or must guess as to the objection’s basis. Neither option is particularly realistic or satisfying. This is reason enough to require a specific objection.

Requiring lawyers to state the basis for their objections is not the same thing as requiring “speaking objections” in which lawyers amplify or argue the basis for their objections. For example, “Objection, hearsay” is a proper objection. By contrast, “Objection, the last assertion by Mr. Jones was an out-of-court statement by Ms. Day, said in the hotel room, that Mr. Jones allegedly heard, that he never testified to in a deposition, and that is now being offered for the truth of Ms. Day’s statement” is an improper speaking objection. I have always required the former and barred the latter.

I recognize, however, that not all courts share my views regarding “form” objections. In fact, some courts explicitly require lawyers to state nothing more than unspecified “form” objections during depositions. See Offshore Marine Contractors, Inc. v. Palm Energy Offshore, L.L.C., No. CIV.A. 10-4151, 2013 WL 1412197, at *4 (E.D. La. Apr. 8, 2013) (“The Court finds that the behavior of counsel for OMC does not warrant sanctions here. Indeed, most of the objections by OMC’s counsel are simple form objections with no unwarranted, lengthy speaking objections.”); Serrano, 2012 WL 28071, at *5 (“But such an objection [to a vague question] to avoid a suggestive speaking objection should be limited to an objection ‘to form,’ unless opposing counsel requests further clarification of the objection.”); Druck Corp. v. Macro Fund (U.S.) Ltd., No. 02 CIV.6164(RO)(DFE), 2005 WL 1949519, at *4 (S.D.N.Y. Aug. 12, 2005) (“Any

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 16 of 34


‘objection as to form’ must say only those four words, unless the questioner asks the objector to state a reason.”); Turner v. Glock, Inc., No. CIV.A. 1:02CV825, 2004 WL 5511620, at *1 (E.D. Tex. Mar. 29, 2004) (“All other objections to questions during an oral deposition must be limited to ‘Objection, leading’ and ‘Objection, form.’ These particular objections are waived if not stated as phrased above during the oral deposition.”); Auscape Int’l v. Nat’l Geographic Soc’y, No. 02 CIV. 6441(LAK), 2002 WL 31014829, at *1 (S.D.N.Y. Sept. 6, 2002) (“Once counsel representing any party states, ‘Objection’ following a question, then all parties have preserved all possible objections to the form of the question unless the objector states a particular ground or grounds of objection, in which case that ground or those grounds alone are preserved.”); In re St. Jude Med., Inc., No. 1396, 2002 WL 1050311, at *5 (D. Minn. May 24, 2002) (“Objecting counsel shall say simply the word ‘objection’, and no more, to preserve all objections as to form.”).13 For the reasons discussed above, I think this approach makes little legal or practical sense.

But, because there is authority validating “form” objections, I do not impose sanctions based on the fact that Counsel used these objections while defending depositions. Counsel’s “form” objections, however, amplified two other issues: witness coaching and excessive interruptions. As I discuss below, those aspects of Counsel’s deposition conduct warrant sanctions. Thus, I impose sanctions related to Counsel’s “form” objections only to the extent that those objections facilitated the coaching and interruptions. Although I do not impose sanctions based on Counsel’s “form” objections


13 The record contains no indication that Counsel knew of, or relied on, these, or similar cases when Counsel made “form” objections during depositions. Counsel did not claim to know of these cases, or similar lines of authority, at the time Counsel made the “form” objections, in Counsel’s response to either of my show-cause orders, or at the sanctions hearing.

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 17 of 34


in this case, lawyers should consider themselves warned: Unspecified “form” objections are improper and will invite sanctions if lawyers choose to use them in the future.

2. Witness Coaching

While there appears to be disagreement about the validity of “form” objections, the law clearly prohibits a lawyer from coaching a witness during a deposition. Under Rule 30(c)(2), deposition “objection[s] must be stated concisely in a nonargumentative and nonsuggestive manner.” See also Fed. R. Civ. P. 30, advisory committee notes (1993 amendments) (“Depositions frequently have been . . . unfairly frustrated, by lengthy objections and colloquy, often suggesting how the deponent should respond.”). This clause mandates what should already be obvious—lawyers may not comment on questions in any way that might affect the witness’s answer:

The Federal Rules of Evidence contain no provision allowing lawyers to interrupt the trial testimony of a witness to make a statement. Such behavior should likewise be prohibited at depositions, since it tends to obstruct the taking of the witness’s testimony. It should go without saying that lawyers are strictly prohibited from making any comments, either on or off the record, which might suggest or limit a witness’s answer to an unobjectionable question.

Hall v. Clifton Precision, 150 F.R.D. 525, 530-31 (E.D. Pa. 1993); see also Specht v. Google, Inc., 268 F.R.D. 596, 598 (N.D. Ill. 2010) (“Objections that are argumentative or that suggest an answer to a witness are called ‘speaking objections’ and are improper under Rule 30(c)(2).”).

Despite the Federal Rules’ prohibition on witness coaching, Counsel’s repeated interjections frequently prompted witnesses to give particular, desired answers to the examiner’s questions. This happened in a number of ways. To start, Counsel often made “clarification-inducing” objections—objections that prompted witnesses to request that the examiner clarify otherwise cogent questions. For example, Counsel regularly

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 18 of 34


objected that questions were “vague,” called for “speculation,” were “ambiguous,” or were “hypothetical.” These objections usually followed completely reasonable questions.

But, after hearing these objections, the witness would usually ask for clarification, or even refuse to answer the question:

Q. Is there—do you believe that there’s—if there’s any kind of a correlation that could be drawn from OAL environmental samples to the quality of the finished product?

COUNSEL: Objection; vague and ambiguous.

A. That would be speculation.

Q. Well, if there were high numbers of OAL, Eb samples in the factory, wouldn’t that be a cause for concern about the microbiological quality of the finished product?

COUNSEL: Object to the form of the question. It’s a hypothetical; lacks facts.

A. Yeah, those are hypotheticals.

. . .

Q. Would that be a concern of yours?

COUNSEL: Same objection.

A. Not going to answer.

Q. You’re not going to answer?

A. Yeah, I mean, it’s speculation. It would be guessing.

COUNSEL: You don’t have to guess.

(Bottock Depo. 106:24 to 108:2). While it is impossible to know for certain what a witness would have said absent Counsel’s objections, I find it inconceivable that the witnesses deposed in this case would so regularly request clarification were they not tipped-off by Counsel’s objections. See McDonough v. Keniston, 188 F.R.D. 22, 24 (D.N.H. 1998) (“The effectiveness of [witness] coaching is clearly demonstrated when

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 19 of 34


the [witness] subsequently adopts his lawyer’s coaching and complains of the broadness of the question . . . .”); Cordova v. United States, No. CIV.05 563 JB/LFG, 2006 WL 4109659, at *3 (D.N.M. July 30, 2006) (awarding sanctions based on a lawyer’s deposition coaching because “it became impossible to know if [a witness’s] answers emanated from her own line of reasoning or whether she adopted [the] lawyer’s reasoning from listening to his objections”).

These same objections spilled over into the trial. The following colloquy occurred during the plaintiff’s cross-examination of Counsel’s expert:

Q. . . . Isn’t [J.M.K.’s mother] saying that every time she used a bottle she boiled it first?

COUNSEL: Your Honor, I would just object that in the—it’s not clear from the context of this one page or several pages what it is they’re talking about in terms of which feedings, if he can point it out to him.

THE COURT: And so what is the nature of that objection? I haven’t ever heard that one before.

COUNSEL: It’s confusing.

THE COURT: Well, it may be confusing to you, but he didn’t ask the question to you. He asked it of the witness.

COUNSEL: Okay. Might be confusing to the witness.

THE COURT: Yeah, that’s suggesting an answer which is exactly the problem I had with your depositions.

COUNSEL: I would just object to the form of the question then, Your Honor.

THE COURT: That’s not a proper objection, so it’s overruled.

A. As I read this, I can’t be certain as to what exactly she’s referring to at what point here.

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 20 of 34


Once again, after Counsel’s objection suggested that the question “might” confuse the witness, the witness replied that he “[couldn’t] be certain” as to what was being asked.

But perhaps the most egregious examples of clarification-inducing objections arose when Counsel defended the deposition of Sharon Bottock. During that deposition, Counsel lodged no fewer than 65 “form” objections, many of which did not specify any particular basis. Immediately after most of these “form” objections, the witness gave the seemingly Pavlovian response, “Rephrase.” At times, the transcript feels like a tag-team match, with Counsel and witness delivering the one-two punch of “objection”– “rephrase”:

Q. . . . I’m wondering if you could perhaps in a . . . little bit less technical language explain to me what they’re talking about in that portion of the exhibit.

COUNSEL: Object to the form of the question.

A. So rephrase.

Q. Could you tell me what they’re saying here?

COUNSEL: Same objection.

A. Rephrase it again.

. . .

Q. So it—that’s what they’re talking about, the two types, the finished product and the overs? Does it separate those two things?

A. Yes.

Q. What’s an “over”?

COUNSEL: Object to the form. He doesn’t want you to characterize it. He wants to know what’s it made out of, I think.14


14 Here, Counsel reinterprets the question for the witness—an issue that I address below.

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 21 of 34


Q. I mean, is it too big?

COUNSEL: Object to the form of the question.

A. Rephrase.

(Bottock Depo. 58:20 to 59:25). Note the witness’s first answer in this colloquy: So rephrase. The witness’s language makes clear that she is requesting—actually, commanding—the examiner to rephrase based on Counsel’s objection.

These clarification-inducing objections are improper. Unless a question is truly so vague or ambiguous that the defending lawyer cannot possibly discern its subject matter, the defending lawyer may not suggest to the witness that the lawyer deems the question to be unclear. Lawyers may not object simply because they find a question to be vague, nor may they assume that the witness will not understand the question. The witness—not the lawyer—gets to decide whether he or she understands a particular question:

Only the witness knows whether she understands a question, and the witness has a duty to request clarification if needed. This duty is traditionally explained to the witness by the questioner before the deposition. If defending counsel feels that an answer evidences a failure to understand a question, this may be remedied on cross-examination.

Serrano, 2012 WL 28071, at *5; see also Hall, 150 F.R.D. at 528-29 (“If the witness does not understand the question, or needs some language further defined or some documents further explained, the witness can ask the deposing lawyer to clarify or further explain the question. After all, the lawyer who asked the question is in a better position to explain the question than is the witness’s own lawyer.” (footnote omitted)); Peter M. Panken & Mirande Valbrune, Enforcing the Prohibitions Against Coaching Deposition Witnesses, Prac. Litig., Sept. 2006, at 15, 16 (“It is improper for an attorney to interpret that the witness does not understand a question because the lawyer doesn’t understand a

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 22 of 34


question. And the lawyer certainly shouldn’t suggest a response. If the witness needs clarification, the witness may ask the deposing lawyer for clarification. A lawyer’s purported lack of understanding is not a proper reason to interrupt a deposition.”).

Counsel’s clarification-inducing objections are reminiscent of the improper objections at issue in Phillips v. Manufacturers Hanover Trust Co., No. 92 CIV. 8527 (KTD), 1994 WL 116078 (S.D.N.Y. Mar. 29, 1994). In Phillips, a lawyer

objected or otherwise interjected during [the examiner’s] questioning of the deponent at least 49 times though the deposition lasted only an hour and a half. Indeed, approximately 60 percent of the pages of the transcript contain such interruptions. Many of these were objections as to form, which are waived if not made at the deposition, Fed. R. Civ. P. 32(d)(3)(B), but on numerous occasions [the lawyer’s] objections appeared to have no basis. . . . Moreover, after 21 of [the lawyer’s] objections as to form, the deponent asked for clarification or claimed he did not understand the question. . . . [The lawyer] objected as to form, and the deponent then stated he did not understand the question, subsequently asking that it be narrowed.

Id. at *3. In considering whether to impose sanctions, the court described the lawyer’s conduct as “inappropriate” and “obnoxious.” Id. The court also noted that the lawyer’s conduct frustrated the deposition:

Such interplay clearly did hamper the free flow of the deposition. Rather than answer [the examiner’s] questions to the best of his ability, the deponent hesitated, asking for clarification of apparently unambiguous questions. . . . In addition, the deponent asked for such clarifications almost exclusively after [the lawyer] objected or interrupted in some fashion.

Id. Finally, the court recognized that the lawyer’s conduct violated Rule 30, but chose not to impose sanctions because, at the time, Rule 30 was newly amended and because

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 23 of 34


the examiner was able to finish the deposition. Id. at *4. The court warned, however, that “a repeat performance [would] result in sanctions.” Id.

Like the lawyer in Phillips, Counsel’s endless “vague” and “form” objections (and their variants described above) frustrated the free flow of the depositions Counsel defended. They frequently induced witnesses to request clarification to otherwise unambiguous questions. Counsel’s “form” objections also emboldened witnesses to quibble about the legal basis for certain questions—e.g., “That would be speculation”— and to stonewall the examiner—e.g., “Not going to answer.” In short, these objections were suggestive and amounted to witness coaching, thereby violating Rule 30.

But Counsel’s clarification-inducing objections are only part of the problem. In a related tactic, Counsel frequently concluded objections by telling the witness, “You can answer if you know” or something similar. Predictably, after receiving this instruction, witnesses would often claim to be unable to answer the question:

Q. Are these the ingredients that are added after preparation or after pasteurization?

COUNSEL: If you know. Don’t guess.

A. If you could rephrase the question. There’s no ingredients on 28.

COUNSEL: So you can’t answer the question.

(Bottock Depo. 47:12-18).

Q. If it’s high enough to kill bacteria, why does Abbott prior to that go through a process of pasteurization?

COUNSEL: If you know, and you’re not a production person so don’t feel like you have to guess.

A. I don’t know.

(Bottock Depo. 48:12-17).

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 24 of 34


Q. Does it describe the heat treatment that you referred to a few moments ago, the heat treatment that occurs in the dryer phase?

. . .

COUNSEL: Okay. Do you know his question? He’s asking you if this is what you’re describing.

A. Yeah, I don’t know.

(Bottock Depo. 57:8-21).

Q. . . . Is there any particular reason that that language is stated with respect to powdered infant formula?

COUNSEL: If you know. Don’t—if you know.

A. No, I—no, not to my knowledge.

COUNSEL: If you know. I mean, do you know or not know?

A. I don’t know.

(Barrett-Reis Depo. 49:10-18). These responses are unsurprising. When a lawyer tells a witness to answer “if you know,” it not-so-subtly suggests that the witness may not know the answer, inviting the witness to dodge or qualify an otherwise clear question. For this reason, “[i]nstructions to a witness that they may answer a question ‘if they know’ or ‘if they understand the question’ are raw, unmitigated coaching, and are never appropriate.” Serrano, 2012 WL 28071, at *5; see also Specht, 268 F.R.D. at 599 (“Mr. Fleming egregiously violated Rule 30(c)(2) by instructing Mr. Murphy not to answer a question because his answer would be a ‘guess.’”); Oleson v. Kmart Corp., 175 F.R.D. 560, 567 (D. Kan. 1997) (noting that an attorney violated Rule 30 when he “interrupted [a] deposition in mid-question, objected to the assumption of facts by the witness, and advised the witness that he was not obligated to assume facts”).

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 25 of 34


Lastly, Counsel often directly coached the witness to give a particular, substantive answer. This happened in a few ways. Sometimes Counsel reinterpreted or rephrased the examiner’s questions:

Q. To what extent do you have knowledge of the testing procedures that Abbott employs in raw materials or the environment, the plant environment or final product?

A. Very limited knowledge, again, because that would be product development.

COUNSEL: He’s just asking you what do you have. Do you have any? If it’s no, then just say “no.”

A. Okay.

(Barrett-Reis Depo. 20:16 to 21:2).

Q. . . . Do you know when that occurs or does it occur on a regular basis?

COUNSEL: Object to the form, regular basis. It says, “Once a year.” He means the same time once a year presumably but—

A. On an annual basis, the time may vary when we close the facility to fumigate.

(Bottock Depo. 34:5-11).

Q. At any rate, you’ll see that on both the first page of Exhibit 22 and the first page of Exhibit 23, there’s a picture of the product, and both of them have the word “NeoSure” on the product. Would you be able to tell me what the difference between those two products is?

. . .

COUNSEL: Well, he said difference between the products. It lacks foundation that there’s a difference between the products.

Q. There may not be. I don’t know. Can you tell me?

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 26 of 34


COUNSEL: Well, the question is—I object to the form of the question. He’s not asking you just about the label. He’s asking you is there a difference in the product. So can you answer that?

(Barrett-Reis Depo. 29:2-20). Sometimes Counsel gave the witness additional information to consider in answering a question:

Q. For that particular infant who is not premature, like in this case was a twin, do you believe that NeoSure is an appropriate version of powdered infant formula?

COUNSEL: Object to the form. Lack of foundation in terms of what this baby—whether this baby was preterm or not. It’s not in evidence in this deposition nor in the record anyplace. And I object to the form of the question as calling for speculation.

Q. Go ahead.

COUNSEL: You can answer.

A. I can’t answer it without more information.

(Barrett-Reis Depo. 99:7-19). Sometimes Counsel answered the examiner’s question first, followed by the witness:

Q. . . . Is that accurate or is there something that they, you know, just chose not to put—

COUNSEL: If you know. She didn’t write this.

A. Yes, I didn’t write this.

(Bottock Depo. 27:20-25)

Q. Okay. The part that counsel just read, is that basically an accurate summary of the process?

COUNSEL: In general.

A. In general.

(Bottock Depo. 28:21-24).

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 27 of 34


Q. . . . And then under “Follow-Up Test” for Eb it’s essentially the same thing as E. sak negative; right?

COUNSEL: It says zero.

A. It says zero.

Q. But which would—that would be the same type of finding if it said E. sak negative; right?

COUNSEL: In other words, there’s no Eb. There’s no Eb; there’s no—

A. It’s zero. There’s no Eb.

(Bottock Depo. 114:14-24). Counsel even audibly disagreed with a witness’s answer, prompting the witness to change her response to a question:

Q. My question is, was that a test—do you know if that test was performed in Casa Grande or Columbus?

A. I don’t.

COUNSEL: Yes, you do. Read it.

A. Yes, the micro—the batch records show finished micro testing were acceptable for the batch in question.

(Bottock Depo. 86:9-15).

All of the objections described in this section violate Rule 30 by suggesting, in one way or another, how the witness should answer a question. More troublingly, these objections allowed Counsel to commandeer the depositions, influencing the testimony in ways not contemplated by the Federal Rules. Instead of allowing for a question-and-answer session between examiner and witness, Counsel acted as an intermediary, which frustrated the purpose of the deposition:

The underlying purpose of a deposition is to find out what a witness saw, heard, or did—what the witness thinks. A deposition is meant to be a question-and-answer conversation between the deposing lawyer and the witness. There is no

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 28 of 34


proper need for the witness’s own lawyer to act as an intermediary, interpreting questions, deciding which questions the witness should answer, and helping the witness to formulate answers. The witness comes to the deposition to testify, not to indulge in a parody of Charlie McCarthy, with lawyers coaching or bending the witness’s words to mold a legally convenient record. It is the witness—not the lawyer—who is the witness.

Hall, 150 F.R.D. at 528 (footnote omitted); see also Alexander v. F.B.I., 186 F.R.D. 21, 52-53 (D.D.C. 1998) (noting that “[i]t is highly inappropriate for counsel for the witness to provide the witness with responses to deposition questions by means of an objection” or to “rephrase or alter the question” asked of the witness); Panken & Valbrune, supra, at 16 (“[C]ounsel is not permitted to state on the record an interpretation of questions, because those interpretations are irrelevant and are often suggestive of a particularly desired answer.”).

In response to my order to show cause, Counsel explains what motivated many ofthe objections that I perceive to be coaching:

In many places during the depositions of Abbott witnesses . . . where it was clear that the plaintiff’s counsel was on the wrong track factually . . . defense counsel attempted to steer him to the correct ground. When things got bogged down, hours in, defense counsel also attempted to speed up the process by helping to clarify or facilitate things, for which the plaintiff’s counsel seemed appreciative.

(Docket no. 193, at 4-5) (footnote omitted). It is not for the defending lawyer to decide whether the examiner is on the “wrong track,” nor is it the defending lawyer’s prerogative to “steer [the examiner] to the correct ground.” While lawyers are encouraged to be collegial and helpful to one another during depositions, Counsel’s conduct, on balance, was neither. It defies common sense to suggest that Counsel’s omnipresent commentary sped up the depositions in this case. Moreover, most of

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 29 of 34


Counsel’s commentary during depositions were objections, not benign attempts to clarify. Because this commentary coached witnesses to give particular answers, I find that sanctions are appropriate.

3. Excessive Interruptions

Beyond the “form” objections and witness coaching, Counsel’s interruptions while defending depositions were grossly excessive. Counsel’s name appears at least 92 times in the transcript of the Barrett-Reis deposition (about once per page), and 381 times in the transcript of the Bottock deposition (approaching three times per page). Counsel’s name appears with similar frequency in the other depositions that Counsel defended. And, as I noted earlier, nearly all of Counsel’s objections and interruptions are unnecessary and unwarranted.

These excessive and unnecessary interruptions are an independent reason to impose sanctions. The notes accompanying Rule 30 provide that sanctions may be appropriate “when a deposition is unreasonably prolonged” and that “[t]he making of an excessive number of unnecessary objections may itself constitute sanctionable conduct . . . .” Fed. R. Civ. P. 30, advisory committee notes (1993 amendments); see also Craig, 384 F. App’x at 533 (“The notes also explain that an excessive number of unnecessary objections may constitute actionable conduct, though the objections be not argumentative or suggestive.”). At least two courts in this circuit have imposed sanctions based, in part, on a lawyer’s excessive and unnecessary objections during depositions. See id. (affirming a monetary sanction against a lawyer who made “a substantial number of argumentative objections together with suggestive objections” that “impeded, delayed, or frustrated [a] deposition”); Van Pilsum v. Iowa State Univ. of Sci. & Tech., 152 F.R.D. 179, 181 (S.D. Iowa 1993) (sanctioning a lawyer who had “no justification for . . . monopoliz[ing] 20% of his client’s deposition” and whose objections “were for the most part groundless, and were only disputatious grandstanding”).

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 30 of 34


By interposing many unnecessary comments, clarifications, and objections, Counsel impeded, delayed, and frustrated the fair examination of witnesses during the depositions Counsel defended. Thus, sanctions are independently appropriate based on Counsel’s excessive interruptions.

C. Appropriate Sanction

Based on Counsel’s deposition conduct, I would be well within my discretion to impose substantial monetary sanctions on Counsel. But I am less interested in negatively affecting Counsel’s pocketbook than I am in positively affecting Counsel’s obstructive deposition practices. I am also interested in deterring others who might be inclined to comport themselves similarly to Counsel. The Federal Rules specifically acknowledge that one function of discovery sanctions should be deterrence. See Fed. R. Civ. P. 26, advisory committee notes (1983 amendments) (“Sanctions to deter discovery abuse would be more effective if they were diligently applied ‘not merely to penalize those whose conduct may be deemed to warrant such a sanction, but to deter those who might be tempted to such conduct in the absence of such a deterrent.’” (quoting National Hockey League v. Metropolitan Hockey Club, 427 U.S. 639, 643 (1976)). Deterrence is especially important given that so many litigators are trained to make obstructionist objections. For instance, at trial, when I challenged Counsel’s use of “form” objections, Counsel responded, “Well, I’m sorry, Your Honor, but that was my training . . . .” While monetary sanctions are certainly warranted for Counsel’s witness coaching and excessive interruptions, a more outside-the-box sanction15 may better serve the goal of


15 For examples of outside-the-box discovery sanctions, see the following cases: St. Paul Reinsurance Co., 198 F.R.D. at 518 (imposing a write-a-bar-journal-article sanction); R.E. Linder Steel Erection Co., Inc. v. U.S. Fire Ins. Co., 102 F.R.D. 39, 41 (D. Md. 1983) (imposing a $5.00-per-inturruption sanction); Huggins v. Coatesville Area Sch. Dist., No. CIV.A. 07-4917, 2009 WL 2973044, at *4 (E.D. Pa. Sept. 16, 2009) (imposing a sit-down-and-share-a-meal-together sanction).

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 31 of 34


changing improper tactics that modern litigators are trained to use. See Matthew L. Jarvey, Note, Boilerplate Discovery Objections: How They Are Used, Why They Are Wrong, and What We Can Do About Them, 61 DRAKE L. REV. 913, 931-36 (2013) (discussing the importance of unorthodox sanctions in deterring discovery abuse).

In light of this goal, I impose the following sanction: Counsel must write and produce a training video in which Counsel, or another partner in Counsel’s firm, appears and explains the holding and rationale of this opinion, and provides specific steps lawyers must take to comply with its rationale in future depositions in any federal and state court.16 The video must specifically address the impropriety of unspecified “form” objections, witness coaching, and excessive interruptions. The lawyer appearing in the video may mention the few jurisdictions that actually require only unspecified “form” objections and may suggest that such objections are proper in only those jurisdictions. The lawyer in the video must state that the video is being produced and distributed pursuant to a federal court’s sanction order regarding a partner in the firm, but the lawyer need not state the name of the partner, the case the sanctions arose under, or the court issuing this order. Upon completing the video, Counsel must file it with this court, under seal, for my review and approval. If and when I approve the video, Counsel must (1) notify certain lawyers at Counsel’s firm about the video via e-mail and (2) provide those lawyers with access to the video. The lawyers who must receive this notice and access include each lawyer at Counsel’s firm—including its branch offices worldwide—who engages in federal or state litigation or who works in any practice group in which at least two of the lawyers have filed an appearance in any state or federal case in the United States. After


16 I am not the first judge to suggest a video-related sanction. In Florida Bar v. Ratiner, 46 So. 3d 35, 41 n.4 (Fla. 2010), the Florida Supreme Court noted that law students and members of the Florida bar could view footage of a videotaped deposition in which a later-suspended lawyer behaved unprofessionally toward his opposing counsel as part of a course on professionalism.

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 32 of 34


providing these lawyers with notice of and access to the video, Counsel must file in this court, under seal, (1) an affidavit certifying that Counsel complied with this order and received no assistance (other than technical help or help from the lawyer appearing in the video) in creating the video’s content and (2) a copy of the e-mail notifying the appropriate lawyers in Counsel’s firm about the video. The lawyer appearing in the video need not state during the video that he or she agrees with this opinion, or that Counsel was the lawyer whose deposition conduct prompted this sanction. Counsel need not make the video publicly available to anyone outside Counsel’s firm. Failure to comply with this order within 90 days may result in additional sanctions.

To be clear, had Counsel made only a handful of improper objections or comments while taking depositions, I would not have raised these issues sua sponte. Depositions can be stressful and contentious, and lawyers are bound to make the occasional improper objection. But Counsel’s improper objections, coaching, and interruptions went far beyond what judges should tolerate of any lawyer, let alone one as experienced and skilled as Counsel. Counsel’s baseless interjections and obstructionist commentary were ubiquitous; they pervaded the depositions in this case and even spilled over into the trial. It is the repeated nature of Counsel’s obstructionist deposition conduct that warrants sanctions here.

Finally, I note that, despite Counsel’s deposition conduct, I was greatly impressed by how Counsel performed at trial. Unlike the “litigators” I discussed earlier, Counsel was extremely well-prepared, had clearly mastered the facts of this case, and did a great job of incorporating electronic evidence into Counsel’s direct- and cross-examinations. Those aspects of Counsel’s noteworthy trial skills, expertise, and preparation are laudable, but they do not excuse Counsel’s pretrial conduct.

If Counsel appeals this sanctions order I will, sua sponte, automatically stay it pending the appeal.

Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 33 of 34



For the reasons stated in this opinion, I find that sanctions are appropriate in response to Counsel’s improper deposition conduct, which impeded, delayed, and frustrated the fair examination of witnesses in the depositions related to this case that Counsel defended. I therefore impose the sanction described above.


DATED this 28th day of July, 2014.





Case 5:11-cv-04017-MWB Document 205 Filed 07/28/14 Page 34 of 34



Do you think Judge Bennett’s order was too easy on this attorney, too strict, or just about right. Why or why not? What do you think about the judge not even naming the attorney? Is that appropriate in a sanctions order? Do you think this kind of discovery misconduct could have had an impact on the outcome of the case? Was it fair to the plaintiff, who lost this very serious case? Does this become grounds for a new trial or appeal? What do you think? Please leave a comment below.

Latest Grossman and Cormack Study Proves Folly of Using Random Search For Machine Training – Part Four

August 3, 2014

This is the conclusion of my four part blog: Latest Grossman and Cormack Study Proves Folly of Using Random Search For Machine Training – Part One and Part Two and Part Three.

Cormack and Grossman’s Conclusions

Maura-and-Gordon_Aug2014Gordon Cormack and Maura Grossman have obviously put a tremendous amount of time and effort into this study. In their well written conclusion they explain why they did it, as well as provide a good summary of their findings

Because SPL can be ineffective and inefficient, particularly with the low-prevalence collections that are common in ediscovery, disappointment with such tools may lead lawyers to be reluctant to embrace the use of all TAR. Moreover, a number of myths and misconceptions about TAR appear to be closely associated with SPL; notably, that seed and training sets must be randomly selected to avoid “biasing” the learning algorithm.

This study lends no support to the proposition that seed or training sets must be random; to the contrary, keyword seeding, uncertainty sampling, and, in particular, relevance feedback – all non-random methods – improve significantly (P < 0:01) upon random sampling.

While active-learning protocols employing uncertainty sampling are clearly more effective than passive-learning protocols, they tend to focus the reviewer’s attention on marginal rather than legally significant documents. In addition, uncertainty sampling shares a fundamental weakness with passive learning: the need to define and detect when stabilization has occurred, so as to know when to stop training. In the legal context, this decision is fraught with risk, as premature stabilization could result in insufficient recall and undermine an attorney’s certification of having conducted a reasonable search under (U.S.) Federal Rule of Civil Procedure 26(g)(1)(B).

This study highlights an alternative approach – continuous active learning with relevance feedback – that demonstrates superior performance, while avoiding certain problems associated with uncertainty sampling and passive learning. CAL also offers the reviewer the opportunity to quickly identify legally significant documents that can guide litigation strategy, and can readily adapt when new documents are added to the collection, or new issues or interpretations of relevance arise.

Evaluation of Machine-Learning Protocols for Technology-Assisted Review in Electronic DiscoverySIGIR’14, July 6–11, 2014, at pg. 9.

The insights and conclusions of Cormack and Grossman are perfectly in accord with my own experience and practice with predictive coding search efforts, both with messy real world projects, and the four controlled scientific tests I have done over the last several years (only two of which have to date been reported, and the fourth is still in progress). I agree that a relevancy approach that emphasizes high ranked documents for training is one of the most powerful search tools we now have. So too is uncertainty training (mid ranked) when used judiciously, as well as keywords, and a number of other methods. All the many tools we have to find both relevant and irrelevant documents for training should be used, depending on the circumstances, including even some random searches.

In my view, we should never use just one method to select documents for machine training, and ignore the rest, even when it is a good method like Cormack and Grossman have shown CAL to be. When the one method selected is the worst of all possible methods, as random search has now been shown to be, then the monomodal approach is a recipe for ineffective, over-priced review.

Why All the Foolishness with Random Search?

random samplingAs shown in Part One of this article, it is only common sense to use what you know to find training documents, and not rely on a so-called easy way of rolling dice. A random chance approach is essentially a fool’s method of search. The search for evidence to do justice is too important to leave to chance. Cormack and Grossman did the legal profession a favor by taking the time to prove the obvious in their study. They showed that even very simplistic mutlimodal search protocols, CAL and SAL, do better at machine training than monomodal random only.

scientist on simpsonInformation scientists already knew this rather obvious truism, that multimodal is better, that the roulette wheel is not an effective search tool, that random chance just slows things down and is ineffective as a machine training tool. Yet Cormack and Grossman took the time to prove the obvious because the legal profession is being led astray. Many are actually using chance as if it that were a valid search method, although perhaps not in the way they describe. As Cormack and Grossman explained in their report:

While it is perhaps no surprise to the information retrieval community that active learning generally outperforms random training [22], this result has not previously been demonstrated for the TAR Problem, and is neither well known nor well accepted within the legal community.

Evaluation of Machine-Learning Protocols for Technology-Assisted Review in Electronic DiscoverySIGIR’14, July 6–11, 2014 at pg. 8.

As this quoted comment suggests, everyone in the information science search community knew this already, that the random only approach to search is inartful. So do most lawyers, especially the ones with years of hands-on experience in search for relevant ESI. So why in the world is random search only still promoted by some software companies and their customers? Is it really to address the so called problem of “not knowing what you don’t know.” That is the alleged inherent bias of using knowledge to program the AI. The total-random approach is also supposed to prevent overt, intentional bias, where lawyers might try to mis-train the AI searcher algorithm on purpose. These may be the stated reasons by vendors, but there are other reasons. There must be, because these excuses do not hold water. This was addressed in Part One of this article.

This bias-avoidance claim must just be an excuse because there are many better ways to counter myopic effects of search driven too narrowly. There are many methods and software enhancements that can be used to avoid overlooking important, not yet discovered types of relevant documents. For instance, allow machine selection of uncertain documents, as was done here with the SAL protocol. You could also include some random document selection into the mix, and not just make the whole thing random. It is not all or nothing, not logically at least, but perhaps it is as a practical matter for some software.

My preferred solution to the problem of “not knowing what you don’t know” is to use a combination of all those methods, buttressed by a human searcher that is aware of the limits of knowledge. In mean, really! The whole premise behind using random as the only way to avoid a self-looping trap of “not knowing what you don’t know” assumes that the lawyer searcher is a naive boob or dishonest scoundrel. It assumes lawyers are unaware that they don’t know what they don’t know. Please, we know that perfectly well. All experienced searchers know that. This insight is not just the exclusive knowledge of engineers and scientists. Very few attorneys are that arrogant and self absorbed, or that naive and simplistic in their approach to search.

No, this whole you must use random only search to avoid prejudice is just a smoke screen to hide real reason a vendor sells software that only works that way. The real reason is that poor software design decisions were made in a rush to get predictive coding software to market. Software was designed to only use random search because it was easy and quick to build software like that. It allowed for quick implementation of machine training. Such simplistic types of AI software may work better than poorly designed keyword searches, but it is still far inferior to more complex machine training system, as Cormack and Grossman have now proven. It is inferior to a multimodal approach.

The software vendors with random only training need to move on. They need to invest in their software to adopt a multimodal approach. In fact, it appears that many have already done so, or are in the process. Yes, such software enhancements take time and money to implement. But we need software search tools for adults. Stop all of the talk about easy buttons. Lawyers are not simpletons. We embrace hard work. We are masters of complexity. Give us choices. Empower the software so that more than one method can be used. Do not force us to use only random selection.

We need software tools that respect the ability of attorneys to perform effective searches for evidence. This is our sand box. That is what we attorneys do, we search for evidence. The software companies are just here to give us tools, not to tell us how to search. Let us stop the arguments and move on to discuss more sophisticated search methods and tools that empower complex methods.

Attorneys want software with the capacity to integrate all search functions, including random, into a mulitmodal search process. We do not want software with only one type of machine training ability, be it CAL, SAL or SPL. We do not want software that can only do one thing, and then have the vendor build a false ideology around their one capacity that says their method is the best and only way. These are legal issues, not software issues.

Attorneys do not just want one search tool, we want a whole tool chest. The marketplace will sort out whose tools are best, so will science. For vendors to remain competitive they need to sell the biggest tool chest possible, and make sure the tools are well built and perform as advertised. Do not just sell us a screwdriver and tell us we do not need a hammer and pliers too.

Leave the legal arguments as to reasonability and rules to lawyers. Just give us the tools and we lawyers will find the evidence we need. We are experts at evidence detection. It is in our blood. It is part of our proud heritage, our tradition.

King_Solomon_JudgeFinding evidence is what lawyers do. The law has been doing this for millennia. Think back to story of the judicial decision of King Solomon. He decided to award the child to the woman he saw cry in response to his sham decision to cut the baby in half. He based his decision on the facts, not ideology. He found the truth in clever ways built around facts, around evidence.

Lawyers always search to find evidence so that justice can be done. The facts matter. It has always been an essential part of what we do. Lawyers always adapt with the times. We always demand and use the best tools available to do our job. Just think of Abraham Lincoln who readily used telegraphs, the great new high-tech invention of his day. When you want to know the truth of what happened in an event that took place in the recent past, you hire a lawyer, not an engineer nor scientist. That is what we are trained to do. We separate the truth from the lies. With great tools we can and will do an even better job.

Many multimodal based software vendors already understand all of this. They build software that empowers attorneys to leverage their knowledge and skills. That is why we use their tools. Empowerment of attorneys with the latest AI tools empowers our entire system of justice. That is why the latest Cormack Grossman study is so important. That is why I am so passionate about this. Join with us in this. Demand diversity and many capacities in your search software, not just one.

Vendor Wake Up Call and Plea for Change

Ralph_x-mas_2013My basic message to all manufacturers of predictive coding software who use only one type of machine training protocol is to change your ways. I mean no animosity at all. Many of you have great software already, it is just the monomondal method built into your predictive coding features that I challenge. This is a plea for change, for diversity. Sell us a whole tool chest, not just a single, super-simple tool.

Yes, upgrading software takes time and money. But all software companies need to do that anyway to continue to supply tools to lawyers in the Twenty-First Century. Take this message as both a wake up call and a respectful plea for change.

Dear software designers: please stop trying to make the legal profession look only under the random lamp. Treat your attorney customers like mature professionals who are capable of complex analysis and skills. Do not just assume that we do not know how to perform sophisticated searches. I am not the only attorney with multimodal search skills. I am just the only one with a blog who is passionate about it. There are many out there with very sophisticated skills and knowledge. They may not be as old (I prefer to say experienced) and loud mouthed (I prefer to say outspoken) as I am, but they are just as skilled. They are just as talented. More importantly, their numbers are growing rapidly. It is a generation thing too, you know. Your next generation of lawyer customers are just as comfortable with computers and big data as I am, maybe more so. Do you really doubt that Adam Losey and his generation will not surpass our accomplishments with legal search. I don’t.

Dear software designers: please upgrade your software and get with the multi-feature program. Then you will have many new customers, and they will be empowered customers. Do not have the money to do that? Show your CEO this article. Lawyers are not stupid. They are catching on, and they are catching on fast. Moreover, these scientific experiments and reports will keep on too. The truth will come out. Do you want to be survive the inevitable vendor closures and consolidation? Then you need to invest in more sophisticated, fully featured software. Your competitors are.

Dear software designers: please abandon the single feature approach, then you will be welcome in the legal search sandbox. I know that the limited functionality software that some of you have created is really very good. It already has many other search capacities. It just needs to be better integrated with predictive coding. Apparently some single feature software already produces decent results, even with the handicap of random-only. Continue to enhance and build upon your software. Invest in the improvements needed to allow for full multimodal, active, judgmental search.


Flashlights_taticalrandom only search method for predictive coding training documents is ineffective. The same applies to any other training method if it is applied to the exclusion of all others. Any experienced searcher knows this. Software that relies solely on a random only method should be enhanced and modified to allow attorneys to search where they know. All types of training techniques should be built into AI based software, not just random. Random may be easy, but is it foolish to only search under the lamp post. It is foolish to turn a blind eye to what you know. Attorneys, insist on having your own flashlight that empowers you to look wherever you want. Shine your light wherever you think appropriate. Use your knowledge. Equip yourself with a full tool chest that allows you to do that.

Latest Grossman and Cormack Study Proves Folly of Using Random Search For Machine Training – Part Three

July 27, 2014

This is part three of what has now become a four part blog: Latest Grossman and Cormack Study Proves Folly of Using Random Search for Machine Training – Part One and Part Two.

Professor Gordon Cormack

Professor Gordon Cormack

Yes, my article on this experiment and report by Professor Gordon Cormack and attorney Maura Grossman is rapidly becoming as long as the report itself, and, believe it or not, I am not even going into all of the aspects in this very deep, multifaceted study. I urge you to read the report. It is a difficult read for most, but worth the effort. Serious students will read it several times. I know I have. This is an important scientific work presenting unique experiments that tested common legal search methods.

The Cormack Grossman paper was peer reviewed by other scientists and presented at the major event for information retrieval scientists, called the annual ACM SIGIR conference. 12_acm-logo-medACM is the Association for Computing Machinery, the world’s largest educational and scientific computing society. SIGIR is the Special Interest Group On Information Retrieval section of ACM. Hundreds of scientists and academics served on organizing committees for the 2014 SIGIR conference in Australia. They came from universities and large corporate research labs from all over the world, including Google, Yahoo, and IBM. Here is a list with links to all of the papers presented.

All attorneys who do legal search should at least have a rudimentary understanding of the findings of Cormack and Grossman on the predictive coding training methods analyzed in this report. That is why I am making this sustained effort to provide my take on it, and make their work a little more accessible. Maura and Gordon have, by the way, generously given of their time to try to insure that my explanations are accurate. Still, any mistakes made on that account are solely my own.

Findings of Cormack Grossman Study

rouletteHere is how Cormack and Grossman summarize their findings:

The results presented here do not support the commonly advanced position that seed sets, or entire training sets, must be randomly selected [19, 28] [contra 11]. Our primary implementation of SPL, in which all training documents were randomly selected, yielded dramatically inferior results to our primary implementations of CAL and SAL, in which none of the training documents were randomly selected.

Evaluation of Machine-Learning Protocols for Technology-Assisted Review in Electronic DiscoverySIGIR’14, July 6–11, 2014, at pgs. 7-8.


Now for the details of the results comparing the previously described methods of CAL, SAL and SPL. First, let us examine the comparison between the CAL and SPL machine training methods. To refresh your memory, CAL is simplistic type of multimodal training method wherein two methods are used. Keyword search results are used in the first round of training. In all following rounds, high probability ranked search results are used. SPL is a pure random method, a monomodal method. With SPL all documents are selected by random sampling for training in all rounds.


Cormack and Grossman found that the “CAL protocol achieves higher recall than SPL, for less effort, for all of the representative training-set sizes.” Id. at pg. 4. This means you can find more relevant documents using CAL than a random method, and you can do so faster and thus with less expense.

To drill down even deeper into their findings it is necessary to look at the graphs in the report that show how the search progressed through all one-hundred rounds of training and review for various document collections. This is shown for CAL v. SPL in Figure 1 of the report. Id. at pg. 5. The line with circle dots at the top of each graph plots the retrieval rate of CAL, the clear winner on each of the eight search tasks tested. The other three lines show the random approach, SPL, using three different training-set sizes.Cormack_Grossman_Fig1  Cormack and Grossman summarize the CAL v. SPL findings as follows:

After the first 1,000 documents (i.e., the seed set), the CAL curve shows a high slope that is sustained until the majority of relevant documents have been identified. At about 70% recall, the slope begins to fall off noticeably, and effectively plateaus between 80% and 100% recall. The SPL curve exhibits a low slope for the training phase, followed by a high slope, falloff, and then a plateau for the review phase. In general, the slope immediately following training is comparable to that of CAL, but the falloff and plateau occur at substantially lower recall levels. While the initial slope of the curve for the SPL review phase is similar for all training-set sizes, the falloff and plateau occur at higher recall levels for larger training sets. This advantage of larger training sets is offset by the greater effort required to review the training set: In general, the curves for different training sets cross, indicating that a larger training set is advantageous when high recall is desired.


The Cormack Grossman experiment also compared the CAL and SAL methods. Recall the SAL method is another simple multimodal method where only two methods are used to select training documents. Keywords are again used in the first round only, just like the CAL protocol. Thereafter, in all subsequent rounds of training machine selected documents are used based on the machine’s uncertainty of classification. That means the search is focused on the midrange ranked documents about which the machine is most uncertain.


Cormack and Grossman found that “the CAL protocol generally achieves higher recall than SAL,” but the results were closer and more complex. Id. At one point in the training SAL became as good as CAL, it achieved a specific recall value with the nearly the same efforts as CAL from that point forward. The authors found that was due to the fact that many high probability documents began to be used by the machine as uncertainty selected documents. This happened after all of the mid-scoring documents had been used up. In other words, at some point the distinction between the two methods was decreased, and more high probability documents were used in SAL, in almost the same way they were used in CAL. That allowed SAL to catch up with CAL and, in effect, become almost as good.

This catch up point is different in each project. As Cormack and Grossman explain:

Once stabilization occurs, the review set will include few documents with intermediate scores, because they will have previously been selected for training. Instead, the review set will include primarily high-scoring and low-scoring documents. The high-scoring documents account for the high slope before the inflection point; the low-scoring documents account for the low slope after the inflection point; the absence of documents with intermediate scores accounts for the sharp transition. The net effect is that SAL achieves effort as low as CAL only for a specific recall value, which is easy to see in hindsight, but difficult to predict at the time of stabilization.

This inflection point and other comparisons can be easily seen in Figure 2 of the report (shown below). Id. at pg. 6. Again the line with circle dots at the top of each graph, the one that always starts off fastest, plots the retrieval rate of CAL. Again, it does better than in each of the eight search tasks tested. The other three lines show the uncertainty approach, SAL, using three different training-set sizes. CAL does better than SAL in all eight of the matters, but the differences are not nearly as great as the comparison between CAL and SPL.

Cormack_Grossman_Fig2 Cormack and Grossman summarize the CAL v. SAL findings as follows:

Figure 2 shows that the CAL protocol generally achieves higher recall than SAL. However, the SAL gain curves, unlike the SPL gain curves, often touch the CAL curves at one specific inflection point. The strong inflection of the SAL curve at this point is explained by the nature of uncertainty sampling: Once stabilization occurs, the review set … (see quote above for the rest of this sentence.)

This experiment compared one type of simple multimodal machine training method with another. It found that with the data sets tested, and other standard procedures set forth in the experiment, the method which used high ranking documents for training, what William Webber calls the Relevance method, performed somewhat better than the method that used mid-ranked documents, what Webber calls the Uncertainty method.

This does not mean that the uncertainty method should be excluded from a full multimodal approach in real world applications. It just means that here, in this one experiment, albeit a very complex and multifaceted experiment, the relevance method outperformed the uncertainty method.

I have found that in the real world of very complex (messy even) legal searches, it is good to use both high and mid-ranked documents for training, what Cormack and Grossman call CAL and SAL, and what Webber calls Relevance, and Uncertainty training. It all depends on the circumstances, including the all important cost component. In the real world you use every method you can think of to help you to find what you are looking for, not just one or two, but dozens.

Grossman and Cormack know this very well too, which I know from private conservations with them on this, and also from the conclusion to their report:

There is no reason to presume that the CAL results described here represent the best that can be achieved. Any number of feature engineering methods, learning algorithms, training protocols, and search strategies might yield substantive improvements in the future. The effect of review order and other human factors on training accuracy, and thus overall review effectiveness, may also be substantial.

Evaluation of Machine-Learning Protocols for Technology-Assisted Review in Electronic DiscoverySIGIR’14, July 6–11, 2014, at pg. 9.

Ralph Losey with some of his many computer tools

My practical takeaway from the Cormack Grossman experiment is that focusing on high ranking documents is a powerful search method. It should be given significant weight in any multimodal approach, especially when the goal is to quickly find as many relevant documents as possible. The “continuous” training aspects of the CAL approach are also intriguing, that is you keep doing machine training throughout the review project and batch reviews accordingly. This could become a project management issue, but, if you can pull it off within proportionality and requesting party constraints, it just makes common sense to do so. You might as well get as much help from the machine as possible and keep getting its probability predictions for as long as you are still doing reviews and can make last minute batch assignments accordingly.

I have done several reviews in such a continuous training manner without really thinking about the fact the machine input was continuous, including my first Enron experiment. Predictive Coding Narrative: Searching for Relevance in the Ashes of Enron. But this causes me to rethink the flow chart shown below that I usually use to explain the predictive coding process. The work flow shown is not a CAL approach, but rather a SAL type of approach where there is a distinct stop in training after step five, and the review work in step seven is based on the last rankings established in step five.


The continuous work flow is slightly more difficult to show in a diagram, and to implement, but it does make good common sense if you are in a position to pull it off. Below is the revised workflow to update the language and show how the training continues throughout the review.


Machine training is still done in steps four and five, but then continues in steps four, five and seven. There are other ways it could be implemented of course, but this is the CAL approach I would use in a review project where such complex batching and continuous training otherwise makes sense. Of course, it is not necessary in any project were the review in steps four and five effectively finds all of the relevant documents required. This is what happened in my Enron experiment. Predictive Coding Narrative: Searching for Relevance in the Ashes of EnronThere was no need to do a proportional final review, step seven, because all the relevant documents had already been reviewed as part of the machine training review in steps four and five. In the Enron experiment I skipped step seven and when right from step six to step eight, production. I have been able to do this is other projects as well.

Strengths of a Relevancy Weighted Type of CAL

The findings in this experiment as to the strengths of using Relevancy training confirm what I have seen in most of my search projects. I usually start with the high end documents to quickly help me to teach the machine what I am looking for. I find that this is a good way to start training. Again, it just makes common sense to do so. It is somewhat like teaching a human, or a dog for that matter. You teach the machine relevance classification by telling it when it is right (positive reinforcement), and when it is wrong. This kind of feedback is critical in all learning. In most projects this kind of feedback on predictions of highly probable relevance is the fastest way to get to the most important documents. For those reasons I agree with Cormack and Grossman’s conclusion that CAL is a superior method to quickly find the most relevant documents:

CAL also offers the reviewer the opportunity to quickly identify legally significant documents that can guide litigation strategy, and can readily adapt when new documents are added to the collection, or new issues or interpretations of relevance arise.

Id. But then again, I would never rely on just Relevancy CAL type searches alone. It gets results fast, but also tends to lead to a somewhat myopic focus on the high end where you may miss new, different types of relevant documents. For that reason, I also use SAL types of searches to include the mid range documents from the Uncertainty method. That is an important method to help the machine to better understand what documents I am looking for. As Cormack and Grossman put it:

The underlying objective of CAL is to find and review as many of the responsive documents as possible, as quickly as possible. The underlying objective of SAL, on the other hand, is to induce the best classier possible, considering the level of training effort. Generally, the classier is applied to the collection to produce a review set, which is then subject to manual review.

Id. at 8.

Similarity and other concept type search methods are also a good way to quickly find as many responsive documents as possible. So too are keyword searches, and not just in the first round, but for any round. Further, this experiment, which is already very complex (to me at least), does not include the important real world component of highly relevant versus merely relevant documents. I never just train on relevancy alone, but always include a hunt for the hot documents. I want to try to train the machine to understand the difference between the two classifications. Cormack and Grossman do not disagree. As they put it, “any number of feature engineering methods, learning algorithms, training protocols, and search strategies” could improve upon a CAL only approach.

There are also ways to improve the classifier in addition to focus on mid range probability documents, although I have found that uncertainty method is the best way to improve relevance classifications. But, it also helps to be sure your training on the low end is also right, meaning review of some of the high probability irrelevant documents. Both relevant and irrelevant training are helpful. Personally, I also like to include some random aspects, especially at first, to be sure I did not miss any outlier type documents, and be sure I have a good feel for the irrelevant documents of these custodians too. Yes, chance has to place too, so long as it does not take over and become the whole show.

Supplemental Findings on Random Search

diceIn addition to comparing CAL with SAL and SPL, Cormack and Grossman experimented with what would happen to the effectiveness of both the CAL and SAL protocols if more random elements were added to the methods. They experimented with a number of different variables, including substituting random selection, instead of keyword, for the initial round of training (seed set).

As you would expect, the general results were to decrease the effectiveness of every search method wherein random was substituted, either for keyword, high ranking relevance, or mid ranking relevance (uncertainty). The negative impact was strongest in datasets where prevalence was low, which is typical in litigation. Cormack and Grossman tested eight datasets where the prevalence of responsive documents varied from 0.25% to 3.92%, which, as they put it: “is typical for the legal matters with which we have been involved.” The size of the sets tested ranged 293,000 documents to just over 1.1 million. The random based search of lowest prevalence dataset tested, matter 203, the one with a 0.25% prevalence rate, was, in their words, a spectacular failure. Conversely, the negative impact was lessened with higher prevalence datasets. Evaluation of Machine-Learning Protocols for Technology-Assisted Review in Electronic DiscoverySIGIR’14, July 6–11, 2014, at pg. 7.

Cormack and Grossman responded to the popular misconception that predictive coding does not work in such low prevalence datasets.

Others assert that these are examples of “low-prevalence” or “low-richness” collections, for which TAR is unsuitable [19]. We suggest that such assertions may presuppose an SPL protocol [11], which is not as effective on low-prevalence datasets. It may be that SPL methods can achieve better results on higher-prevalence collections (i.e., 10% or more responsive documents).

Id. at 9.

In fact, information scientists have been working with low prevalence datasets for decades, which is one reason Professor Cormack had a ready collection of pre-coded documents by which to measure recall, a so-called gold standard of assessments from prior studies. Cormack and Grossman explain that the lack of pre-tested datasets with high prevalence is the reason they did not use such collections for testing. They also speculate that if such high prevalence datasets are tested, then the random only (SPL) method would do much better than it did in the low prevalence datasets they used in their experiment.

However, no such collections were included in this study because, for the few matters with which we have been involved where the prevalence exceeded 10%, the necessary training and gold-standard assessments were not available. We conjecture that the comparative advantage of CAL over SPL would be decreased, but not eliminated, for high-prevalence collections.


They are probably right, if the datasets have a higher prevalence, then the chances are that random samples will find more relevant documents for training. But that still does not make the blind draw a better way to find things than looking with your eyes wide open. Plus, the typical way to attain high yield datasets is by keyword filtering out large segments of the raw data before beginning a predictive coding search. When you keyword filter like that before beginning machine training the chances are you will leave behind a significant portion, if not most of the relevant documents. Keyword filtering often has low recall, or when broad enough to include most of the relevant documents, it is very imprecise. Then you are back to the same low prevalence situation.

Better to limit filtering before machine training to obvious irrelevant, or ESI not appropriate for training, such as non-text documents like photos, music and voice mail. Use other methods to search for those types of ESI. But do not use keyword filtering on text documents simply to create an artificially high prevalence just because the random based software you use will only work that way. That is the tail wagging the dog.

For more analysis and criticism on using keywords to create artificially high prevalence, a practice Cormack and Grossman call Collection Enrichment, see another excellent article they wrote: Comments on “The Implications of Rule 26(g) on the Use of Technology-Assisted Review”7 Federal Courts Law Review 286 (2014) at pgs. 293-295, 300-301. This article also contains good explanations of the instant study with CAL, SAL and SPL. See especially Table 1 at pg. 297.

The negative impact of random elements on machine training protocols is a no duh to experienced searchers. See eg. the excellent series of articles by John Tredennick, including his review on the Cormack Grossman study: Pioneering Cormack/Grossman Study Validates Continuous Learning, Judgmental Seeds and Review Team Training for Technology Assisted Review.

It never helps to turn to lady luck, to random chance, to improve search. Once you start relying on dice to decide what to do, you are just spinning your wheels.

Supplemental Findings on Keywords and Random Search

go fishCormack and Grossman also tested what would happen if keywords were used instead of random selections, even when the keywords were not tested first against the actual data. This poor practice of using unverified keywords is what I call the Go Fish approach to keyword search. Child’s Game of “Go Fish” is a Poor Model for e- Discovery Search(October 2009). Under this naive approach attorneys simply guess what keywords might be contained on relevant documents without testing how accurate their guesses are. It is a very simplistic approach to keyword search, yet, nevertheless, is still widely employed in the legal profession. This approach has been criticized by many, including Judge Andrew Peck in his excellent Gross Construction opinion, the so called wake-up call for NY attorneys on search. William A. Gross Construction Associates, Inc. v. American Manufacturers Mutual Insurance Co., 256 F.R.D. 134 (S.D.N.Y. 2009).

Cormack and Grossman also tested what would happen if such naive keyword selections were used instead of the high or mid probability methods (CAL and SAL) for machine training. The naive keywords used in these supplemental comparison tests did fairly well. This is consistent with my multimodal approach, where all kinds of search methods are used in all rounds of training.

The success of naive keyword selection for machine training is discussed by Cormack and Grossman as an unexpected finding (italics and parens added):

Perhaps more surprising is the fact that a simple keyword search, composed without prior knowledge of the collection, almost always yields a more effective seed set than random selection, whether for CAL, SAL, or SPL. Even when keyword search is used to select all training documents, the result is generally superior to that achieved when random selection is used. That said, even if (random) passive learning is enhanced using a keyword-selected seed or training set, it (passive learning) is still dramatically inferior to active learning. It is possible, in theory, that a party could devise keywords that would render passive learning competitive with active learning, but until a formal protocol for constructing such a search can be established, it is impossible to subject the approach to a controlled scientific evaluation. Pending the establishment and scientific validation of such a protocol, reliance on keywords and (random) passive learning remains a questionable practice. On the other hand, the results reported here indicate that it is quite easy for either party (or for the parties together) to construct a keyword search that yields an effective seed set for active learning.

Id. at 8.

Cormack and Grossman summarize their findings on the impact of keywords in the first round of training (seed set) on CAL, SAL and SPL:

In summary, the use of a seed set selected using a simple keyword search, composed prior to the review, contributes to the effectiveness of all of the TAR protocols investigated in this study.

Keywords still have an important place in any multimodal, active, predictive coding protocol. This is, however, completely different from using keywords, especially untested naive keywords, to filter out the raw data in a misguided attempt to create high prevalence collections, all so that the random method (passive) might have some chance of success.

To be continued . . . in Part Four I will conclude with final opinions and analysis and my friendly recommendations for any vendors still using random-only training protocols. 


Get every new post delivered to your Inbox.

Join 3,484 other followers