e-Discovery Team

Guest Blog by William F. Hamilton and Lawrence C. Chapin.

Bill Hamilton is an attorney with nearly thirty years of experience in business litigation who is a partner at Quarles & Brady. Bill also serves as the Dean of the E-Discovery Department of Bryan University, which includes an online educational program in e-discovery project management. Bill is also an Adjunct Law Professor teaching Electronic Discovery and Digital Evidence at the University of Florida, and has frequently contributed to this blog. See Eg. The E-Discovery Crisis: An Immediate Challenge to Our Nation’s Law Schools, and The E-Discovery Sanctions Cube.

Larry Chapin is an attorney with 30+ years experience, including corporate Wall Street law, who now works as a contract review lawyer in New York City. Larry has taught at the New School for Social Research in NYC and currently serves on the Board of Directors for an asset management company in Stockholm Sweden. Larry is the first graduate of our e-Discovery Team Training program. He contributed a must-read blog here earlier this year entitled Contract Coders: e-Discovery’s “Wasting Asset”?

_____________

EDITOR’S NOTE: Over the last several blogs on the Secrets of Search we have examined the latest scientific research on manual and automated reviews. The research shows that although brute-force manual linear review is as dead as a doornail, or should be, there is still an important place for skilled human reviewers and review, even in the latest predictive coding models. But the emphasis is on skilled human reviewers and skilled methods. Simply asking some lawyers to look at documents all day on a computer screen for weeks on end and decide relevance or not is unacceptable. If that is how you conduct manual reviews, and just bid things out to the lowest paid reviewers, then you are inviting error. You probably would be better off turning it over to the Borg, and just skipping final quality control reviews altogether. But if you care about quality, if you are diligent in the protection of client confidentiality – and as a lawyer you have a clear ethical duty to do so – then you must improve and innovate on manual review. This guest blog by professional reviewer, Larry Chapin, and an expert in e-discovery and project management, Bill Hamilton, help show the way.

In Part III of Secrets of Search I listed a nine-point checklist for quality reviews. Point number six was: “New tools and psychological techniques (e.g. game theory, story telling) to facilitate prolonged concentration … ” This guest blog will flesh out a new approach that Chapin and Hamilton have developed to use storytelling to improve the quality of contract reviews. I think this is a great idea. Lawsuits are essentially a battle of competing stories. They can become high drama as the Casey Anthony trial that took place across from my office in Orlando showed in 2011. Good trial lawyers already know the importance of story to a case. They should quickly understand this idea and appreciate how this new review technique could help their cause. All attorneys, and especially companies that do contract review work, should look into including this new technique into their projects. Feel free to email Bill Hamilton or Larry Chapin to see how they may be able to assist.

_____________

By putting its faith in logic, control and optimization, command-and-control management has lost sight of the crucial role that passion plays in human action.

Stephen Denning, The Leader’s Guide to Storytelling

_________

Storytelling: The Shared Quest  For Excellence in Document Review

by William F. Hamilton and Lawrence C. Chapin

What is the future of large-scale human document reviews? With the startling advances of search technology, is human document review about to be consigned to the dustbin of history? Some believe so. Yet, others think that the death of human review has been grossly exaggerated. There is no doubt that computer assisted reviews will be increasingly important for large and even moderate scale reviews. However, the contest between human and computer, between manual and automated review is far from over. In this blog, Ralph Losey recently discussed some of the implications of the fascinating work of information scientist William Webber. It seems that in the proper setting, the best human reviewers can still out-perform the automated review.

Watson may be the Jeopardy winner, and IBM’s Deep Blue the chess champion, but the identification and evaluation of documents in the litigation context stretches the utility of computer algorithms. In document review setting, well-trained, well-led and properly motivated women and men are, in fact, able to excel. How can we build reviews to maximize human review performance? What can be done about the powerful disincentives of long hours of dreadfully monotonous work at rates of pay already low and still in decline? Put more constructively, what can be done to tap the intelligence, marshal the talents, and harness the energies of the contract lawyers who fill the ranks in the typical review? How do we rid ourselves of the upstairs-downstairs mentality that isolates and confines our reviewers, turns them into servants and cripples their reviews?

We believe that the answer may be found in an approach to document review that harkens back to a simpler time, before litigators faced the enormous volumes of documents common in our digital age. That is to say, answers are to be found in building reviews around the art of storytelling. Shakespeare was right: the entire world’s a stage, and all the men and women players. Certainly, litigation is drama. It is the drama of competing and clashing human passions. It is the stuff of stories. Document reviews must be understood as a central player in the litigation storytelling process.

A fundamental shift in the way that lawyers think, speak, and conduct document reviews is required. We propose a new paradigm. We propose building  “story-centric” reviews. First, though, let’s face it. Storytelling usually gets a hard knock. It’s for children. It’s the stuff of fairy tales. Storytelling is said to have no place in the hard-edged, logic driven, command-and- control culture to which the legal and business communities have grown accustomed. Euphemisms – like “business narrative” – have been invented so that stories might have a place of some kind in the working world.

Yet, storytelling has long been a part of lawyering. Good trial lawyers have always known that cases are won on the strength of their story. Even crazy ones can be convincing. Empirical studies also show that appellate briefs, too, are more persuasive if they tell stories rather than rely on logic alone.  A case can’t resonate with a judge or jury – emotionally, intellectually, or intuitively – unless it’s tied to a compelling story. The litigation team itself can’t know what evidence most belongs before the court unless it knows the story to which the evidence belongs. The discovery process serves to yield the elements and the contours of the story, and shed light on the connections between the cause and effect that are at its heart. It is the job of the entire team – including document reviewers – to construct the most persuasive story possible, and to diminish and discredit the tale told by the other side.

Our experience, unfortunately, is that too many lawyers separate document review from that creative process. They fail to see document reviewers for what they are: investigators sharing fully in the common tasks of discerning, shaping, and telling the client’s story. This kind of engagement requires that the review structure and evaluation adopt the elements and language of the story. It’s an orientation that triggers active reviewer participation and has real potential to address the problems now plaguing review. We believe that the failure to engage the review team in this way results in a process that is less true and just than it might be.

Suggestions to Add Story to Document Reviews

Accordingly, we offer a series of suggestions for the use of storytelling in the discovery process, toward building a story-centric review.

First, at the outset, use the client’s story and its themes to define the goal of the review project. Articulate clearly the central purpose of every reviewer’s contribution: to enable the story to be told. The story needs to remain the constant center of their focus. We might liken reviewers to crew members who sailed in search of new lands during the great age of exploration. Not every day was filled with adventure. During more days than we realize, their ships were becalmed on windless seas. What got them through those days was their purpose for being there, the vision of things that had launched their journey. So they kept their focus, mindful always of the possibility of a sighting and the promise of discovery. In that way, let the story of the case be what drives and sustains the review team. Remember that the critical document, like new land, may be just a moment away. Everyone needs to stay alert.

Project metrics should be designed to reflect this orientation. Story-centric metrics should measure: linkage, the degree to which documents pull the story together tightly to help tell the tale; gravity, the degree to which the document collection gives weight, heft and power to the tale; and resonance, the degree to which documents provide compound richness to the story.

“Linkage Docs” provide the basic story line. They establish the necessary cause and effect that transforms otherwise isolated facts into a real story. They reflect the fact that every story is composed of details that unfold at a time, place, during a temporal extension, and that involve human motivations and conflicts. They are the sinews, the connecting tissues without which a story does not exist. For example, in a case involving a business breach of contract for failure to maintain premises, a document that shows the defendant’s  financial distress shortly before the breach establishes linkage. Linkage allows the story to begin to congeal.

Links are related to gravity, but different. “Gravity Docs” are those documents that move the story events out of stasis towards resolution. They function as a pivotal column or anchor that marks a transition, direction or resolution within the story. We ultimately want links that tie to these pivotal columns. The documents with gravity are the turning point documents.

Finally, “Resonance Docs” are those documents that strike a chord in us. They evoke sympathies in ways that align us with the actors in the story. They establish decisive commonalities between persons hearing the story and those person within it. In helping the story ring true, they persuade us. The lead us safely past any temptation to turn to unpersuasive  clichés, triteness, and banality in telling the story. A document that provides resonance will  tie story links (sub-plots) and pivotal gravity markers (the main plot) together.

The story can have links  and pivoting documents, and still be unpersuasive. Resonating documents provide  understanding, the “now I get it feeling,” and are often documents that directly speak to human motivation and intention ( or give rise to strong presumptions of actual motivation).  The irony of the traditional review is that a  review team shackled by  traditional coding blinders can row past a proverbial “smoking gun” document and not recognize its value to the story. Reviewers should not resemble the galley rowers portrayed in Ben-Hur who are driven to exhaustion as the pace of the review escalates to ramming speed.

The review team must be able to recognize documents with story-centric values, not merely label documents  as responsive or non-responsive according to abstract coding rules. A good review team requires graphics. The review team’s identification of Linkage Docs, Gravity Docs and Resonance Docs compose the story as the review progresses. The review team needs to literally see the story mapped as it develops. The story-centric review replaces the  traditional white board with a large story board that simultaneously shapes and is shaped by the review.

Linkage, gravity, and resonance can be seen as three overlapping circles. In practice, depending upon the story, the circles may vary in size and shape (e.g. oblong),  but in the overlapping section we are likely to find the 7±2 documents that the trial team needs to tell the winning story.

So invest your own time in a solid understanding of the client’s story. Invest more time still in discussing it with the review team, so that together you reach a shared grasp of its themes and important facts. This initial investment may turn out to be substantial, but the rewards will be enormous. Don’t make the mistake of taking more time to talk about the software the team will be using than on the story they will be helping to tell.

On one project of which we are aware, the trial and discovery teams developed a highly detailed, rule-based review book. It was more than one hundred and fifty pages long, but devoted fewer than one hundred words in not even ten lines of text to actually telling the client’s story. Don’t do that. Don’t let a narrow focus on the chains of logic obscure the compelling threads of the underlying narrative.

Second, use storytelling with the review team to create a sense of quest. Remember again our metaphor of voyage. The reviewers are, of course, engaged in a real pursuit – weaving a tight, compelling story worthy of being told. Beyond that, quests intimate a feeling of authentic commitment – even a passion – among members of the review team. The power of the story transforms the document review experience. Stories have a unique ability to bind members of the team to a broader purpose, and to each other. As we work together, we are reminded of the human drama that has already unfolded for our client. We remember, too, that our own stories are still unfolding in our work together. On several levels, then, we feel connected. The present has new and important depth.

The organization of the review teams is critical to a sense of quest. The reviewers must identify with the quest to face its hardships and celebrate its victories. The review itself should be seen as a story that has drama, disappointments, dead ends, clues, and ultimately triumph. Banish forever the factory concept of document review as a mass production based on the principles of Taylorism and Fordism.

Third, use of a lawsuit’s stories serves to continually define and redefine the team’s analytical tasks, and to sharpen their focus as the review progresses. Use graphics and models to demonstrate the elements and cohesion of the story as the review is taking place. If the reviewers can’t understand and relate to your story, no judge or jury ever will. Emphasize that the story being told to them is provisional, and that their investigation may, in fact, bring about a retelling of the story. Reiterate key themes as you talk to the members of the team. Challenge them to discern both its strengths and its weaknesses. Provide opportunities for them to share their impressions and their hunches, their discoveries and concerns. This might be as simple and productive as it was on one recent project in which every day or so, one of the law firm’s associates on the case went among the reviewers and asked them, “What are you finding? What do you think?”

It is hard to exaggerate the importance of these interactions. They’re not drive-by questions that are all too easily answered with a yes or no. They are chances for leaders to demonstrate deep listening. They are open-ended invitations to contribute to the group’s learning. They are small streams of one-on-one talk that contribute to what Denning has called the river of conversation that keeps the project moving forward. They are also brief opportunities for members of the team to be acknowledged and affirmed in their work. The goal is to create short but meaningful exercises in team building and flushing out the law suit’s story.

Fourth, share “discoveries” among the team. After all, many of the decisions made by reviewers are close calls, and need to be shared and socialized for consistency and accuracy. In part, this question of sharing is a matter for science.  There are, no doubt, a wide variety of wiki-like technologies that might be brought to bear for purposes of shared learning. But there are several things to be remembered in that regard. First, the technologies seem to be variations on the same theme. That is, they provide ways in which reviewers can articulate their rule-based questions, which are then migrated upwards for consideration by someone on the trial team. The review team is then given access to a database containing all the questions and their answers.  There are many other technologies available for broader, more open learning, but sadly they are rarely employed. It is ironic that in this digital era that has spawned massive reviews, few of the readily available social networking and communications tools have been applied to “humanize” the review process. Then again, the reason is clear: non-story-centric reviews seem to have little use for creativity and collaboration.

The reviewers should be organized into “review teams.” Review teams should ideally be small teams (10-15 reviewers) located in physical proximity. The identification of Linkage Docs, Gravity Docs, and Resonance Doc should be quickly shared and celebrated. Review team members should encourage one another. Review metrics should not exclusively focus on number of documents reviewed per hour. All genuine work and creativity has valleys and plateaus. A review should not be a forced march. The football team regroups in the huddle before each play as it creatively marches down the field. A good, productive review will have its own rhythm. To facilitate this rhythm the successes of one review team should be shared with other teams. Success encourages success and friendly goal oriented competition. Reporting, feedback, and encouragement should be emphasized.

Why have we ignored the lessons of sports competition in our document reviews? Sports motivation coaches are paid millions to inspire athletes and teams. Yet in million dollar reviews, and where even more is at stake in the litigation, we tolerate performance that would be banished elsewhere. What is needed are the genuine “review coaches.”

Fifth, collaboration thrives on human face-to-face contact. The 2009 Text Retrieval Conference (TREC) validated this important point. The TREC team sponsored by the School of Information Sciences of the University of Pittsburgh was provided with shared digital space that allowed them to communicate with each other and to store and organize results. Early on, communication between the searchers consisted mostly of texting, with very little actual, verbal communication. Later on, as tasks became more difficult and the need to collaborate became greater, real talk between the searchers virtually replaced texting, as trust and familiarity developed.

The Pittsburgh team results suggest that while wiki-like technologies are useful in knowledge sharing, trust-based communication such as that involved in document review will gravitate towards ordinary face-to-face communication. It also reminds us that, especially in knowledge sharing exercises, “talk is work” as Stephen Denning has said. This may be another surprise for readers. Absolute silence may not simply mean a focused project. It may be signal a failure to share critical information.

It is precisely in such spontaneous conversations that members of the team draw from the pool of cognitive diversity. A good team will comprise individuals with different strengths, training and backgrounds. When left to themselves high functioning teams learn to take full advantage of their diversity. Good leaders will make sure that team members know their neighbors. Sadly, that rarely happens. On one project related to the life sciences, one reviewer had nearly a decade of law firm experience in that field. But the rest of the team never found out, because the supervisors never thought or wanted to ask. In another project involving the global capital markets, one of the reviewers had two decades of high-level experience trading financial instruments. He decided not to reveal that to anyone. Somehow, the message had gotten across to him that the smart approach to “surviving” document review was to “keep your head down.” It’s a saying you hear a lot on the project floor. What a terrible reflection upon the kind of “supervision“ and “management” to which document reviewers are commonly subjected!

Sixth, use storytelling to generate the connections that will make document review a meaningful experience. The most profound concerns about document review have always revolved around the lack of connection between the purposes of the work and those doing it. Storytelling, on the other hand, is all about connections. Remember what stories are: accounts of causally connected events. So, document review is really an investigation into the nature of those connections. Further, stories are a shared human experience; we all have our own stories. In working together to formulate the story of the case, our own stories become part of the story of the group.

Storytelling establishes common meanings and transmits the values characteristic of high-performing teams. Denning writes that the most striking thing about being part of a great team is the meaningfulness of the experience. “People talk about being part of something larger than themselves, of being connected, or being generative … their experiences as part of truly great teams stand out as singular periods of life lived to the fullest.” We have seen the reviewers’ faces light up, their smiles appear, and genuine excitement erupt when participating in story-centric reviews.

In our view, these are issues of leadership, more than management. The dominant language of document review management reflects the values of traditional command-and-control culture. Such management is about structure, schedules, budgets and the like. This management operates out of hierarchical schemes and derives its presumed effectiveness from the power of authority. Naturally, such things have their place in well-run reviews, as most published literature attests. Metrics matter; things need to be measured and counted. But traditional measures of performance are not always the most revealing.

Consider, for example, the story told in the movie Moneyball about Billy Bean’s discovery that the “five tools” traditionally used to evaluate baseball players missed the mark. Metrics such as batting average and speed on the bases mattered, but they were really pointing to something else that was the most telling factor between ball players on winning and losing teams, that is, on base percentage. What mattered was how often batters got on base by any means. What if the metrics relied upon in review command-and-control structures – such as documents per reviewer per hour – are off the mark?

Seventh, remember that the document review may have to be explained and defended. If challenged as to its reasonableness, the review will have its own story to be told. The McDermott case now is a powerful reminder of what may be at stake. Stories about the labors of well equipped, fully engaged, and highly motivated reviewers are bound to be the most persuasive stories of all.

Conclusion

Good storytelling lies at the very heart of good litigation. Neither the information revolutions of the digital age, nor the dizzying advances of technology have changed that.

The challenge lawyers face is that of adapting the storytelling art to the requirements and capacities of our day. Discovery and review must articulate the client’s most compelling story. It must disable the counter-story told by the other side. Story-centric reviews serve as powerful levers for the other assets – both human and hard – committed to the work of review excellence. This is important work. Justice depends on a compelling story and injustices arise when we forget that.

This entry was posted on Sunday, January 8th, 2012 at 4:58 pm and is filed under Evidence, Lawyers Duties, Review, Search, Technology. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.