Return to the TAR-Pit

Take note all, there is a new predictive coding case by Judge Andrew Peck. The good Judge waded into the TAR-pit of transparency, which in my opinion has caused much unnecessary problems with judges and parties who believe “transparency” is required when predictive coding is used, mandating the disclosure of seed sets. I do not think the Federal Rules of Civil Procedure place such a burden on producing parties by attacking the work-product doctrine or compelling the production of irrelevant information.

Judge Peck summarized that “where the parties do not agree to transparency, the decisions are split and the debate in the discovery literature is robust.” Rio Tinto Plc v. Vale S.A., 2015 U.S. Dist. LEXIS 24996, 8 (S.D.N.Y. Mar. 2, 2015).

Judge Peck’s new opinion did not rule on the issue of seed transparency, because the parties had agreed to an ESI protocol that “disclosed all non-privilege documents in the control sets.” Rio Tinto Plc., at *10. However, Judge Peck provided his thoughts on what I would call a bias against technology-assisted review:

One point must be stressed -it is inappropriate to hold TAR to a higher standard than keywords or manual review. Doing so discourages parties from using TAR for fear of spending more in motion practice than the savings from using TAR for review.

Rio Tinto Plc, at *10, emphasis added.

That statement is extremely important. There is no valid reason to treat “predictive coding,” or any other form of analytics such as conceptual searching, email threading, or clustering, to higher discovery standards because “technology” is used.

The issue is whether or not the production is adequate. A producing party should not have to disclose attorney work product in analyzing their case or proving the technology functions. Courts do not hold competency hearings on whether an attorney properly knows how to conduct online legal research to ensure attorneys are maintaining their ethical duty of candor to the court. The same logic applies to technology-assisted review and productions.

Judge Peck did state while he generally believed in cooperation, “requesting parties can insure that training and review was done appropriately by other means, such as statistical estimation of recall at the conclusion of the review as well as by whether there are gaps in the production, and quality control review of samples from the documents categorized as now responsive.” Rio Tinto Plc., at *9-10, referencing Grossman & Cormack, Comments, supra, 7 Fed. Cts. L. Rev. at 301-12.

The focus of discovery should be whether or not a production is adequate. Are there production gaps? Is the parent-child relationship maintained between email messages and attachments? Is the producing party conducting quality assurance testing? Is the producing party documenting their efforts?

Judge Peck’s latest opinion will not be the final word on the issue of “transparency.” However, Judge Peck is a well-respected jurist who understands technology-assisted review. His statement that “it is inappropriate to hold TAR to a higher standard than keywords or manual review” is a very welcome one.

Advertisement

What Did We Learn About eDiscovery in 2014?

As 2014 draws to a close, it is time to reflect on the cases from this year in eDiscovery. One of the biggest trends I took away from caselaw in 2014, is that more Judges have a greater understanding of eDiscovery, resulting in practical opinions.

Here are the practice areas I found to be the most interesting in 2014, which can be heard in full on my 2014 eDiscovery Year in Review on iTunes or Buzzsprout (Presented by Paragon):

Application of Proportionality Analysis

Judges Questioning Why The Court Was Asked Permission to Use Predictive Coding

We still have Form of Production issues eight years after the 2006 eDiscovery Amendments to the Federal Rules of Civil Procedure

The Importance of Documenting Services for Taxation of Costs

What will 2015 hold for us in the world of electronic discovery? I think we will see proportionality analysis focus on the value of the information sought in relation to the case and not solely just the cost of the discovery. Parties will have to explain how the information is useful, such as how it relates to a claim, opposed to merely saying, “It is expensive.” This will require counsel to focus on the merits of the case and how the requested discovery will help advance the litigation.

I personally hope litigants stop asking Judges for permission to use predictive coding. No one asks, “Can I de-dup the data? Is it ok to use clustering? May I please use conceptual search in addition to keywords?”

The issue with all productions is whether or not the production is adequate. In my view, parties going to war over predictive coding as a means to review electronically stored information is asking the Court to issue an advisory opinion. The time to fight is when the there actually is a dispute because a production is lacking, instead of engaging in arguments of how much a human being can read in an hour compared to a computer algorism.

To learn more on the issues from the past year, please check out my 2014 eDiscovery Year in Review audio podcast on iTunes or Buzzsprout.

I want to thank Paragon for sponsoring the 2014 eDiscovery Year in Review. Please check out their website and recent blog post on the Convergence of eDiscovery and Information Security to learn more about their services.

Even A Judged Questioned Why Ask for Permission to Use Predictive Coding

I do not normally want to high five Federal judges, but Judge Ronald Buch, a Tax Judge in Texas, sure deserved one after his Dynamo Holdings opinion.

The discovery dispute can be summed up as a battle over backup tapes that had confidential information. The Requesting Party wanted the tapes; the Producing Party wanted to use predictive coding to produce what was relevant, because the cost for reviewing the material for privilege and relevancy would cost $450,000 with manual review. Dynamo Holdings v. Comm’r, 2014 U.S. Tax Ct. LEXIS 40 (Docket Nos. 2685-11, 8393-12. Filed September 17, 2014.)

The Requesting Party wanted the backup tapes to analyze metadata on when ESI was created. Moreover, the Requesting Party called “Predictive Coding” an “unproven technology.” The Requesting Party attempted to address the Producing Party’s cost concern with a clawback agreement. Dynamo Holdings, at *3.

After an evidentiary hearing with experts on the use of predictive coding, the Court granted the Producing Party’s motion to use predictive coding. Judge Buch had a “dynamo” quote on the entire issue of asking to use predictive coding:

 “And although it is a proper role of the Court to supervise the discovery process and intervene when it is abused by the parties, the Court is not normally in the business of dictating to parties the process that they should use when responding to discovery. If our focus were on paper discovery, we would not (for example) be dictating to a party the manner in which it should review documents for responsiveness or privilege, such as whether that review should be done by a paralegal, a junior attorney, or a senior attorney. Yet that is, in essence, what the parties are asking the Court to consider–whether document review should be done by humans or with the assistance of computers. Respondent fears an incomplete response to his discovery. If respondent believes that the ultimate discovery response is incomplete and can support that belief, he can file another motion to compel at that time. Nonetheless, because we have not previously addressed the issue of computer-assisted review tools, we will address it here.

Dynamo Holdings, at *10-11.

It is so refreshing to see a Judge address the issue of requesting to use a specific technology. No one does a motion to compel asking for permission on what lawyers should do document review. Moreover, no moving party asks permission to use visual analytics, de-duplication, or any of the other outstanding technology available to conduct eDiscovery.

The opinion ends with that if the Requesting Party believed the discovery response was incomplete, then a motion to compel could be filed, which is exactly the way the process should work. The issue should not be “can we use this technology,” whether the production is adequate or not, which requires evidence of a production gaps or other evidence that not all responsive information was produced.

Well done Judge Buch.

 

Does Proportionality Disappear If a Lawyer Says “Predictive Coding” Three Times?

The In re Bridgepoint Education case is not one about the merits of predictive coding, but one of proportionality over expanding the scope of discovery by nine months. In re Bridgepoint Educ., 2014 U.S. Dist. LEXIS 108505, 10-11 (S.D. Cal. Aug. 6, 2014). While the cost of document review and the use of predictive coding have a starring role in the opinion, let’s not forget the second discovery dispute in the case ultimately is about proportionality.

The Defendants claimed that expanding the scope of discovery by nine months would increase their review costs by 26% or $390,000 (based on past review efforts in the case). In re Bridgepoint Educ., at *6-7.

The Plaintiffs countered that the review costs would more likely be $11,279, because of the predictive coding system the Defendants would use instead of manual review. In re Bridgepoint Educ., at *7.

The Defendants countered that “predictive coding” did not make “manual review” for relevance elective, because the predictive coding software assigned a percentage estimate to each record on the record’s probability of being relevant. Id. As such, attorney review is still required for relevance and privilege review. Id.

The Court denied expanding the scope of discovery by nine months based on the “proportionality” rule of Federal Rule of Civil Procedure Rule 26(b)(2)(C). The Rule states a Court can limit discovery if the “burden or expense of the proposed discovery outweighs the likely benefit.” In re Bridgepoint Educ., at *9-10.

The Court found expanding the scope to be unduly burdensome. Moreover, while there might have been relevant information in the expanded timeframe, the Court agreed with the Defendants that relevant information would be in the originally agreed timeframe. In re Bridgepoint Educ., at *10-11.

Predictive Coding was also at the center of the fourth discovery dispute. The Plaintiffs argued discovery produced from three Individual Defendants should be added to the Defendants’ predictive coding software. In re Bridgepoint Educ., at *12. According to the Plaintiffs, the Defendants “unilaterally-selected search terms” to identify the original production. Id.

The Defendants argued their review process for the original production was reasonable. Moreover, adding the original production to the predictive coding process could “negatively impact the reliability of the predictive coding process.” In re Bridgepoint Educ., at *12-13. However, the Defendants were willing to run additional searches on the Individual Defendants’ production. In re Bridgepoint Educ., at *13.

The Court noted that the Defendants’ linear search methodology for the three Individual Defendants had been approved by the Court. As such, the Court ordered the parties to meet and confer on additional search terms on the original production for the Individual Defendants. Id.

Bow Tie Thoughts

In re Bridgepoint Education is an interesting spin on predictive coding cases, because effectively the REQUESTING party is arguing for the producing party to use predictive coding to reduce proportionality issues.

First things first: Saying, “predictive coding” has no magical properties. Nor will review costs decline by getting the opposing party to say Rumpelstiltskin. Even if an attorney is somehow tricked into saying Mister Mxyzptlk’s name backwards in a hearing, a Court will always be concerned about proportionality before expanding the scope of discovery.

Proportionality will always be concerned with cost of review, but discovery review does not exist independent of the case. The scope of discovery should not be expanded because the cost of review can be reduced leveraging advanced search capabilities alone. The issue is whether there are relevant records in the expanded universe and if the “burden or expense of the proposed discovery outweighs the likely benefit.”

Finally, just because one search methodology was used to identify records over another does not devalue the responsiveness of the production. If a requesting party is concerned with the adequacy of a production, challenge it accordingly by showing production gaps or other evidence to demonstrate the production is inadequate.

How Not to Be Progressive: Court Rejects Predictive Coding Not Agreed to By Parties

Fighting over discovery search methodology makes me think of President Richard Nixon’s resignation speech: “Always remember, there are those who hate you. And the only way to keep them from winning is to hate them right back. And then you destroy yourself.”

Attacking a party who used predictive coding to reduce discovery review time to save money and time, only to result in a larger collection of ESI produced upon the requesting party, means the requesting party now has more to review. This is the end result of Progressive Cas. Ins. Co. v. Delaney.   

I have never been a fan of agreeing to the use of predictive coding in ESI Protocols. I do not believe such agreements are required at all by the Federal Rules of Civil Procedure. The issue is whether or not a production is adequate. That requires the requesting party offering evidence that the production is somehow delinquent with facts. That discussion does not happen without first having a production.

Parties do not have a veto power over which review application is used by a reviewing party and what features they may or may not use.  The requesting party should not attack a review methodology, unless there is a production to challenge. The entire discussion is premature and the issue is not ripe for the Court.

In the case at bar, the original dataset was narrowed by search terms to 565,000 “hits” from the original 1.8 million dataset. This search term methodology had been agreed to in an ESI Protocol by the parties. After one month of document review, the producing party realized that it could take 6 to 8 months to manually review the narrowed dataset for responsiveness or privilege. The party unilaterally decided to use predictive coding instead. Moreover, after telling the other side about their change in technology, motion practice followed. Progressive Cas. Ins. Co. v. Delaney, 2014 U.S. Dist. LEXIS 69166.

The Requesting Party wanted the Producing Party to 1) produce the 565,000 culled from the 1.8 million data set using the parties’ agreed-upon search terms subject to a clawback for privileged documents, or 2) the Producing Party apply the predictive coding methodology to the entire 1.8 million record dataset. Progressive, at *12 and *15.

The Producing Party did not want to do either approach, specifically concerned that searching the entire dataset would result in a larger privilege review.

The Court noted what has been judge-made law on using technology assisted review: Courts require the producing party to provide the requesting party with “full disclosure about the technology used, the process, and the methodology, including the documents used to “train” the computer.” Progressive, at *27-28, citing Da Silva Moore 2012 U.S. Dist. LEXIS 23550 (S.D.N.Y. Feb. 24, 2012).

The Court ordered the producing party to follow the original agreed to protocol and produce the “hit” documents to the Requesting Party within fourteen days without further review. Progressive, at *30.

The Court stated that following the Requesting Party’s protocol, the cost of review would be shifted to them. Progressive, at *31. Moreover, the Requesting Party believed they had the manpower to complete the review within one month. Id. 

The Requesting Party could apply privilege filters before production and produce a privilege log. Id. 

Bow Tie Thoughts

I do not encourage clients to have ESI Protocols that limit their ability to review discovery efficiently. I also strongly argue against the idea you need approval from the opposing party on what search methodology you can use to identify responsive discovery. It is the producing party’s right to use keywords, concept search, visual analytics, email threading, clustering, find similar, or any other form of technology-assisted review. That includes predictive coding. The requesting party does not get a veto power over what technology the producing party can use. The requesting party has “the burden of proving that a discovery response is inadequate.” Abt v. Jewell, 2014 U.S. Dist. LEXIS 50766, 12-14 (D.D.C. Apr. 11, 2014), citing Barnes v. D.C., 289 F.R.D. 1, 6 (D.D.C. 2012) and Equal Rights Ctr. v. Post Props., Inc., 246 F.R.D. 29, 32 (D.D.C. 2007).

You cannot prove a discovery response is inadequate if you do not have a discovery response. The entire idea of attacking the use of predictive coding is premature if there is no production to be reviewed.

The ugly wrinkle in this case is the search and production methodology mandated in the ESI Protocol agreed to by the parties. I strongly encourage parties to not agree to actual technological steps to be used, because it limits the ability to conduct discovery in a cost effective manner. These agreements are often enacted without the advice of an eDiscovery Consultant who knows how the actual technology works.

What parties should discuss and codify in an ESI Protocol is the subject matter of the lawsuit. Who are the key players? What are the relevant date ranges? What are the terms of art used by the parties? What is the data actually making up the case? Those, and many others, are the topics parties need to agree to, not whether you can use visual analytics to identify date ranges or data clustering to determine what was relevant.

The use of predictive coding has been twisted because lawyers think they need permission to use it. They do not. The burden is on the opposing side to prove a production was inadequate, not the fact the producing party used one form of technology-assisted review over another.