Today’s post is from Peter Leasure J.D. (current Ph.D. candidate, University of South Carolina, Department of Criminology and Criminal Justice).
*****
This previous FCPA Professor post and related article showed that FCPA researchers and the entire legal field, especially those publishing legal journals, could greatly benefit from a better understanding of statistics.
This post will once again demonstrate this need. In an article recently published (Annalisa Leibold, The Extraterritorial Application of the FCPA Under International Law, 51 Willamette L. Rev. 225 (2015) and a subsequent post on the FCPA Blog, it was stated that firms with foreign headquarters were paying larger fines than firms with U.S. headquarters in FCPA cases. The author said as follows:
“The data shows a clear discrepancy in the amount of fines paid by foreign versus domestic firms. [Thirty percent] of FCPA cases have been brought against non-U.S. companies. Yet these 30% of companies have paid for 67% of the total FCPA fines. Thus, foreign firms are paying more than five times the FCPA fines paid by domestic firms. It may be the case that foreign firms are generally more corrupt than domestic U.S. firms and thus the DOJ and SEC are simply appropriating higher fines from the more serious violator. It also may be the case that foreign firms do not readily cooperate with U.S. authorities. If this is not the case and foreign firms are not more corrupt and do cooperate readily, then it must be true that the SEC and the DOJ are unfairly targeting foreign firms with higher FCPA fines.”
To arrive at this conclusion, the author totaled and then compared average fines for both U.S. and foreign headquartered firms. This means that the author essentially utilized two variables. The first variable looked at the number of cases brought against U.S. and foreign headquartered firms. The second variable looked at a firm’s total fine. There are several issues with using such an approach. I will note a few of the issues below.
- Other Variables
While the author points to other variables such as cooperation/voluntary disclosure and seriousness in the above paragraph, they are not accounted for in the actual analysis. While noting the other possibly influential variables is certainly a positive step, it falls short of adequate rigor. If we can identify and operationalize possible influential variables, those variables should be included in the analysis.
- Outliers
The author also makes no mention of the possible influential effect of an outlier. An outlier is a single observation that can significantly influence an entire outcome. Here, an outlier would be a case possessing unusually high or unusually low total fines. It could be the case that only one or two foreign firms’ total fines significantly influence the overall average. And this is likely given the Siemens case by itself. Using a median instead of average total fine for U.S. and foreign firms would have been better as a median value is more resistant to outliers. However, even using median values would fall short because of the failure to account for the other variables mentioned above.
- Reporting
There is also a bit of a problem in how the author reported results. This is so because the author began discussing results by stating that “[t]he data shows a clear discrepancy in the amount of fines paid by foreign versus domestic firms.” This is problematic because readers not trained in statistics may simply take that statement and run with it. An analogous point is the ranking of U.S. cities as “Most Dangerous” that we occasionally see pop up on various websites. These rankings are incredibly irresponsible because they fail to account for vital facts. For example, perhaps a certain city, who does not view the police as “on their side” (a case in some urban neighborhoods), has a large amount of crime not reported to any agency. Perhaps another city’s residents report any little thing that happens because they welcome police contact. Further, several local law enforcement agencies do not even report their crime rates to the data sets used by the city rankings. The carelessness to account for these possibilities by the city rankings entities means that their conclusions are junk. But a good deal of readers, understandably so, only see the headlines and inquire no further. The more responsible approach is to avoid making such strong statements until a better statistical method is utilized and limitations are clearly noted.
- Method
Simple analyses utilizing such descriptive statistics as mean and median should rarely be used to draw strong conclusions. This is so because such descriptive analyses cannot account or control for other variables. Regression, a common method of controlling for multiple variables, would be a good approach. Fortunately, I collected my own census data of FCPA cases for another paper and that paper also examined whether foreign firms do indeed pay significantly higher fines than U.S. firms.
Substantially improving the methods in the previous study, my result shows that U.S. firms received 13.9% higher total fines than foreign firms. However, if we were to take a conservative approach and include 95% confidence intervals in the findings, we would not be able to draw any certain conclusions from the data as to whether foreign or U.S. firms pay higher fines.[1]
So even with the improved methods, what do we know about foreign or U.S. firms paying higher fines? The responsible answer is very little if anything at all. That answer can certainly change over time. As of now, we just do not have the data to give a confident answer.
As a final note, it is also important to understand that studies using less rigorous or irresponsible statistical analysis can actually create the very problems they are seeking to bring attention to and ultimately cure. This happens because analyses with little rigor or flawed approaches create findings with strong conclusions that practitioners and academics then adopt.
In the case of the current and previous studies under critique, readers are led to believe that there is no benefit to voluntary disclosure and that foreign firms are unfairly targeted by paying higher penalties. This leads to people seeing those enforcing the FCPA as unfair. And there is a large amount of empirical support for a theory stating that people are less likely to comply with the law when they believe the criminal justice system and its agents are illegitimate or unfair. [2]
This surely does not mean to dismiss any work calling attention to a government shortcoming. If rigorous research uncovers an unfair practice, it should be welcomed. But no longer can we blindly accept a study’s conclusions. This post is another reminder of why.
Though this post once again demonstrates significant errors in statistical FCPA research, such research cannot be abandoned. On the contrary, such pieces further confirm that a better understanding of statistics is indeed vital for FCPA research and the legal field to progress.
*****
[1] Crudely speaking, confidence intervals produce a range for estimates rather than a single point estimate because most estimates have a window of error. Confidence intervals are therefore more conservative estimates. Theoretically, using significance values and confidence intervals on population data is not necessary (See Bushway, S. D., Sweeten, G., & Wilson, D. B. (2006). Size matters: Standard errors in the application of null hypothesis significance testing in criminology and criminal justice. Journal of Experimental Criminology, 2(1), 1-22). However, because some cases were excluded here when a bribe amount or the presence of voluntary disclosure could not be detected, confidence intervals may be the more conservative approach.
[2] Tyler, T. (1990). Why people obey the law. New Haven, CT: Yale University Press and Tyler, T. (1997). The psychology of legitimacy: A relational perspective on voluntary deference to authorities. Personality and Social Psychology Review, 1, 323-45.
