P-values: misunderstood and misused

Journal article

P-values are widely used in both the social and natural sciences to quantify the statistical significance of observed results. The recent surge of big data research has made p-value an even more popular tool to test the significance of a study. However, substantial literature has been produced critiquing how p-values are used and understood. In this paper we review this recent critical literature, much of which is routed in the life sciences, and consider its implications for social scientific research. We provide a coherent picture of what the main criticisms are, and draw together and disambiguate common themes. In particular, we explain how the False Discovery Rate is calculated, and how this differs from a p-value. We also make explicit the Bayesian nature of many recent criticisms, a dimension that is often underplayed or ignored. We also identify practical steps to help remediate some of the concerns identified, and argue that p-values need to be contextualised within (i) the specific study, and (ii) the broader field of inquiry.

Authors: Vidgen, B; Yasseri, T

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Frontiers Media
• Journal: Frontiers in Physics
• Volume: 4
• Article number: 6
• Publication date: 2016-03-04
• Acceptance date: 2016-02-19
• DOI: 10.3389/fphy.2016.00006
• EISSN: 2296-424X
• Language: English
• Keywords: big data; p-hacking; p-value; statistics; significance; Bayes; prevalence