Elaborating on issues with Bayes factors

Abstract

Problems with frequentist statistics in general and null hypothesis significance testing in particular are a central issue in Psychology. Recent results related to the lack of reproducibility of statistical findings have urged researchers to change their way of doing science. One proposal that has received considerable attention in the literature is that of replacing $p$‐values with Bayes factors. Bayes factors are often praised for offering a rational means of assessing evidence between two competing hypotheses or models. However, even though there are indeed reasons to prefer Bayes factors over $p$‐values, Bayes factors are affected by a set of issues of their own. In this talk, we will highlight several standing issues with Bayes factors, which include: Sensitivity to the choice of priors, lack of coherence with parameter estimation, lack of proper justification for the so‐called default priors, questionable support towards (point) null hypothesis, and theoretical limitations of marginal likelihoods (on which Bayes factors are based). Our goal is two‐fold: Further clarify what one can (and cannot) expect from Bayes factors and discuss alternative analytic approaches.