Knowing the signs: a direct and generalizable motivation of two-sided tests

Many well-known problems with two-sided p-values are due to their use in hypothesis tests, with 'reject-accept' conclusions about point null hypotheses. We present an alternative motivation for p-value-based tests, viewing them as assessments of only the sign of an underlying parameter, where we can conclude that the parameter is positive or negative, or simply say nothing either way. Our approach is decision theoretic, but-unusually-we consider the whole set of possible utility functions available. Doing this we show how, in a specific sense, close analogues of familiar one- and two-sided tests are always the optimal decision. We argue that this simplicity could aid non-experts' understanding and use of tests-and help them to think critically about whether or not tests are appropriate tools for answering their questions of interest. Several extensions are also considered, showing that the simple idea of determining the signs of parameters yields a rich framework for inference.

See More