Research Methods & Statistics – Bayesian statistics summary (UNIVERSITY OF AMSTERDAM)

Probability refers to the proportion of occurrence when a particular experiment is repeated infinitely often under different circumstances. It is a long-term relative frequency, does not apply to unique events and is dependent on the reference category.

Subjective probability refers to the subjective degree of conviction in a hypothesis. Objective probability refers to the long-term relative frequency and is the same probability used in classical statistics.

The p-value is the probability of finding a test statistic at least as extreme as the one observed, given that the null hypothesis is true. An X% confidence interval for a parameter is an interval that in repeated use has an X% chance to capture the true value of the parameter. The p-values are only concerned about the null hypothesis, although it is not possible to make statements about the probability of a hypothesis in classical statistics.

If the null hypothesis is true, then the p-values drift randomly. Therefore, it is possible that the p-value is significant by chance. This is why stopping rules are imperative in classical statistics. In Bayesian statistics, the Bayes factor does not drift randomly but drifts towards the correct decision.

In classical statistics, the stopping rules (1), the timing of explanations (posthoc test or not) (2) and multiple tests influence the conclusion. This is not the case in Bayesian statistics.

Classical statistics does not allow for probabilities to be assigned to hypotheses or parameters, whereas Bayesian statistics does allow this.

Bayesian statistics is a method of learning from prediction errors. It assumes that probability does not exist but only uncertainty, which has to be quantified in a principled manner. Therefore, in Bayesian statistics, probability can be assigned to a single hypothesis.

The data drive an update from prior knowledge to posterior knowledge. This method investigates whereas classical statistics investigates  .

The Bayes factor can also be seen as the predictive updating factor for the posterior belief. It is the ratio of likelihoods. The likelihood refers to the probability of obtaining the data given the hypothesis. Bayesian statistics use Bayes rule:

The prior distribution determines the posterior distribution, therefore, a high predictive updating factor in favour of the alternative hypothesis does not necessarily mean that the alternative hypothesis is better. It only predicts the dataset X times better than the null hypothesis in this case.

The posterior belief and the Bayes factor are the same if the prior belief is that the distribution is 50/50. Otherwise, the posterior belief and the Bayes factor are not the same.

The Bayes factor can be used as evidence, although these categories are arbitrary. Statistical evidence refers to a change in conviction concerning a hypothesis brought about by the data. It is easier to detect the presence of something than the absence of something.

A Bayes factor greater than 1, provides evidence for the alternative hypothesis over the null hypothesis. A Bayes factor small than 1, provides evidence for the null hypothesis over the alternative hypothesis. A Bayes factor of approximately 1, indicates that the experiment was not sensitive enough to differentiate between the two hypotheses. This is how power is incorporated into Bayesian statistics.

The likelihood principle states that all the information relevant to inference contained in data is provided by the likelihood. A hypothesis having the highest likelihood does not mean that the hypothesis has the highest probability of being true, it means that the data support the hypothesis the most.

In a distribution, the p-value is the area under the curve, whereas the likelihood is the height of the distribution at a certain point.

There are several advantages to the Bayes factor:

1. The Bayes factor provides a continuous degree of evidence without requiring an all-or-none decision (p-value).
2. The Bayes factor allows evidence to be monitored during data collection.
3. The Bayes factor differentiates between support for the null hypothesis (evidence for absence of an effect) and non-informative data (absence of evidence).

There are several objections to Bayesian statistics:

1. Bayesian statistics forces people to specify predictions in detail.
2. Bayesian statistics do not control for Type I and Type II errors.
3. Bayesian statistics use subjective (and arguably arbitrary) priors.
4. Bayesian statistics allows extremely odd world-views in priors.
5. Bayesian statistics do not take into account that a new theory influences confidence in the old theory, without this necessarily being based on the data.
6. Bayesian statistics assume that classical logic is the only form of logic.
7. Bayesian statistics do not allow old evidence to support a new theory as the prior is already set.
8. Bayesian statistics assume logical omniscience – the idea that logical truths have a probability of 1 and logical contradictions have a probability of 0, although this does not occur in humans (humans make flawed logical decisions).
9. Frequency-based prior probabilities do not exist.
10. Researchers’ introspection does not confirm the calculation of probabilities.
11. The set of hypotheses needed for the prior probability distribution is not known.

Bayesian statistics might be less sensitive to questionable research practices, as probability is tied to the procedure for frequentist statistics and not for Bayesian statistics. Probability is used in psychological research through random sampling and random assignment. The sampling distribution of the statistic is known because of random sampling.

Bayesian statistics allows for more precise and more specific predictions which make for more stringent tests, although extraordinary beliefs require extraordinary evidence as strong priors will not change without strong likelihoods.

There are several advantages of frequentist statistics:

1. The p-value is objective as the probability of the data given the null hypothesis (P(data | hypothesis) is an objective probability.
2.  Frequentist statistics also allows to control for Type I and Type II error rates.
3. Frequentist statistics are very practical as almost all research designs can use null hypothesis testing.
4. The p-value always has the same interpretation.

The probability of a single event (e.g. P(hypothesis)) does not exist according to frequentists. In Bayesian statistics, the same background knowledge should lead to the same conclusion, therefore, the prior is subjective but not arbitrary.

There are several practical advantages of Bayesian statistics:

1. Bayesian statistics allows for learning from prediction errors.
2. Bayesian statistics allows quantifying evidence in favour of a hypothesis.
3. Bayesian statistics allows adjusting knowledge while conducting research.
4. Bayesian statistics allows to obtain answers to meaningful questions (e.g. P(hypothesis|data)).

In Bayesian statistics, there are three types of distributions:

1. Uniform distribution
   This is a distribution where every value is equally likely.
2. Normal distribution
   This is a distribution where one value is most likely with the values on both sides of this value being equally likely as the distribution is symmetrical.
3. Half-normal distribution
   This is a distribution which is centred on zero with only one tail (e.g. positive or negative).

Bayesian statistics are not intuitive statistics, as people tend to rely on heuristics and fallacies (e.g. base rate fallacy).