What criteria should be held by good qualitative research?

Critical thinking
Article: Borsboom, Rhemtulla, Cramer, van der Maas, Scheffer and Dolan (2016)
Kinds versus continua: a review of psychometric approaches to uncover the structure of psychiatric constructs

The present paper reviews psychometric modelling approaches that can be used to investigate the question whether psychopathology constructs are discrete or continuous dimensions through application of statistical models.

Introduction

The question of whether mental disorders should be thought of as discrete categories or as continua represents an important issue in clinical psychology and psychiatry.

• The DSM-V typically adheres to a categorical model, in which discrete diagnoses are based on patterns of symptoms.

But, such categorizations often involve apparently arbitrary conventions.

Measurement theoretical definitions of kinds and continua

All measurement starts with categorization, the formation of equivalence classes.
Equivalence classes: sets of individuals who are exchangeable with respect to the attribute of interest.
We may not succeed in finding an observational procedure that in fact yields the desired equivalence classes.

• We may find that individuals who have been assigned the same label are not indistinguishable with respect to the attribute of interest.
  Because there are now three classes rather than two, next to the relation between individuals within cases (equivalence), we may also represent systematic relations between members of different cases.
• One may do so by invoking the concept of order.
  But, we may find that within these classes, there are non-trivial differences between individuals that we wish to represent.

If we break down the classes further, we may represent them with a scale that starts to approach continuity.

The continuity hypothesis formally implies that:

• in between any two positions lies a third that can be empirically instantiated
• there are no gaps in the continuum.

In psychological terms, categorical representations line up naturally with an interpretation of disorders as discrete disease entities, while continuum hypotheses are most naturally consistent with the idea that a construct varies continuously in a population.

• in a continuous interpretation, the distinction between individuals depends on the imposition of a cut-off score that does not reflect a gap that is inherent in the attribute itself.

Kinds and continua as psychometric entities

In psychology, we have no way to decide conclusively whether two individuals are ‘equally depressed’.
This means we cannot form the equivalence classes necessary for measurement theory to operate.
The standard approach to dealing with this situation in psychology is to presume that, even though equivalence classes for theoretical entities like depression

Critical thinking
Chapter 4 of Understanding Psychology as a science by Dienes
Bayes and the probability of hypotheses

Objective probability: a long-run relative frequency.
Classic (Neyman-Pearson) statistics can tell you the long-run relative frequency of different types of errors.

• Classic statistics do not tell you the probability of any hypothesis being true.

An alternative approach to statistics is to start with what Bayesians say are people’s natural intuitions.
People want statistics to tell them the probability of their hypothesis being right.
Subjective probability: the subjective degree of conviction in a hypothesis.

Subjective probability

Subjective or personal probability: the degree of conviction we have in a hypothesis.
Probabilities are in the mind, not in the world.

The initial problem to address in making use of subjective probabilities is how to assign a precise number to how probable you think a proposition is.
The initial personal probability that you assign to any theory is up to you.
Sometimes it is useful to express your personal convictions in terms of odds rather than probabilities.

Odds(theory is true) = probability(theory is true)/probability(theory is false)
Probability = odds/(odds +1)

These numbers we get from deep inside us must obey the axioms of probability.
This is the stipulation that ensures the way we change our personal probability in a theory is coherent and rational.

• People’s intuitions about how to change probabilities in the light of new information are notoriously bad.

This is where the statistician comes in and forces us to be disciplined.

There are only a few axioms, each more-or-less self-evidently reasonable.

• Two aximons effectively set limits on what values probabilities can take.
  All probabilities will lie between 0 and 1
• P(A or B) = P(A) + P(B), if A and B are mutually exclusive.
• P(A and B) = P(A) x P(B|A)
  • P(B|A) is the probability of B given A.

Bayes’ theorem

H is the hypothesis
D is the data

P(H and D) = P(D) x P(H|D)
P(H and D) = P(H) x P(D|H)

so

P(D) x P(H|D) = P(H) x P(D|H)

Moving P(D) to the other side

P(H|D) = P(D|H) x P(H) / P(D)

This last one is Bayes theorem.
It tells you how to go from one conditional probability to its inverse.
We can simplify this equation if we are interested in comparing the probability of different hypotheses given the same data D.
Then P(D) is just a constant for all these comparisons.

P(H|D) is proportional to P(D|H) x P(H)

P(H) is called the prior.
It is how probable you

Critical thinking
Article: Borsboom, D. and Cramer, A, O, J. (2013)
Network Analysis: An Integrative Approach to the Structure of Psychopathology
doi: 10.1146/annurev-clinpsy-050212-185608

Introduction

The current dominant paradigm of the disease model of psychopathology is problematic.
Current handling of psychopathology data is predicated on traditional psychometric approaches that are the technical mirror of of this paradigm.
In these approaches, observables (clinical symptoms) are explained by means of a small set of latent variables, just like symptoms are explained by disorders.

• From this psychometric perspective, symptoms are regarded as measurements of a disorder, and in accordance, symptoms are aggregated in a total score that reflects a person’s stance on that latent variable.
• The dominant paradigm is not merely a matter of theoretical choice, but also of methodological and pragmatic necessity.

In this review, we argue that complex network approaches, which are currently being developed at the crossroads of various scientific fields, have the potential to provide a way of thinking about disorders that does justice to their complex organisation.

• In such approaches, disorders are conceptualized as systems of causally connected symptoms rather than as effects of a latent disorder.
• Using network analysis techniques, such systems can be represented, analysed, and studied in their full complexity.
• In addition, network modeling has the philosophical advantage of dropping the unrealistic idea that symptoms of a single disorder share a single causal background, while it simultaneously avoids the realistic consequence that disorders are merely labels for an arbitrary set of symptoms.
  • It provides a middle ground in which disorders exists as systems, rather than as entities

Symptoms and disorders in psychopathology

We know for certain that people suffer from symptoms and that these symptoms cluster in a non-arbitrary way.
For most psychopathological conditions, the symptoms are only empirically identifiable causes of distress.

• Mental disorders are themselves not empirically identifiable in that they cannot be diagnosed independently of their symptoms.
  • It is impossible to identify any of the common mental disorders as conditions that exists independently of their symptoms.

In order for a disease model to hold, it should be possible to conceptually separate conditions from symptoms.

• It must be possible (or at least imaginable) that a person should have a condition/disease without the associated symptoms.

This isn’t possible for mental disorders.
As an important corollary, this means that disorders cannot be causes of these symptoms.
This strongly suggests that the treatment of disorders as

Critical thinking
Article: Gigerenzer, G. & Marewski, J, N. (2015)
Surrogate Science: The Idol of a Universal Method for Scientific Inference
doi: 10.1177/0149206314547522

Introduction

Scientific inference should not be made mechanically.
Good science requires both statistical tools and informed judgment about what model to construct, what hypotheses to test, and what tools to use.

This article is about the idol of a universal method of statistical inference.

In this article, we make three points:

• There is no universal method of scientific inference, but, rather a toolbox of useful statistical methods. In the absence of a universal method, its followers worship surrogate idols, such as significant p values.
  The inevitable gap between the ideal and its surrogate is bridged with delusions.
  These mistaken beliefs do much harm. Among others, by promoting irreproducible results.
• If the proclaimed ‘Bayesian revolution’ were to take place, the danger is that the idol of a universal method might survive in a new guise, proclaiming that all uncertainty can be reduced to subjective probabilities.
• Statistical methods are not simply applied to a discipline. They change the discipline itself, and vice versa.

Dreaming up a universal method of inference

The null ritual

The most prominent creation of a seemingly universal inference method is the null ritual:

• Set up a null hypothesis of ‘no mean inference’ or ‘zero correlation’. Do not specify the predictions or your own research hypothesis.
• Use 5% as a convention for rejecting the null. If significant, accept you research hypothesis. Report the result as p<.05, p<.01, p<.001, whichever comes next to the obtained p value.
• Always perform this procedure.

Level of significance has three different meanings:

• A mere convention
• The alpha level
• The exact level of significance

Three meanings of significance

The alpha level: the long-term relative frequency of mistakenly rejecting hypothesis H₀ if it is true, also known as Type I error rate.
The beta level: the long-term frequency of mistakenly rejecting H₁ if it is true.

Two statistical hypothesis need to be specified in order to be able to determine both alpha and beta.
Neyman and Pearson rejected a mere convention in favour of an alpha level that required a rational scheme.

• Set up two statistical hypotheses, H₁, H₂, and decide on alpha, beta and the sample size before the experiment, based on subjective cost-benefit considerations.
• If the data fall into the rejection region of H₁, accept H₂, otherwise accept H₁

This magazine contains all the summaries you need for the course WSRt at the second year of psychology at the Uva.