Critical thinking
Chapter 1
Introducing Arguments

To attempt to persuade by giving good reasons is to give an argument.

Beginning to think critically: recognizing arguments

Not all attempts to persuade (using language) are attempts to persuade by argument.

• some are attempts to persuade by means of rhetorical devices

Rhetoric: any verbal or written attempt to persuade someone to believe, desire or do something that does not attempt to give good reasons for the belief, desire or action, but attempts to motivate that belief, desire or action solely through the power of the words used.

An attempt to persuade by argument is an attempt to provide you with reasons for believing a claim, desiring something or doing something.
Arguments appeal to your critical faculties, your reason.

Rhetoric tends to rely on the persuasive power of certain words and verbal techniques to influence your beliefs, desires and actions by appeal to your desires, fears and other feelings.

• threats and bribes are arguments (not rhetorics), for they give a reason to do something

Rhetorical techniques can be manipulative and coercive. Their use should generally be avoided by those who aspire to think critically and to persuade by reason.
When analysing attempts to persuade, we have to perform three tasks:

• Identify the issue being discussed, and determine whether or not the writer or speaker is attempting to persuade by means of argument. Is an argument being presented?
• Reconstructing the argument so as to express it clearly, and so as to demonstrate clearly the steps and form of the argument’s reasoning
• Evaluating the argument, asking what’s good about it and what’s bad about it.

When we put forward an argument we are either advancing an opinion or recommending an action.
In either case we give a number of claims intended to support the claim or the recommendation.
These two types of argument can be collapsed into one.

All arguments can be understood as attempting to provide reasons for thinking that some claim is true (it states how things really are).
To say that a claim is true is to say that what is claimed is how things actually are.

A

Wst-r
Chapter 3
Logic: deductive validity

Argument reconstruction: the representation of arguments in standard form, so as to give us a clear and comprehensive view of them.

Argument assessment: the determination of whether or not arguments provide good reasons for accepting their conclusions.

The principle of charity

An argument is a system of propositions.
Propositions: a set of premises advanced in support of a conclusion.
People succeed in expressing the propositions they have in mind in varying degrees of clarity. An argument may depend upon premises that the arguer does not state at all, but which he or she is implicitly assuming.

Since the purpose of argument-reconstruction is to determine exactly what argument has been given, part of the task is to clarify what the arguer actually said, and to supplement what the arguer actually said (to make explicit what was merely implicit in the arguer’s statements).

• The sentences we use in a reconstruction of the argument need not to be the very same sentences as used by the arguer in giving their argument. We may employ sentences that more clearly or precisely express the propositions that constitute the argument.
• Our reconstructed version of the argument may contain premises that are not expressed by any of the sentences actually used by the arguer.

Argument-reconstruction is essentially a task of interpretation.

The principle of charity.

In such facts pertaining to the context in which the argument is given, together with the specific words used by the person, will constitute the total evidence you have for reconstructing the argument.

• In some cases, the context is known, and makes it obvious what the arguer was implicitly assuming.
• In other cases, we may have to learn more about the context.
• In some other cases, however, we may learn all the relevant contextual factors, yet it remains possible to represent the person’s argument in more than one way.

If, in the third case, you have to chose what representation of the argument is true, it depends on your purpose.

• If you are hoping to convince others that the person is wrong, you are most likely to succeed if you represent it as a bad one.
• If what

Critical thinking
Chapter 6
Issues in argument assessment

Rational persuasiveness

The role of an argument is to give us reasons for accepting its conclusion as true.
The aim is to give an argument by which the intended audience is ought to be persuaded.

We cannot always tell whether or not the argument is sound.
Sound arguments must have true premises.

• An deductively sound argument is one that has true premises and which is deductively valid.
• An inductively sound argument is one with true premises that is inductively forceful.

Since we do not always know which propositions are true and which false, we cannot always tell whether an argument is sound or not.

To say that an inductively forceful argument is defeated for a person: the person reasonably believes the premises, but, nevertheless, reasonably rejects the conclusion.
(They have, for example, extra information).

An inductively forceful argument whose premises you have reason to accept is rationally persuasive only if your total evidence does not defeat the argument for you.

To say that an argument is rationally persuasive for a person:

• the argument is either deductively or inductively forceful
• the person reasonably believes the argument’s premises (at the time)
• it is not an inductively forceful argument that is defeated for that person (at that time).

Rationally unpersuasive argument: an argument that is deductively sound and valid, but gets you no closer to knowing the truth-value of the conclusion.

Rational persuasiveness is doubly relative.

• An argument is or is not rationally persuasive for a person at a particular time.

Since people are in different states of information at different times, an argument may be rationally persuasive for one person, but not for another.

Seven points to bear in mind as regards rational persuasiveness:

1 It  is not possible for the conclusion of a deductively valid argument to be defeated by a person’s total evidence. This is only possible for inductively forceful arguments.

• If you accept with good reason the premises of an argument that you recognize to be deductively valid, you must accept the conclusion as well.
• The adverb ‘probably’ (or a similar term) before the conclusion of an inductively forceful argument allows the possibility that the premises are true and the conclusion is false.

2 Rational persuasiveness is not part of the definition that the argument be sound (either deductively or inductively).

• The notion of rational persuasiveness is intended to capture what it is about an argument that constitutes its rational claim on a person.
  (For example: an argument can have a false premise,

This magazine contains all the summaries you need for the course WSRt at the second year of psychology at the Uva.

Critical thinking
Article: Simmons, Nelson, & Simonsohn (2011)
False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant

Abstract

This article is about two things:

• despite empirical psychologists’ nominal endorsement of a low rate of false-positive findings, flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to false find evidence that an effect exists than to correctly find evidence that it does not.
• a solution to that problem.

Beginning

One of the most costly errors is a false positive.

• The incorrect rejection of the null hypothesis.
• Once they appear in the literature, they are persistent.
  • Because null results have many possible causes, failures to replicate previous findings are never conclusive.
  • Because it is uncommon for prestigious journals to publish null findings or exact replication, researchers have little incentive to even attempt them.
• False positives waste resources

They inspire investment in fruitless research programs and can lead to ineffective policy changes.

Ambiguity is rampant in empirical research.

Solution

As a solution to the flexibility-ambiguity problem, there are offered six requirements for authors and four guidelines for reviewers.

This solution substantially mitigates the problem but imposes only a minimal burden on authors, reviewers, and readers.
Leaves the right and responsibility of identifying the most appropriate way to conduct research in the hands of researchers, requiring only that authors provide appropriately transparent descriptions of their methods so that reviewers and readers can make informed decisions regarding the credibility of their findings.

Requirements for authors

1. Authors must decide the rule for terminating data collection before data collection begins and report this rule in the article.

2. Authors must collect at least 20 observations per cell or else provide a compelling cost-of-data collection justification.
Samples smaller than 20 per cell are not powerful enough to detect most effects.

3. Authors must list all variables collected in a study
Prevents researchers from reporting only a convenient subset of the many measures that were collected, allowing readers and reviewers to easily identify possible researcher degrees of freedom.

4. Authors must report all experimental conditions, including failed manipulations
Prevents authors from selectively choosing only to report the condition comparisons that yield results that are consistent with their hypothesis.

5. If observations are eliminated, authors must also report what the statistical results are if those observations are included.
Makes transparent the extent to which a finding is reliant on the exclusion of observations, puts appropriate pressure on authors to justify

Critical thinking
Article: Dienes (2003)
Neyman, Pearson and hypothesis testing

Introduction

In this article, we will consider the standard logic of statistical inference.
Statistical inference: the logic underlying all the statistics you see in the professional journals of psychology and most other disciplines that regularly use statistics.

The underlying logic of statistic (Neyman-Pearson) is both highly controversial, frequently attacked (and defended) by statisticians and philosophers, and more frequently misunderstood.

Probability

The meaning of probability we choose determines what we can do with statistics.
The proper way of interpreting probability remains controversial, so there is still debate over what can be achieved with statistics.
The Neyman-Pearson approach follows form one particular interpretation of probability. The Bayesian approach considered follows form another.

Interpretations often start with a set of axioms that probabilities must follow.
Two interpretations of probability:

• the subjective interpretation: a probability is a degree of conviction of a belief
• the objective interpretation: locate probability in the world.

The most influential objective interpretation of probability is the long-run relative frequency interpretation. Here, probability is a relative frequency.
Because the long-run relative frequency is a property of all the events in the collective, it follows that a probability applies to a collective, not to any single event.
A single event could be a member of different collectives. So a singular event does not have a probability, only collectives do.

Objective probabilities do not apply to single cases. They also do not apply to the truth of hypotheses.
A hypothesis is simply true or false, just as a single event either occurs or does not.
A hypothesis is not a collective, it therefore does not have an objective probability.

Data and hypotheses

Data = D

Hypothesis = H

P(H|D) is the inverse of the conditional probability p(D|H). Inverting conditional probabilities makes a big difference.
P(A|B) can have a very different value from p((B|A).
If you know P(D|H) does not mean you know what p(H|D) is.
There are two reasons for this:

• inverse conditional probabilities can have very different values
• in any case, it is meaningless to assign an objective probability to a hypothesis.

Critical thinking
Article: Dienes (2008)
Degrees of falsifiability

Falsifiability

A potential falsifier of a theory: any potential observation that would contradict the theory.
One theory is more falsifiable than another if the class of potential falsifiers is larger.

Scientists prefer simple theories.
Simple theories are better testable.

A theory can gain in falsifiability not only by being precise, but also be being broad in range of situations to which the theory applies.
The greater the universality of a theory, the more falsifiable it is. Even if the predictions are not very precise.

Revisions to a theory may make it more falsifiable by specifying fine-grained causal mechanisms.
As long as the steps in a proposed causal pathway are testable, specifying the pathway gives you more falsifiers.

Psychologists sometimes theorize and make predictions by constructing computational models.
A computational model is a computer simulation of a subject, where the model is exposed to the same stimuli subjects receive and gives actual trial-by-trial responses.

A theory that allows everything explains nothing.
The more a theory forbids, the more it says about the world. The empirical content of a theory increases with its degree of falsifiability.

The more falsifiable a theory is, the more open it is to criticism.
So the more falsifiable our theories are, the faster we can make progress, given progress comes from criticism.

Science aims at the maximum falsifiability it can achieve: successive theories should be successively more falsifiable. Either in terms of universality or precision.

Make sure that any revision or amendment to theory can be falsified. That way theory development is guaranteed to keep its empirical character.

Observations

Observations are always ‘theory impregnated’.
Falsification is not so simple as pitting theory against observation.
Theories determine what an observation is.

Critical thinking
Article: Pearl (2018)
Confounding and deconfounding: or, slaying the lurking variable

Introduction

Confounding bias occurs when a variable influences both who is selected for the treatment and he outcome of the experiment.
Sometimes the confounders are known. Other times they are merely suspected and act as a ‘lurking third variable’.

If we have measurements of the third variable, then it is very easy to deconfound the true and spurious effects.

Statisticians both over- and underrate the importance of adjusting for possible confounders

• Overrate in the sense that they often control for many more variables than they need to and even for variables that they should not control for
• Underrate in the sense that they are loath to talk about causality at all, even if the controlling has been done correctly.

The chilling fear of confounding

Knowing the set of assumptions that stand behind a given conclusion is not less valuable than attempting to circumvent those assumptions with and RCT, which has complications on its own.

The skillful interrogation of nature: why RCTs work

The one circumstance under which scientists will abandon some of their reticence to talk about causality is when they have conducted a randomized controlled trial (RCT).

Randomization brings two benefits:

• It eliminates confounder bias
• It enables the researcher to quantify his uncertainty

Another ways is, if you know what all the possible counfounders are, to measure and adjust for them.
But, randomization had one great advantage: it servers every incoming link to the randomized variable, including the ones we don’t know about or cannot measure.

RCTs are preferred to observational studies.
But, in some cases, intervention may be physically impossible or unethical.

Provisional causality: causality contingent upon the set of assumptions that our causal diagram advertises.

The principal objective of an RCT is to eliminate confounding.

The new paradigm of confounding

Confounding is not a statistical notion. It stands for the discrepancy between what we want to assess (the causal effect) and what we actually do assess using statistical methods.
If you can’t articulate mathematically what you want to assess, you can’t expect to define what constitutes a discrepancy.

Historically, the concept of ‘confounding’ has evolved around two related conceptions:

• Incomparability
• A lurking third variable.

Both

Critical thinking
Article: Marewski, & Olsson, (2009)
Beyond the null ritual, formal modeling of psychological processes

Beyond the null ritual

Rituals can be characterized by a range of attributes including:

• Repetitions of the same action
• Fixations on special features such as numbers
• Anxieties about punishments for rule violations
• Wishful thinking

Each of these characteristics is reflected in null hypothesis significance testing.

One good way to make theories more precise is to cast them as formal models.
In doing so, researchers can move beyond the problems of null hypothesis significance testing, and simple difference searching.

What is a model?

In the broadest sense, a model is a simplified representation of the world that aims to explain observed data.
A model is a formal instantiation of a theory that specifies the theory’s predictions. This category also includes statistical tools, such as structural equation or regression models.
Statistical tools are not typically meant to mirror the workings of psychological mechanisms.

What is the scope of Modeling?

Modeling is not meant to be applied equally to all research questions. Each method has its specific advantages and disadvantages.

Modeling helps researchers answer involved questions and understand complex phenomena.
In psychology, modeling is especially suited for basic and applied research about the cognitive system.

Advantages of formally specifying theories

Four closely interrelated benefits of increasing the precision of theories by casting them as models:

• Models allow the design of strong tests of theories
• They can also sharpen research questions
• Models can lead beyond theories built on the general linear model
• Modeling helps to address real-world problems

Designing strong tests of theories

Models provide the bridge between theories and empirical evidence.
They enable researchers to make competing quantitative predictions, which in turn lead to strong comparative tests of theories.
Any quantitative prediction can be systematically better or worse than any other.

But, as soon as one starts to compare quantitative predictions from different models, the use of null hypothesis testing can become inappropriate or meaningless.

Sharpening research questions

Null hypothesis tests are often used to evaluate verbal, informal theories.
But, in such theories are underspecified, then they can be

Critical thinking
Article: Kievit, Frankenhuis, Waldorp, & Borsboom (2013)
Simpson's paradox in psychological science: a practical guide

Introduction

Simpson’s paradox: the direction of an association at the population-level may be reversed within the subgroups comprising that population.

Simpson showed that a statistical relation observed in a population could be reversed within all of the subgroups that make up that population.

What is Simpson’s paradox?

Simpson’s paradox is a counter-intuitive feature of aggregated data, which may arise when (causal) inferences are drawn across different explanatory levels. (like population to subgroup or subgroup to individual).

Simpson’s paradox is conceptually and analytically related to many statistical challenges and techniques.
The underlying shared theme of these techniques is that they are concerned with the nature of (causal) inference. The challenge is what inferences are warranted based on the data we observe.

Simpson’s paradox in individual differences

One can only be sure that a group-level finding generalizes to individuals when the data are ergodic, which is a very strict requirement.
Since this requirement is unlikely to hold in many data sets, extreme caution is warranted in generalizing across levels.
The dimensions that appear in a covariance structure analysis describe patterns of variation between people, not variation within individuals over time.

A person X may have a position on five dimensions compared to other people in a given population, but this does not imply that person varies along this number of dimensions over time.

Two variables may correlate positively across a population of individuals, but negatively within each individual over time.

A survival guide to Simpson’s paradox

Simpson’s paradox may occur in a wide variety of research designs, methods, and questions.
There is no single mathematical property that all instances of SP have in common. Therefore, there will not be a single, correct rule for analysing data so as to prevent cases of SP.

What we can do is consider the instances of SP we are most likely to encounter, and investigate them for characteristic warning signals.

The most general danger of psychology is that we might incorrectly infer that a finding at the level of the group generalizes to subgroups, or to individuals over time.

Preventing Simpson’s paradox

Develop and test mechanistic explanations

The first step in addressing SP is to carefully consider when it may arise.

The mechanistic inference we propose to explain the data may be incorrect.
This danger arises when we use data

Critical thinking
Article: Scott O. Lilienfeld (2005)
The 10 commandments of helping students distinguish science from pseudoscience in psychology

The ten commandments of helping students distinguish science from pseudoscience in psychology

The first commandment

It is important to communicate to students that the differences between between science and pseudoscience, although not absolute or clear-cut, are neither arbitrary or subjective.

Warning signs that characterize most pseudoscientific disciplines:

• A tendency to invoke ad hoc hypotheses, which can be thought of as ‘escape hatches’ or loopholes, as a means of immunizing claims from falsification.
• An absence of self-correction and an accompanying intellectual stagnation
• An emphasis on confirmation rather than refutation
• A tendency to place the burden of proof on sceptics, not proponents, of claims
• Excessive reliance on anecdotal and testimonial evidence to substantiate claims
• Evasion of the scrutiny afforded by peer review
• Absence of ‘connectivity’, a failure to build on existing scientific knowledge
• Use of impressive-sounding jargon whose primary purpose is to lend claims of facade of scientific respectability
• An absence of boundary conditions. A failure to specify the settings under which claims do not hold.

Non of these warnings signs is by itself sufficient to indicate that a discipline is pseudoscientific.
But, the more of these warning signs a discipline exhibits, the more suspect it should become.

The second commandment

Learning to distinguish scepticism from cynicism.
One danger of teaching students to distinguish science from pseudoscience is that we can inadvertently produce students who are reflexively dismissive of any claim that appears implausible.

Scepticism, which is the proper mental set of the scientist, implies two seemingly contradictory attitudes:

• An openness to claims
• A willingness to subject these claims to incisive scrutiny.

Cynicism implies close-mindedness.

The third commandment

Distinguish methodological scepticism from philosophical scepticism.

• Methodological (scientific) scepticism: an approach that subjects all knowledge claims to scrutiny with the goal of sorting out true from false claims
• Philosophical scepticism: an approach that denies the possibility of knowledge.

There is a continuum of confidence in scientific claims.

The fourth commandment

Distinguish pseudoscientific claims from claims that are merely false.
The key difference between science and pseudoscience lies not in their content but in their approach to evidence.

• Science seeks out contradictory information and eventually incorporates such information into its corpus of knowledge
• Pseudoscience tends to avoid contradictory information and thereby fails to foster the self-correction that is essential to scientific progress.

The fifth commandment

Distinguish science from scientists.

The scientific method is a toolbox of skills that scientists have developed to prevent themselves from confirming their own biases.

The sixth commandment

Explain

This magazine contains all the summaries you need for the course WSRt at the second year of psychology at the Uva.