HC6+7: Bias

Medical research

Medical research often consists of the quantification of a phenomenon, for example quantifying the effect of cholesterol lowering treatment on the risk of cardiovascular events among those with elevated serum cholesterol levels. Ideally, results apply not only to those who participate in the study (the study population/sample), but also to others with elevated cholesterol levels (the domain).

The domain consists of a large group for whom the results of the study should apply. Of this domain, a sample is selected based on:

• In-/exclusion criteria
• Region
• Time period
• Et cetera

While the study is done within this sample, the results of the study should apply to the domain. If there is a systematic error, results of the study do not apply to the domain.

Systematic errors:

Systematic errors in design, conduct or analysis may lead to erroneous results → bias. An example of a systematic error is a broken measuring tape when measuring the length of something. Each time the measuring tape is used, results are incorrect → a systematic error is not solved by a large sample size.

Random errors:

Random errors can be decreased by increasing the sample size. As the study size increases, random errors start playing a increasingly smaller role.

Sources of bias

There are 3 sources of bias:

• Measurement errors → information bias
• Missing data → selection bias
• Incomparability of study groups → confounding

There can be bias in different parts of studies, depending on the study type:

• Randomized trial
  • Outcome
• Cohort study
  • Exposure
  • Outcome
  • Confounders
• Case-control study
  • Exposure
  • Outcome
  • Confounders

Measurement errors

Examples of measurement errors are:

• Questionnaires for measuring body weight instead of scales
• Drug use measured by “prescription information” instead of controlling if the drugs actually are taken
• Measuring smoking status with “yes/no” instead of pack years

This changes the risk ratio → results of the study are (often) incorrect.

Consequences:

Measurement errors can have different consequences:

• Can result in over-/underestimation
• Better to prevent than to cure
• Correction is possible

Missing data

In case of missing data, certain information is not available:

• Questionnaire is not returned
• Question is not answered
• Study drop-outs
• Individuals not willing to participate

The extent of the impact of missing data often depends on the research question, study design, data collection, etc. This changes the risk ratio → a small percentage of missing data (3%) can lead to 100% bias, but this doesn’t necessarily happen → 50% missing data can also result in no bias (the ratio remains the same). Depending on if it is a selective or random process, missing data has a different influence on the results.

Selection bias:

Selection bias arises when in either of the exposure groups, a selection was made on stroke. Selecting on the outcome in a follow-up study hardly ever occurs since the participants have not yet developed the outcome at the start of follow-up.

Meta-analysis:

In meta-analysis, information is taken from multiple studies and is meta-analyzed → an average of all the available information is taken. For instance, some studies will be in favor of a certain treatment, while some are in favor of a placebo. These results are all taken into account and an average is made. Missing data in a meta-analysis may be caused by some studies not being reported → often happens if the results are in favor of the placebo. This leads to the meta-analysis showing that the drug is more effective, while it actually is not. This is called publication bias.

Consequences:

In conclusion, missing data:

• Can result in over-/underestimation
• Is better to prevent than cure
• Can be corrected → imputation
  • This doesn’t solve the problem completely

Confounding

Confounding arises due to incomparability of exposure groups, for example if 1 exposure group is on average much older than the other exposure group. This may lead to the claim that an observed relation between exposure and outcome is causal, while it actually is not. The observed relation is partly or completely due to other reasons called “mixing of effects”, but does not represent a casual effect.

Incomparability:

If the mean age among exposed is 80 years, and the mean age among unexposed 60 years, the RR will be confounded → incomparability. The RR isn’t due to the exposure effect anymore, because the 2 groups are incomparable.

Conditions:

A confounder disrupts the relation between the exposure and the outcome. To be a confounding factor, 3 conditions must be met:

• Being associated with the exposure
• Being associated with the outcome
  • Independently of exposure
• Not being an intermediate
  • Cannot be a consequence of the exposure → not in the causal pathway

Numerical example:

The relation between the exposure and outcome of the following sample is investigated:

It needs to be investigated whether the RR of 1,11 represents the causal effect of exposure on the outcome and if the observed difference in outcomes between exposure groups is due to the exposure. The observed RR of 1,11 may be the result of incomparability between the exposure groups → confounding. To find out if this is the case, the sample is split in 2 groups of men and women individually.

These tables show the following:

• The risk of unexposed men developing the outcome is 30%, while the risk of unexposed women developing the outcome is 90% → gender is a risk factor for the outcome
• 25% of men are exposed and 75% of women are exposed → gender is not only a risk factor for the outcome, but is also related to exposure status
• The effect of exposure in both women and men is estimated to be around 0,67, while the overall estimated effect is 1,11

This shows that the overall RR is 1,11 and among men and women the RR is 0,67. The observed relation of 1,11 is a mixture of the “true” effect (0,67) and the effect of incomparability of exposure groups with respect to gender. Among men and women separately, exposure groups are comparable with respect to gender → in the subgroups, there is no confounding by gender anymore.

Does smoking increase the risk of a heart attack?

There is a relation between smoking and a heart attack:

• Heart attack: may be influenced by smoking and sex + age
• Smoking: influenced by sex + age, causes heart attacks
• Confounding factors: influences smoking and heart attacks
  • Sex, age, stress, SES, etc.
• Mediators of the causal effects of smoking on heart attacks: blood pressure and cholesterol
  • Not confounding variables

Several things can be done to prevent bias:

• Restriction: the study is only of elderly men
  • By restricting the study population to a group of individuals who are all male and have the same age, confounding can be prevented
  • A heart attack mainly occurs in elderly men and elderly men are more likely to smoke
  • Downside: the sample is smaller and the results are less precise
• Stratification: analysis in subgroups
  • All the possible subgroups are evaluated and a separate analysis within each subgroup is made
• Matching: creates comparability
• Statistical adjustment: regression analysis
  • Makes a statistical adjustment for confounding

Matching:

In matching, for each individual who is exposed, a comparable individual who isn’t exposed is found → are taken as a pair and matched. This way, a data set is made with exposed and unexposed groups who are very comparable is made. Matching is done in both cohort studies and case-control studies:

• Cohort studies
  • Exposed and non-exposed are matched with the aim to prevent confounding
  • Analogy is done with randomized trials
  • There only is comparability with respect to matched, observed, variables
  • There is potential for unmeasured confounding
• Case-control studies
  • Diseased and controls are made artificially → not always a good thing
    • Diseased and controls should differ with respect to many (prognostic) factors
    • Matching cases and controls in case-control studies is “unnatural” and overrated
    • Can lead to is seeming like there no longer is an association between the exposure and the outcome → not the true effect
  • Is used to increase the efficiency
  • Can be matched or unmatched
    • 100 cases of whom 95 are female
    • 100 controls of whom 50 are female
    • Unmatched: because more females have BRC than males, the gender ratio in cases will be different than the ratio in controls
      • Within the subgroup of females → 95 cases : 50 controls
      • Within the subgroup of males → 5 cases : 50 controls
    • Matched
      • Within the subgroup of females → 95 cases : 95 controls
      • Within the subgroup of males → 5 cases : 5 controls

Consequences:

Confounding can:

• Be relevant in observational studies of casual effects
• Can result in over-/underestimation
• Can be prevented
  • E.g. with randomization
• Correction is possible, but only for measured confounding variables
• There is always potential for unmeasured confounding due to variables that were not measured

Risk difference

The risk difference is calculated by subtracting the cumulative incidence in the unexposed group (or least exposed group) from the cumulative incidence in the group with the exposure. 

Number needed to harm:

The number needed to harm (NNH) is an epidemiological measure that indicates how many persons on average need to be exposed to a risk factor over a specific period to cause harm in an average of one person who would not otherwise have been harmed. It is calculated as follows:

• NNH = 1/risk difference