Summary and study notes

Welke onderwerpen worden behandeld in het hoorcollege?

Pre-registration = stating as clearly and specifically as possible what you plan to do, and how, before doing it, in a manner that is verifiable by others. 

Publication bias = scientists publish only the significant results, because journals mostly publish only the significant studies. 

Harking (hypotheses after results known) = changing or creating the hypotheses after seeing the results.

P-hacking = researchers collect or select data or statistical analyses until nonsignificant results become significant. 

Selective reporting = when you only report the significant results. 

Welke onderwerpen worden besproken die niet worden behandeld in de literatuur?

In dit college worden geen andere onderwerpen besproken dit niet worden behandeld in de literatuur.  

Welke recente ontwikkelingen in het vakgebied worden besproken? 

Vanwege de toename in Registered Reports zijn er steeds meer journals en wetenschappers die ook non-significante resultaten publiceren, waardoor publicatie bias wordt verminderd. Daarnaast helpt pre-registratie mee aan het reproduceren en repliceren van studies. 

Welke opmerkingen worden er tijdens het college gedaan door de docent met betrekking tot het tentamen?

Er worden geen opmerkingen gedaan die betrekking hebben tot het tentamen. 

Welke vragen worden behandeld die gesteld kunnen worden op het tentamen? 

Er worden geen tentamenvragen behandeld. 

Hoorcollege aantekeningen

Pre-registration

• The results have to be beyond the control of a scientist, but at the same time the results are the most important thing for a scientist to make a good reputation. That’s why scientists often publish only the significant results = publication bias. 
• Harking (hypotheses after results known) = changing or creating the hypotheses after seeing the results. If you look at one hypothesis, your type I error rate will be equal to your alpha value. If you pick your hypothesis after you look at the results, your type I error rate will be more than your alpha. It is only harking when scientist have a hypothesis beforehand and change that hypothesis after the results, and then only publish the results that support their new hypothesis. The problem of the Brunell and Fisher experiment is that the finding could be the one that is incorrect (since there is a high experiment wise error rate). 
• P-hacking = researchers collect or select data or statistical analyses until nonsignificant results become significant. 
• Selective reporting = when you only report the significant results. 

Solution is to make results a dead currency in quality evaluation. The results shouldn’t have so much influence. You can make this happen by pre-registration. Researchers decide already what their hypotheses, procedures and analyses are before data collection. It doesn’t matter what the results are, if you do what you planned, you will get published. Pre-registration = stating as clearly and specifically as possible what you plan to do, and how, before doing it, in a manner that is verifiable by others. In your introduction you state your research question and your hypothesis. In your method you state the sample (size), design, variables, measures, exclusion criteria and analysis plan. It works in two stages, whereby the first stage is where the author submits the introduction and methods and in stage 2 the discussion and all the data. 

This prevents p-hacking, harking and selective reporting because the hypotheses and design are known on forehand and you cannot change it. 

The advantages for the scientific community are 

• a rigorous view of theory and methods, 
• eliminated publication bias and reporting bias 
• and increases the reproducibility of science. 

The advantages for scientists are 

• keeping track of what you did and why, 
• peer review when it is most helpful 
• and publication guaranteed regardless of the results. 

Common misconceptions are

• Pre-registration prevents the exploration of your data/creativity. No, pre-registration allows for exploratory science. It simply prevents reporting exploration analyses as confirmatory. Just say that you do a explanatory study. 
• Pre-registration does not allow for making changes and I cannot predict what will happen. You might make mistakes, and you can change it, as long as you report it and it is a good reason to change. 

Registered reports offered as an option at all reputable empirical journals, so that they can be legitimate career option for every researcher. All clinical trials published as registered reports. While also recognizing that registered reports are not applicable for all modes of research. 

• Person data = look up what for person you are dealing with. You have to report this.
• Data archive = save all your output for reproduction. 
• Replication = when your study can be done again, this will make your results more reliable. 

Sharing and open source

• Github = platform where you can share your material. 
• Pre-prints = first manuscript when you hand it in by a journal. 
• Open source software = free software for analysis, for example JASP or R.
• R Markdown = this way you can share code and show results from R easily.