aantekeningen lecture 2 Academic Research Methods and Statistics (ARMS) module 6 Unuasal data-sets 22/23 Universiteit Utrecht

• Raw score: score the test using test manual/instructions. For example what are the requirements to score a 8 on a test. 
• Look up the norm data. For example people usually score a 6
• Standardize based on norm data: Compute adjusted score considering age, sex, and/or education OR compare to norm group matched on age, sex, and/or education. You do this by using the Adjusted score calculation underneath the table of the norm data. Here you will fill in the information about your patient, this will give you their adjusted score. Get scaled score, percentile, and/or z-score (depending on the norm data), for this you can look at the tables (Equivalent score=your computed adjusted score). Then look at your scale to percentille z-scores, to computed your z-score. z = NORMSINV(percentile of your score example 0.50) = ....
• Interpret: Check whether z-score/percentile is significant using the standard normal curve (score is lower then -1.645) or two-sided (score is lower then -1.96 or higher then 1.96)). If your score doenst fall in this you can say your score is significant. mention if the test was one or two sided!
• Report: percentile, z-score, one-sided or two-sided, interpretation.

Norm data: You want to know if the patient peforms differently than before the onset of the disease or illness. Most of the time you don't have such a baseline measure and have to make a estimate. You can do this with:

• Patient interview: ask the patient self and/or their loved ones about their impairments. You want to things like their job, a architect will have more visuospatial constructions skills then a secretaris.
• Norm data: waht do people score that are similar to your patient on a given test? does he fall within the expected range of performance or not? The used norm group must match your patient. You can collect norm data with standarized tests. All clinicals or researchers should use the same test materials and test protocol. Side not: Always think critically about which subpopulation the norm data came from. If this doenst match it can lead to wrong conclusions. --> when using norm data look at age, the subpopulation, when was it collected etc

steps for Single Case statistics without norm data

• Raw score
• Collect data of a control group: Calculate the mean and standard deviations for this group. using excel: Mean = AVERAGE(…) or GEMIDDELDE(…). SD = STDEV(…)

• Standardize basend on control data using z-score: Where does your participants score lie on a normal distribution. To compute this use:Formulas in excel: z = NORMSINV(p) or z = STAND.NORM.INV(p). p = NORMSDIST(z) or STAND.NORM.VERD(z)

• Interpret: Do this the same way as you would when you do have the norm data. 

Reasons to use a small control group:

• Obtain a homogeneous, well-matched group: Large control groups are typically heterogenous with respect to certain background variables. However, there might be a strong need to match the control group on all or certain crticial demograpic variables to the patient => allows stronger matching
• Effortful to compose a large control group (e.g. brain scans) Composing a large control group could be very effortfull (for example when also brain scans in the controls must be collected)
• Small group of control patients. Control groups could be a small group of control patients
• Certain patient behaviors instantly seen as deviant. Certain patient behaviors can more or less instantly to be seen as deviant. Hence only a few controls suffices (i.e. use an absolute score rather than relative score). For example, if a patient does not recall his own name this is so clearly erroneous that there is no need for a large control sample 

Modified t-test: you use this to correct for the higher chance at a false  positive/type 1 error when using a small control group. Compared to the z-test it also takes the sample size in account. H0= patneint did come from the control population. H1= patient did not come from the control population

Monte Carlo Simulation: to test whatever using a z-test or modified t-test leads to a higher change at type 1 error. 

• Take a sample size (N)
• Take this sample size plus a observation from a normal distribution
• Test this using both a z-test and modified t-test and repeat this
• Compare the results --> z-test results in lager % of type 1 error, especially in smaller sample sizes. 

problems with assumptions: There is often no normal distribution. The control/normal group scores really high (positvely skewed distribution), while the patient group scores really low (negatively skewed distribution). Or leptokurtric distribution of control scores, the score range is limited on either side. When using a z-test you get a more extreme skew.  

• Raw score
• Compute mean and standard deviation of control group
• Standarize based on control data using modified t-test.
• Manually: use the t-test calculation shown above in excel. See the example below
• SINGLIMS_ES program: you can fill in all the data and the program will calculate this for you. It will also give you the p-value and effect size and other additional data. 
• Interpret: comporte the t tot the critical t-value. Is it higher or lower? It has to fall in the significant part of the normal distribution

• Patient differs from controls on task X
• Patient doesn’t differ from controls on task Y
• Patient’s performance on task X and Y differ

Double dissociation: Patient A is impaired on task X (e.g. speaking), and not on task Y (e.g. comprehension) Patient B is impaired on task Y (e.g. comprehension), and not on task X (e.g. speaking)