Descriptive statistics

Deals with methods of organizing, summarizing and presenting data in a convenient and informative way

Inferential statistics

Body of methods used to draw conclusions or inferences about characteristics of populations based on sample data

Exit polls

For example: voters in the USA. A random sample of voters who exit the polling booth are asked for whom they voted.

Population

Group of all items of interest to a statistical practioner

Parameter

Descriptive measure of a population

Sample

Set of data drawn from studied population

Statistic

Descriptive measure of a sample

Statistical inference

Process of making an estimate, prediction or decision about a population based on sample data.

Confidence level

Proportion of times that an estimating procedure will be corrected

Significance level

How frequently a conclusion will be wrong

Variable

Some characteristic of a population or sample

Value (of a variable)

Possible observations of the variable

Data

Observed values of a variable

Interval data

A real number, such as weight / height. Also referred to as quantative or numerical

Nominal data

Values are categories, there is no meaning in their order. They can also be called qualitative or categorical

Ordinal data

The order of values has a meaning

Frequency distribution

Summarize data in a table which presents the categories and their counts

Relative frequency distribution

Lists the categories and the proportion with which each one occurs

Bar chart

A chart with rectangular bars with lengths proportional to the values that they represent. Used to display frequencies

Pie chart

A circular chart divided into sectors, illustrating numerical proportion. It shows relative frequencies

Univariate

Techniques applied to single sets of data

Bivariate

Techniques applied to depict a relationship between variables

Cross-clarification (cross-tabulation) table

Used to describe the relationship between two nominal variables

Classes

Number of observations that fall into a series of intervals (that cover a complete range of observations)

Histogram

A graph, created by drawing rectangles whose bases are the intervals and whose heights are the frequencies

Class width

Calculated by subtracting the smallest observation from the largest observation and dividing that number by the number of classes

Symmetric histogram

When you draw a vertical line in the centre, the two sides of the histogram are identical in shape and size

Skewness

A histogram is skewed when it has a long tail extending to the right (positively skewed) or the left (negatively skewed)

Mode

Observation that occurs with the greatest frequency

Modal class

Class with the largest number of observations

Unimodal histogram

A histogram with a single peak

Bimodal histogram

A histogram with two peaks, which can be unequal in height

Bell-shape

A special type of a symmetrical unimodal histogram

Stem-and-leaf display

A device for presenting quantitative data in a graphical format, similar to a histogram, to assist in visualizing the shape of a distribution. Observations must be placed in ascending order. Each observation will be divided into a stem and a leaf. Typically, the leaf contains the last digit of the number and the stem contains all of the other digits.

Ogive

Graphical representation of the cumulative relative frequencies

Cumulative relative frequency distribution

A tabular summary of a set of data showing the relative frequency of items less than or equal to the upper class class limit of each class. Relative frequency is the fraction or proportion of the total number of items.

Credit scorecards

Used by banks and financial institutions to determine whether the applicant will receive a loan

Time-series data

Represent measurements of successive points in time

Cross-sectional data

The observations are measured at the same time

Line chart

A type of chart which displays information as a series of data points called 'markers' connected by straight line segments.

Scatter diagram

A type of mathematical diagram using Cartesian coordinates to display values for two variables for a set of data.

Dependent variable

Variable depends to some degree on the other variable

Linear relationship

A straight line is drawn in the scatter diagram. There is a linear relationship if most points fall close to this line

Positive linear relationship

If one variable increases when the other does

Negative linear relationship

When two variables tend to move in the opposite direction

Graphical excellence

Is achieved when certain characteristics apply

Measures of central location

Mean, Median, Mode

Arithmetic mean

Often called mean or average. Computed by summing the observations and dividing by the number of observations

Median

Calculated by placing all observations in order. The middle observation is the median. When there are two middle observations, you take their average to calculate the median

Mode

Observation which occurs with the greatest frequency

Geometric mean

A type of mean or average, which indicates the central tendency or typical value of a set of numbers by using the product of their values (as opposed to the arithmetic mean which uses their sum).

Measures of variability

Range, variance, standard deviation and coefficient of variation

Range

The largest observation minus the smallest observation

Variance

Measures how far a set of numbers is spread out.

Mean absolute deviation (MAD)

The average absolute value of the deviations

Standard deviation

Shows how much variation or dispersion from the average exists. Computed by the positive square root of the variance

Empirical Rule

Only used if the histogram is bell-shaped. Approximately 68% of all observations fall within one standard deviation of the mean. 95% is within two standard deviations of the mean and 99.7% is within three standard deviations of the mean

Chebycheff’s Theorem

The proportion of observations in any sample or population that lie within k standard deviations of the mean is at least 1-(1/k²) for k > 1 – If k=2, at least 75% of all observations lie within two standard deviations of the mean

Coefficient of variation

The standard deviation of observations divided by their mean.

Measures of relative standing

Designed to provide info about the position of particular values relative to an entire data set

Percentile

The Pth percentile is the value for which P percent are less than that value and (100-P)% are great than that value

Quartiles

The first quartile is the 25^th percentile. The median is the 50^th percentile. The upper quartile is at the 75^th percentile.

Interquartile range

Spread of the middle 50% of observations; the 75^th percentile minus the 25^th percentile

Box plots

A graphical representation of five statistics: the minimum and maximum observations plus the 25^th, 50^th and 75^th percentile

Whiskers

Lines extending vertically from the boxes, indicating variability outside the upper and lower quartile

Outliers

Unusually large or small observations, any point that lies outside the whiskers

Measures of linear relationship

Covariance, coefficient of correlation, coefficient of determination

Covariance

A measure of how much two random variables change together.

Coefficient of correlation

Covariance divided by the standard deviations of the variables

Breakeven analysis

How much sales volume your company needs to start making profit

Least squares method

A standard approach to the approximate solution of over determined systems, i.e., sets of equations in which there are more equations than unknowns. 'Least squares' means that the overall solution minimizes the sum of the squares of the errors made in the results of every single equation.

Coefficient of Determination

Measures the amount of variation in the dependent variable that is explained by the variation in the independent variable. Denoted as R² (R-squared)

Random experiment

Action or process that leads to one of several possible outcomes

Exhaustive

All possible outcomes are included

Mutually exclusive

No two outcomes can occur at the same time

Sample space (of a random experiment)

List of all the possible outcomes of the experiment. They must be exhaustive and mutually exclusive

Classic approach

To assigning probabilities, associated with game or chance. For example, to flip a coin or toss a die

Relative frequency approach

To assigning probabilities, defines probability as a long run relative frequency with which an outcome occurs.

Subjective approach

To assigning probabilities, defines probability as a degree of belief that we hold in the occurrence of an event

Event

A collection or set of one or more simple events in a sample space

Simple event

The individual outcome of a sample space

Probability of an event

Sum of probabilities of the simple event that constitute the event

Joint probability

The probability of the intersection of events A and B, that is, the event that occurs when both A and B occur

Marginal probability

Computed by adding across rows or down columns, named so because they are calculated in the margins of the table

Conditional probability

Probability of one event given the occurrence of another related event

Independent events

Two events are independent if the probability of one event is not affected by the occurrence of another event

Union

The union of events A and B is the event that occurs when either A or B or both occur

Complement rule

The complement of event A is the event that occurs when event A does not occur. The probability of an event and the probability of an event’s complement must sum to 1.

Multiplication rule

Used to calculate the joint probability of two events

Addition rule

Used to calculate the probability of the union of two events

Probability tree

Shows all the possible events. The first event is represented by a dot. From the dot, branches are drawn to represent all possible outcomes of the event. The probability of each outcome is written on its branch.

Bayes’s Law

Method for computing conditional probabilities.

Prior probability

Determined prior to a decision

Likelihood probability

Conditional probability

Posterior (Revised) probability

Prior probability that is revised after a decision is taken

False-positive result

In medicine, the patient does not have the disease, the test shows positive

False-negative result

In medicine, the patient has the disease, the test shows negative

Random variable

A function or rule that assigns a number to each outcome of an experiment

Discrete random variable

Countable number of values

Continuous random variable

The values are uncountable

Probability distribution

A table, formula or graph that describes the values of a random variable and the probability associated with these values

Expected value

The population mean is the weighted average of all of its values. The weights are probabilities. Represented by E(X)

Population variance

Weighted average of squared deviations from the mean

Bivariate distribution

Provides probabilities of combinations of two variables

Bivariate probability distribution

Of X and Y, table of formula that lists the joint probabilities for all pairs of values of x and y

Binomial distribution

Result of binomial experiment, which consists of a fixed number of trails, n. The outcome of each trail is either success or failure. The trails are independent.

Bernoulli process

If all properties of a binomial experiment are satisfied

Binomial random variable

The number of successes in n trails

Cumulative probability

The probability that a random variable is less than or equal to a value

Poisson distribution

Result of Poisson experiment

Poisson random variable

The number of successes that occur in a period of time or an interval of space in a Poisson experiment

Probability Density Function

Since a continuous random variable can take an uncountable number of values, we take the probability of a range of values, which results in the probability density function

Uniform (rectangular) probability distribution

Used to find the area under the curve that describes a probability density function

Normal density function

Probability density function of a normal random variable

Standard normal random variable

Denoted as Z. Standardization of a random variable, calculated by subtracting its mean and dividing by its standard deviation. The probability of Z can be found in a table

Exponential distribution

The probability distribution that describes the time between events in a Poisson process. The exponential distribution has no memory.

Student t distribution

A family of continuous probability distributions that arises when estimating the mean of a normally distributed population in situations where the sample size is small and population standard deviation is unknown.

Degrees of freedom

the number of values in the final calculation of a statistic that are free to vary

Chi-Squared distribution

With k degrees of freedom is the distribution of a sum of the squares of k independent standard normal random variables.

F distribution

A continuous statistical distribution which arises in the testing of whether two observed samples have the same variance.

Sampling distribution

Created by sampling

Standard error of the mean

Standard deviation of the sampling distribution, calculated by the standard deviation of the population divided by the sample size

Central Limit Theorem

The sampling distribution of the mean of a random sample drawn from any population is approximately normal for a sufficiently large sample size. The larger the sample size, the more closely the sampling distribution will resemble a normal distribution

Sample proportion

The estimator of a population proportion of successes.

Continuity correction factor

When using a normal approximation to the binomial distribution, draw rectangles whose bases where constructed by adding/subtracting 0.5 to the values of X. Omit the factor when computed a range of values of X

Point estimator

Draws inference about a population by estimating the value of an unknown parameter using a single value or point

Interval estimator

Draws inferences about a population by estimating the value of an unknown parameter using an interval

Unbiased estimator

Of a population parameter, is an estimator whose expected value is equal to that parameter

Consistency

Of an unbiased estimator. Consistency is there if the difference between the estimator and the parameter grows smaller as the sample size grows larger

Relative efficiency

The one of the two unbiased estimators of parameter with a smaller variance is relative efficient.

Confidence level

The probability of 1-α

LCL

Lower confidence limit, calculated with the confidence interval estimator

UCL

Upper confidence limit, calculated with the confidence interval estimator

Confidence interval (CI)

A type of interval estimate of a population parameter and is used to indicate the reliability of an estimate.

Error of estimation

The difference between an estimated value and the true value of a parameter or, sometimes, of a value to be predicted.

Bound error of estimation

Labelled as B, the maximum error of estimation we are willing to tolerate

Hypothesis test

Refers to the formal procedures used by statisticians to accept or reject statistical hypotheses.

Statistical hypothesis

Assumption about a population parameter

Null hypothesis

Denoted by H0, is usually the hypothesis that sample observations result purely from chance.

Alternative (research) hypothesis

Denoted by H1, is the hypothesis that sample observations are influenced by some non-random cause.

Type | error

To reject a true null hypothesis

Type || error

To not reject a false null hypothesis

Significance level

The probability of a type | error, α

Test statistic

Criterion on which we base our decision about the hypothesis

Rejection region

Range of values such that if the test statistic falls into that range, we decide to reject the null hypothesis in favour of the alternative hypothesis

Standardized test statistic

Denoted as z, calculated by a formula

Statistically significant

When H0 is rejected, a test is said to be statistically significant at whatever significance level it was conducted

p-value

Of a test is the probability of observing a test statistic at least as extreme as the one computed given that the null hypothesis is true

Highly significant

p-value less than 0.01 – overwhelming evidence to infer that H1 is true

Significant

p-value between 0.01 and 0.05 – strong evidence to infer that H1 is true

Not statistically significant

p-value between 0.05 and 0.10 – weak evidence to indicate that H1 is true. If the p-value > 0.1, there is little to no evidence.

One tail test

Whenever H1 specifies that the value is bigger or smaller than the value stated in H0.

Two tail test

Whenever H1 specifies that the value is not equal to the value stated in H0

Power of a test

The probability that the test will reject the null hypothesis when the alternative hypothesis is true. Denoted as 1-β, where β is the probability of a type || error

t-statistic

A ratio of the departure of an estimated parameter from its notional value and its standard error.

Robust

A student t distribution is robust; which means that if the population is nonnormal, the results of the t-test and confidence interval estimator are still valid provided that the population is not extremely nonnormal.

t-test

Any statistical hypothesis test in which the test statistic follows a Student's t distribution if the null hypothesis is supported.

Large populations

Populations at least 20 times the sample size

Chi-squared statistic

Estimator of the sample variance

p-test

A statistical method used to test one or more hypotheses within a population or a proportion within a population.

Wilson Estimator

Used when there are zero successes in a sample