12. Multiple Regression

• Regression objectives are either to predict the value of the dependent variable, or to estimate the marginal effect of each independent variable.
• A population multiple regression model is a model that includes multiple independent variables.
• Standard multiple regression assumptions include the four standard simple regression assumptions, plus a fifth one: It is not possible to find a set off nonzero numbers such that the sum of the coefficients equals zero.
• Multiple regression models include an error term, ε, that represents variability caused by variables not included in the model.
• In multiple regression coefficients are estimated using least squares, but these estimates become less reliable the higher the correlations between independent variables are.
• Any regression coefficient in a multiple regression model is dependent on all independent variables, and are thus referred to as conditional coefficients.
• Mean square regression (MSR) shows the proportion of the variability by the dependent variable that can be explained by the regression model.
• In a multiple regression model the sum-of-squares (SST; or sample variability) can be split into the sum of squares regression (SSR; or explained variability) and the sum of squares error (SSE; or unexplained variability). This is referred to as sum-of-squares decomposition.
• The coefficient of determination, R², describes the strength of the linear relationship between the independent variables and the dependent variables, and is calculated by 1 – SSE/SST.
• Adding more independent variables leads to a misleading increase in R², which can be avoided by calculating the adjusted coefficient of determination.
  
• The coefficient variance estimator, s²_b, is calculated as:
  The square root of s²_b is the coefficient standard error.
• Multiple regression models can be transformed into non-linear models, namely quadratic models and logarithmic models.
• Dummy variables can be used to represent categorical data in a regression model, and have a value of either 0 or 1.

13. Additional Topics in Regression Analysis

• Models are developed through four steps: model specification (selecting the variables, the algebraic form, and the data), coefficient estimation, model verification (checking whether the model is still accurate), and interpretation and inference.
• Dummy variables can be used to represent more than two categories by using multiple dummy variables. The rule is: number of categories -1 = number of dummy variables.
• In time series data the values of the dependent variable are related, this is then referred to as a lagged dependent variable.
• Not including important independent variables in a model can make any conclusions drawn from this model faulty.
• Multicollinearity is the phenomenon of two highly correlated independent variables. This leads to misleading estimated coefficients.
• Correlations between error terms are called auto-correlated errors. This leads to the estimated standard errors for the coefficients being biased, the null hypotheses falsely being rejected, and confidence intervals being too narrow. Autocorrelation can be formally tested with the Durbin-Watson test.

15. Analysis of Variance

• An Analysis of Variance (ANOVA) can be used to analyze data at more than two levels. It can compare more than two populations and uses assessments of variation.
• A one-way ANOVA tests the equality of population means.
• The total sum of squares (SST) in a one-way ANOVA is made up of a within-group sum of squares (SSW) and a between-groups sum of squares (SSG).
• The Kruskal-Wallis test is a nonparametric alternative to the one-way ANOVA and is used when there is a strong indication that the parent population distributions are markedly different from the normal.
• A two-way ANOVA test the equality of the population means when there is a second independent variable.
• The SST in a two-way ANOVA is made up of a between-blocks sum of squares (SSB), a between-groups sum of squares (SSG), and an error sum of squares (SSE).
• It is possible to have a two-way ANOVA with multiple observations per cell. This adds another factor that makes up the SST, the interaction sum of squares (SSI).

16. Time-Series Analysis and Forecasting

• Time series data involves measurements that are ordered over time, in which the sequence of observations is important.
• Most time-series have four components: a trend component (steady increases or decreases), a seasonality component (changes specific to seasons), a cyclical component (a repeating pattern), and an irregular component (representing unpredictability, similar to ε).
• Moving averages can be used to adjust time-series data by removing the irregular component and/or seasonal component. This is done by replacing a value by the average of itself and its two neighbouring values (for removing the irregular component) or producing four-period moving averages (removing the seasonal component).
• The effect of the seasonal component can be calculated through the seasonal index method which compares the smoothed data with the original data.
• When the data series is non-seasonal and has no consistent trends simple exponential smoothing can be used to predict future data in the time series. This is done through estimation of a weighted average of current and past values, where more weight is given to the most recent observations (with decreasing weight the older the observation is).
• The Holt-Winters exponential smoothing procedure allows for trend, by using the added variable of the trend estimateT_t-1. This procedure can also be extended to allow for seasonality by using a set of recursive estimates from the time-series.
• Based on autocorrelation patters between adjacent periods, the procedure of autoregressive models can use the available time-series data to estimate the parameters of a model of the process that could have generated this data.
• The Box-Jenkins approach is a flexible approach to prediction time-series data, based on how to choose the appropriate model.
• ARIMA models are autoregressive integrated moving average models.

17. Additional Topics in Sampling

• Stratified sampling involves breaking the population into strata (a.k.a. subgroups) according to a specific identifiable characteristic in such a way that each member of the population belongs to only one strata.
• Stratified random sampling is the process of selecting independent simple random samples from each strata.
• Sampling effort can be allocated among the strata by either using proportional allocation (sample-strata proportion is equal to strata-population proportion) or optimal allocation (strata with higher variance receive more sample effort).
• The method of optimal allocation is only optimal when trying to estimate an overall population parameter (mean/total/proportion) as precisely as possible.
• Cluster sampling involves breaking the population into clusters, making a random collection of clusters, and contact each member of these clusters for data.
• Two-Phase sampling involves an additional pilot study with a smaller sample previous to the study itself.
• Non-random sampling can either take the form of non-probabilistic sampling, where a convenient sample is chosen, or of quota sampling, where it is predetermined how many members of each group will be included.