### Multiple Linear Regression

#### Formula for Multiple Linear Regression
The general formula for multiple linear regression is:

$$
Y_i = \beta_0 + \beta_1X_{1i} + \beta_2X_{2i} + \cdots + \beta_kX_{ki} + \epsilon_i
$$

In matrix notation, the compressed form is:

$$
\mathbf{Y} = \mathbf{X} \boldsymbol{\beta} + \boldsymbol{\epsilon}
$$

Where:
- $\mathbf{Y}$: $n \times 1$ vector of the dependent variable.
- $\mathbf{X}$: $n \times (k+1)$ matrix of predictors (including a column of 1's for the intercept).
- $\boldsymbol{\beta}$: $(k+1) \times 1$ vector of coefficients.
- $\boldsymbol{\epsilon}$: $n \times 1$ vector of residuals.

---

### Deriving the OLS (Ordinary Least Squares)

The goal of OLS is to minimize the sum of squared residuals:

$$
S(\boldsymbol{\beta}) = \sum_{i=1}^n \epsilon_i^2 = \sum_{i=1}^n (Y_i - \mathbf{X}_i \boldsymbol{\beta})^2
$$

1. **Residual Vector**:
   $$ 
   \boldsymbol{\epsilon} = \mathbf{Y} - \mathbf{X} \boldsymbol{\beta}
   $$

2. **Objective Function**:
   $$ 
   S(\boldsymbol{\beta}) = (\mathbf{Y} - \mathbf{X} \boldsymbol{\beta})^\top (\mathbf{Y} - \mathbf{X} \boldsymbol{\beta})
   $$

3. **Expand the Function**:
   $$ 
   S(\boldsymbol{\beta}) = \mathbf{Y}^\top \mathbf{Y} - 2 \boldsymbol{\beta}^\top \mathbf{X}^\top \mathbf{Y} + \boldsymbol{\beta}^\top \mathbf{X}^\top \mathbf{X} \boldsymbol{\beta}
   $$

4. **Take Derivative with Respect to $\boldsymbol{\beta}$**:
   $$ 
   \frac{\partial S(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}} = -2 \mathbf{X}^\top \mathbf{Y} + 2 \mathbf{X}^\top \mathbf{X} \boldsymbol{\beta}
   $$

5. **Set Derivative to Zero**:
   $$ 
   \mathbf{X}^\top \mathbf{Y} = \mathbf{X}^\top \mathbf{X} \boldsymbol{\beta}
   $$

6. **Solve for $\boldsymbol{\beta}$**:
   $$ 
   \boldsymbol{\beta} = (\mathbf{X}^\top \mathbf{X})^{-1} \mathbf{X}^\top \mathbf{Y}
   $$

---

#### Heteroskedasticity (Breusch-Pagan Test) (refer notes)
1. **Using $R^2$ of Auxiliary Regression**:
   - Regress the squared residuals ($\hat{\epsilon}_i^2$) on the independent variables.
   - Compute:
     $$
     \text{Test Statistic (Breusch-Pagan)} = n \cdot R^2
     $$
   - Compare to $\chi^2$ distribution with $k$ degrees of freedom.
   - **Interpretation**:
     - If the test statistic exceeds the critical value, reject the null hypothesis of homoscedasticity (variance is constant).

#### Multicollinearity (VIF and $1 - R^2$ method)
2. **Variance Inflation Factor (VIF)**:
   $$ 
   \text{VIF}_j = \frac{1}{1 - R_j^2}
   $$
   Where $R_j^2$ is the $R^2$ obtained by regressing the $j$th predictor on all other predictors.
   - **Interpretation**:
     - VIF > 10 indicates severe multicollinearity.

3. **$1 - R^2$ Method**:
   - Regress each predictor variable on the other predictors and calculate $R_j^2$.
   - If $1 - R_j^2$ is large for a predictor, it suggests multicollinearity.
   - **Interpretation**:
     - High $1 - R^2$ values suggest that the predictor has a high linear dependency on other predictors.

---

#### Testing for Autocorrelation

**Durbin-Watson Test**:
- Statistic:
  $$ 
  d = \frac{\sum_{i=2}^n (\hat{\epsilon}_i - \hat{\epsilon}_{i-1})^2}{\sum_{i=1}^n \hat{\epsilon}_i^2}
  $$
- **Interpretation**:
  - $d \approx 2$: No autocorrelation.
  - $d < 2$: Positive autocorrelation.
  - $d > 2$: Negative autocorrelation.

---
