#### Standardized Residuals
Standardized residuals adjust residuals ($\hat{\epsilon}_i$) by their estimated standard deviation:

$$di =\frac{e_i}{\sqrt{MS_{\text{RES}}}}$$​
Where:
- $e_i$ is the **residual** for the i-th observation. A residual is the difference between the observed value and the predicted value from the regression model, i.e., $e_i = y_i - \hat{y}_i$
- $MS_{RES}$ is the **mean square residual**, which is an estimate of the **variance** of the error terms:

#### Studentized Residuals

**Internally Studentized Residuals**:

$$r_i = \frac{\epsilon_i}{\sqrt{\sigma_i^{2} (1 - h_i)}}$$

Where:
- $ε_i$: Residual for the i-th observation.
- $σ$: Residual standard error using all observations.
- $h_i$: Leverage for the i-th observation (from the hat matrix).


**Externally Studentized Residuals**:

$$
t_i = \frac{\epsilon_i}{\sqrt{S_i^{2} (1 - h_i)}}
$$

Where:
- $\epsilon_i$: Residual of the the $i$-th observation.
- $\sigma^{(i)}$: Standard error of the residual excluding the $i$-th observation (leave-one-out variance).
- $h_i$: Leverage for the $i$-th observation.

$$
S_i^{2} = \frac{(n - p)}{(n - p - 1)} \left(MS_{\text{RES}} - \frac{e_i^2}{(1 - h_i)} \right)
$$
---

### Cook's Distance
Cook’s Distance measures the influence of an observation:
$$
D_i = \frac{(\hat{\boldsymbol{\beta}} - \hat{\boldsymbol{\beta}}_{(-i)})^\top \mathbf{X}^\top \mathbf{X} (\hat{\boldsymbol{\beta}} - \hat{\boldsymbol{\beta}}_{(-i)})}{(k+1) \hat{\sigma}^2}
$$
Alternatively: (BETTER)
$$D_i = \frac{\sum_{j=1}^n \left( \hat{Y}_j - \hat{Y}_{j(i)} \right)^2}{p \times \text{MSE}}$$


---

### DFBETAS
DFBETAS measures how much each coefficient changes when an observation is removed:
$$
DFBETAS_{ij} = \frac{\hat{\beta}_j - \hat{\beta}_{j(-i)}}{\hat{\sigma}_{(-i)} \sqrt{c_{jj}}}
$$
Where:
- $\hat{\beta}_{j(-i)}$: Estimated coefficient after removing $i$th observation.
- $c_{jj}$: $j$th diagonal element of $(\mathbf{X}^\top \mathbf{X})^{-1}$.
- **Interpretation**:
  - Large absolute values ($> 2/\sqrt{n}$) indicate high influence.

---

### DFFITS
DFFITS measures how much the fitted value changes when an observation is removed:
$$
DFFITS_i = \frac{\hat{Y}_i - \hat{Y}_{i(-i)}}{\hat{\sigma}_{(-i)} \sqrt{h_{ii}}}
$$
Alternatively:
$$
DFFITS_i = t_{i(-i)} \sqrt{\frac{h_{ii}}{1 - h_{ii}}}
$$
- **Interpretation**:
  - A large absolute value ($> 2\sqrt{(k+1)/n}$) suggests high influence.

---

### AIC (Akaike Information Criterion)
$$
AIC = 2k + 2\ln(L)
$$
Where:
- $\hat{\sigma}^2$: Residual variance.
- $k$: Number of predictors (excluding intercept).
- **Interpretation**:
  - Lower AIC indicates a better model.

---

### BIC (Bayesian Information Criterion)
$$
BIC = \ln(n)k + 2\ln(L)
$$
- **Interpretation**:
  - Like AIC but penalizes complexity more. Lower is better.

---

### Mallows’ $C_p$ Criterion
$$
C_p = \frac{\text{RSS}}{\hat{\sigma}^2} + 2p - n
$$
Where:
- $\text{RSS}$: Residual sum of squares.
- $\hat{\sigma}^2$: Estimate of error variance.
- $p$: Number of predictors (including intercept).
- $n$: Number of observations.
- **Interpretation**:
  - Models with $C_p \approx p$ are preferred.
