OLS Fit
The Problem
Prediction Check
◆ THE BLUEPRINT
The Problem with OLS

OLS fits a straight line through any data. It has no concept of a bounded response. The model will predict wherever the line goes.

What Goes Wrong
OLS: \(\hat{y} = \beta_0 + \beta_1 x\)

For binary data, predicted probabilities go above 1 or below 0. For counts, predicted counts go negative. These are nonsensical.

The GLM Fix

A GLM wraps the linear predictor in a link function that constrains predictions to the correct domain.

GLM: \(g(\mu) = \eta = X\beta\), so \(\mu = g^{-1}(\eta)\)
Link Functions & Their Inverses
Identity: \(g(\mu) = \mu\), so \(\mu = \eta\) — maps to \((-\infty, +\infty)\)
Logit: \(g(\mu) = \log\frac{\mu}{1-\mu}\), so \(\mu = \frac{e^\eta}{1+e^\eta}\) — maps to \((0, 1)\)
Log: \(g(\mu) = \log(\mu)\), so \(\mu = e^\eta\) — maps to \((0, +\infty)\)
Inverse: \(g(\mu) = 1/\mu\), so \(\mu = 1/\eta\) — maps to \((0, +\infty)\) for \(\eta > 0\)
Key Insight

OLS is a GLM with a normal distribution and identity link. The rest of this app explores what happens when you change the link.

Explore More
Regression & Regularization see how loss functions determine the OLS fit Bias-Variance Tradeoff connect MSE to the bias-variance decomposition
Data + Fitted Curves
Deviance Residuals vs Fitted
Model Metrics
◆ THE BLUEPRINT
What You're Looking At

Data generated from a specific GLM family, then fit by several candidate models. The fitted curves, residual plots, and metrics reveal which link function is appropriate.

Key Metrics
AIC: \(\text{AIC} = -2\ell + 2k\)
Deviance: \(D = 2(\ell_{\text{saturated}} - \ell_{\text{model}})\)
Reading the Residuals

A model with the wrong link function shows systematic patterns in deviance residuals. The correct model's residuals scatter randomly around zero.

Some model-data combinations will fail entirely. You cannot fit a binomial GLM to continuous data outside [0, 1]. The failure itself is informative.

Which Family?

Look at your response variable's domain.

Response Family / Link
Binary (0/1) Binomial / Logit
Counts (0, 1, 2, ...) Poisson / Log
Positive continuous Gamma / Inverse
Continuous (any real) Gaussian / Identity
X-Range Sliders

Restrict the x domain to see how models behave on limited data. On a narrow range, even a wrong link can look reasonable. Widen the range to see the mismatch.

Explore More
Bootstrap & CV use cross-validation to compare models when the true model is unknown Bias-Variance Tradeoff model misspecification as a source of bias
Response Curve
Classification Metrics
◆ THE BLUEPRINT
GLM Coefficient Interpretation

In OLS, the slope is the change in y per unit x. Always constant. In a GLM, coefficients live on the link scale. To interpret them you must invert the link.

Logistic Regression (Binomial / Logit)
Linear predictor: \(\eta = \beta_0 + \beta_1 x\)
Link: \(\eta = \ln\left(\frac{p}{1-p}\right)\) (log-odds)
Odds: \(\frac{p}{1-p} = e^{\eta}\)
Probability: \(p = \frac{e^{\eta}}{1 + e^{\eta}}\)

Increasing x by 1 adds \(\beta_1\) to \(\eta\). On the odds scale, this multiplies the odds by \(e^{\beta_1}\). That ratio is constant. But the probability change is NOT constant. It depends on where you start on the curve.

Classification Threshold

The model outputs \(\hat{p}\), a probability. To make a binary decision (0 or 1), you choose a threshold \(t\): predict 1 if \(\hat{p} \geq t\), else 0.

Moving the threshold trades off two kinds of errors:

  • Type I (False Positive): predict 1 when truth is 0. Rate = FP / (FP + TN).
  • Type II (False Negative): predict 0 when truth is 1. Rate = FN / (FN + TP).

A lower threshold catches more true positives but also more false positives. A higher threshold is more conservative. There is no free lunch.

Poisson Regression (Poisson / Log)
Linear predictor: \(\eta = \beta_0 + \beta_1 x\)
Link: \(\eta = \ln(\lambda)\) (log of expected count)
Expected count: \(\lambda = e^{\eta}\)

Increasing x by 1 multiplies \(\lambda\) by \(e^{\beta_1}\). This multiplicative effect is constant regardless of x.

The Key Difference from OLS

In OLS the effect of x on y is additive and constant. In logistic and Poisson regression the effect is multiplicative on the natural scale (odds or rate). The nonlinearity of the link means the same coefficient produces different absolute changes depending on where you are on the curve.

Explore More
Regression & Regularization the linear predictor is the same systematic component
RIGHT MODEL —
WRONG MODEL —
Residuals (Right)
Residuals (Wrong)
◆ THE BLUEPRINT
What You're Looking At

The same data, two different models. The right model (left column) fits appropriately. The wrong model (right column) shows systematic issues in the fit and residuals.

Common Mismatches

An identity link on binary data produces predictions outside [0, 1]. The fitted line extends beyond the valid probability range.

An identity link on count data ignores the exponential mean-variance relationship. The linear fit underpredicts large counts and can predict negative counts.

A Poisson/log link on normal data forces the fitted curve to be exponential. When the true relationship is linear, the log link distorts both tails.

A Gamma/inverse model on normal data can fail entirely because Gamma requires strictly positive responses.

Which Family?
Response Family / Link
Binary (0/1) Binomial / Logit
Counts (0, 1, 2, ...) Poisson / Log
Positive continuous Gamma / Inverse
Continuous (any real) Gaussian / Identity
Explore More
Regression & Regularization wrong model ≈ wrong loss function; both bias the fit Bootstrap & CV use cross-validation to compare models
Mystery Data
Response Type Guide
Look For Family / Link
Only 0s and 1s Binomial / Logit
Non-negative integers Poisson / Log
Positive, right-skewed Gamma / Inverse
Positive and negative Gaussian / Identity
◆ THE BLUEPRINT
The Anvil

Each round, a random GLM family generates data. Your job is to identify which one. Study the scatter plot's domain and range. Binary data has only 0s and 1s. Counts are non-negative integers. Gamma data is strictly positive and often right-skewed. Normal data can go negative.

Which Family?

The link function maps the linear predictor to the appropriate domain for the response variable.

Response Family / Link
Binary (0/1) Binomial / Logit
Counts (0, 1, 2, ...) Poisson / Log
Positive continuous Gamma / Inverse
Continuous (any real) Gaussian / Identity
Tips

Binary data is the easiest to spot. Poisson vs Gamma can be tricky. Poisson data is discrete (integers only). Gamma data is continuous. Both are positive.

Gaussian data can include negative values. If you see values below zero, it is almost certainly Gaussian.

Explore the Forge
Bias-Variance Tradeoff model complexity and the bias-variance tradeoff Regression & Regularization loss functions and coefficient paths Bootstrap & CV resampling methods and model selection