Home / Supervised ML / Dummy Variables

Supervised ML · Chapter 3 · 18 min read

Dummy Variables: Teaching Regression to See Categories

OLS regression is naturally built for numbers. But the real world is full of categories: gender, education level, country, job type. How do you include "has a university degree" or "lives in Italy" in a regression? You turn it into a number — specifically a 0 or 1. That's a dummy variable. Simple in concept, powerful in practice.

The Big Picture: What Problem Are We Solving?

Imagine you're studying wages and you want to control for whether someone attended university. You can't just write "university" in a regression — OLS needs numbers. And you can't assign 0 = no university, 1 = university and treat it like a continuous variable, because there's no meaningful "one unit of university." It's a category, not a scale.

The solution: create a binary indicator variable — a dummy variable $D$ — that equals 1 if the condition is true and 0 if it isn't. The coefficient on $D$ then measures the difference in the outcome between the two groups, controlling for everything else in the model. That's exactly the question you wanted to answer.

One Dummy Variable: The Binary Case

Suppose you're regressing wage $Y$ on years of experience $X$ and a dummy $D$ (1 = university graduate, 0 = non-graduate). The model is:

where $Y_i$ is the wage for individual $i$, $D_i \in \{0, 1\}$ is the dummy variable (1 = university graduate), $X_i$ is years of experience, $u_i$ is the error term, and $\beta_0$, $\beta_1$, $\beta_2$ are the coefficients to be estimated.

To understand what $\beta_1$ means, let's look at what the model predicts for each group separately:

The difference between the two lines is exactly $\beta_1$. The non-graduate group is the baseline (intercept = $\beta_0$). The graduate group has a shifted intercept ($\beta_0 + \beta_1$) but the same slope $\beta_2$ on experience. So:

The phrase ceteris paribus is Latin for "all else equal" — it means we're isolating the group difference by holding experience fixed. If $\hat{\beta}_1 = 4.2$, graduates earn on average €4.20/hr more than non-graduates with the same experience. The significance of this difference is tested with a standard t-test on $\hat{\beta}_1$, with $H_0: \beta_1 = 0$ (no difference between groups).

Multiple Dummies — Combining Groups

You can include more than one dummy variable in a regression. For instance, if you also want to control for gender (dummy $D_{\text{female}}$) alongside the education dummy $D_{\text{grad}}$:

The baseline group is now: non-graduate, male (both dummies = 0). Each coefficient measures the shift relative to that baseline:

You can include as many binary dummies as you like. Each one asks the same question: how much does belonging to this group shift the expected outcome, holding everything else fixed?

Categorical Variables: More Than Two Groups

So far the dummies were binary. But what about a variable with more than two categories — like education level: dropout, high school, or college degree? Here you need to be careful.

Why not just code dropout = 0, high school = 1, college = 2 and use it as a regular numeric variable? Because that would force OLS to assume that the gap between dropout and high school equals the gap between high school and college — a very specific and often wrong assumption.

The correct approach: create one dummy for each category, except one. If there are $K$ categories, you include $K - 1$ dummies. In the education example with $K = 3$ groups:

The omitted category (dropout, where $D_1 = D_2 = 0$) is the baseline. The model becomes:

Now $\beta_1$ = wage premium for having a high school degree vs. being a dropout (for the same experience), and $\beta_2$ = wage premium for having a college degree vs. being a dropout. The gap between high school and college is then $\beta_2 - \beta_1$ — and unlike the naive coding, this gap can be different from the gap between dropout and high school. That's the flexibility dummy variables provide.

The Dummy Variable Trap

If you include all $K$ dummies — one per category, including the baseline — you create perfect multicollinearity. The three dummies (dropout + high school + college) always sum to exactly 1 for every observation. That means one dummy is a perfect linear combination of the others plus the intercept column. OLS cannot be computed — the design matrix $\mathbf{X}^\top\mathbf{X}$ becomes singular (non-invertible).

This is called the dummy variable trap. The fix is simple: always omit one category. Which one you omit defines the baseline — all other coefficients are interpreted relative to it. The choice of baseline doesn't change the overall fit of the model, only the interpretation of the intercept and slope coefficients.

Estimation — It's Still Just OLS

Estimating a model with dummy variables uses the exact same OLS formulas as always:

where $\mathbf{X}$ is the design matrix, $\mathbf{Y}$ is the vector of outcomes, and $\hat{\boldsymbol{\beta}}$ is the vector of estimated coefficients. The only difference from a model with only continuous predictors: the design matrix $\mathbf{X}$ has columns that contain only 0s and 1s (for the dummies), rather than continuous values. This means many entries in those columns are 0, but the math is identical — OLS doesn't care whether a column is binary or continuous.