Home / Supervised ML / Log-Linear Regression

Supervised ML · Chapter 2 · 18 min read

Log-Linear Regression: When Percentages Matter More Than Units

Sometimes the relationship between two variables isn't about absolute changes at all — it's about proportional ones. Does a 10% increase in advertising lead to a 5% increase in sales? Does doubling city size double a city's economic output? These are questions about elasticities, and the log-linear model is built to answer them. This page explains why and how.

Why Log-Transform? The Intuition

Standard OLS assumes that a one-unit change in $X$ always gives the same change in $Y$ — whether $X$ goes from 1 to 2 or from 1,000 to 1,001. But many real relationships don't work that way. Income grows exponentially. Company revenues span orders of magnitude. A doubling of firm size tends to produce more output regardless of whether you start at 10 or 1,000 employees.

The fix is elegant: take the natural logarithm of both variables. The key logarithm rule that makes this work:

And the most important fact for econometrics: for small changes, the difference in log values approximates the percentage change. Formally, if $Y$ changes by a small amount $\Delta Y$ (where $\Delta$ is the Greek letter "delta" and always means change in):

This means $d\ln(Y) = \frac{dY}{Y}$ — where $d$ (the differential operator) represents an infinitesimally small change, the limiting version of $\Delta$ as the change becomes tiny. In other words, the differential of $\ln(Y)$ equals the relative change in $Y$, i.e. a percentage change (without needing to multiply by 100). This is the key insight that drives everything below.

The Log-Linear (Log-Log) Model

Starting from a non-linear relationship $Y = e^{\beta_0} \cdot X^{\beta_1} \cdot e^u$ (a power-law or multiplicative relationship), taking the natural log of both sides:

This is the log-log model (also called log-linear). It looks exactly like OLS — because it is OLS, just applied to the log-transformed variables. So we estimate it exactly the same way, using the transformed data $\ln(Y_i)$ and $\ln(X_i)$.

The crucial interpretive difference from standard OLS: $\hat{\beta}_1$ is now an elasticity. To see why, note that:

Here, $\partial$ (the "curly d") is the partial derivative symbol — it means "the rate of change of the numerator with respect to the denominator, holding everything else fixed." The middle term $\frac{\partial Y/Y}{\partial X/X}$ is a ratio of relative changes, which is the definition of elasticity. So $\hat{\beta}_1$ tells you: if $X$ increases by 1%, $Y$ changes by $\hat{\beta}_1$ percent, ceteris paribus (Latin for "all else equal"). This interpretation holds regardless of the units of $X$ or $Y$ — which is a major advantage when comparing across datasets or countries.

Predicting Percentage Change in Y

Using the log-log model to predict how $Y$ responds to a percentage change in $X$ is straightforward:

Example: if $\hat{\beta}_1 = 0.94$ and $X$ increases by 5% ($\Delta X / X = 5\%$), then $Y$ is expected to increase by $0.94 \times 5 = 4.7\%$.

Predicting the Absolute Value of Y

What if you want to predict an actual value of $Y$ (not just a percentage change) for a given value of $X$? You need two steps:

Remember: the model gives you $\widehat{\ln(Y)}$ — you must exponentiate it to get back to the original scale. Forgetting this step is the most common mistake with log models. Also recall that $e^{\ln(Y)} = Y$, i.e., the exponential and logarithm are inverses of each other.

The Differential and Elasticity — A Deeper Look

In calculus, the differential $df(x)$ of a function $f(x)$ represents how the function changes in response to an infinitesimally small change $dx$ in the input (think of $dx$ as $\Delta x$ shrunk to be infinitely tiny):

For the logarithm specifically: $\frac{d\ln(Y)}{dY} = \frac{1}{Y}$, so $d\ln(Y) = \frac{dY}{Y}$ — the differential of the log equals the relative (percentage) change in $Y$. This is exactly why the elasticity interpretation works.

In the log-log model, $\hat{\beta}_1 = \frac{d\ln(Y)}{d\ln(X)} = \frac{dY/Y}{dX/X}$ is the ratio of relative changes — the elasticity. No multiplication by 100 needed because the differential of the log already gives you the percentage change directly.

OLS Assumptions in the Log-Log Model

Since the log-log model is OLS on transformed data, the same four assumptions apply (see the OLS Regression page for details): homoskedasticity, no autocorrelation, normality of residuals, and no multicollinearity. These now apply to the log-scale residuals $\hat{u}_i = \ln(Y_i) - \widehat{\ln(Y_i)}$.

An important implication: the log transformation often fixes heteroskedasticity that exists in the raw data. If the raw residuals fan out (bigger values have larger errors), the log-scale residuals are often much more uniform — because taking logs compresses large values relative to small ones.