Wald Tests: From Standard Error to Test

Divide the estimate by its standard error. That ratio is the whole test.

The shortest hypothesis test in statistics

The previous page priced the uncertainty of the heart-rate slope: $\hat\beta_1 = -1.271$ with standard error $0.2915$. Now comes the question those two numbers were made for: could the true slope be zero? Could exercise have no relationship with resting heart rate at all, with the $-1.271$ just sampling noise?

The Wald test answers by measuring the distance from the estimate to the hypothesised value in standard-error units:

$$z \;=\; \frac{\hat\beta - \beta^{H_0}}{\text{SE}(\hat\beta)} \qquad\text{usually with } \beta^{H_0} = 0$$

Here $z = -1.271 / 0.2915 = -4.36$: the estimate sits 4.36 standard errors below zero. If the true slope really were zero, the sampling distribution from the curvature argument says estimates this extreme turn up with probability $p \approx 1.3 \times 10^{-5}$ — about once in 77,000 studies. Either we have witnessed a small miracle, or the true slope is not zero.

Why zero? $\beta = 0$ means the predictor drops out of the model entirely — it is the sceptic's default, “no effect”. That is why it is the hypothesis every summary() table tests by default. But the formula accepts any $\beta^{H_0}$, and the second interactive below exploits that.

Interactive: the rejection picture

Under the null hypothesis, $z$ follows a standard normal distribution — the curve below. The shaded tails are the rejection region: values so extreme that, were the null true, they would occur with probability less than the chosen level. Pick a real test from the buttons and see where its needle lands.

TEST STATISTIC z
CRITICAL VALUE
TWO-SIDED p
VERDICT

Interactive: slide the null — and find the confidence interval

Zero is not special to the machinery. Drag the hypothesised value $\beta^{H_0}$ along the axis and watch the test re-run against each candidate. Some nulls are rejected (red); others survive (green). The survivors form a familiar object.

NULL VALUE βH₀
z AGAINST THIS NULL
TWO-SIDED p
VERDICT

The duality. The green stretch of surviving nulls is the confidence interval: a $95\%$ CI is exactly the set of hypothesised values a $5\%$-level Wald test cannot reject. Interval and test are two costumes on one calculation — which is why the interval $\hat\beta \pm 1.96\,\text{SE}$ from the curvature page reappears here without new mathematics.

Every summary() row is a Wald test

You have been reading Wald tests since Tutorial 1. Each row of a fitted model's coefficient table performs one: estimate, standard error, their ratio, and the tail probability — highlighted below in the actual outputs for this section's two datasets.

# Gaussian: resting heart rate ~ exercise hours (n = 60)
summary(lm(y ~ x, data = hr))

# Logistic: treatment response ~ biomarker (n = 30)
summary(glm(y ~ x, data = trial, family = binomial))

# lm coefficient table:
#             Estimate Std. Error t value Pr(>|t|)
# (Intercept)  71.7714     1.7521 40.9634   <2e-16
# x            -1.2711     0.2965 -4.2867 6.93e-05

# glm coefficient table:
#             Estimate Std. Error z value Pr(>|z|)
# (Intercept)   0.9059     0.6618  1.3690   0.1710
# x             2.4329     0.8625  2.8209   0.0048
import statsmodels.api as sm

# Gaussian (n = 60)
print(sm.OLS(y, sm.add_constant(x)).fit().summary().tables[1])

# Logistic (n = 30)
print(sm.Logit(resp, sm.add_constant(bio)).fit().summary().tables[1])

# OLS:      coef   std err        t     P>|t|
# const  71.7714     1.752   40.963     0.000
# x1     -1.2711     0.296   -4.287     0.000

# Logit:    coef   std err        z     P>|z|
# const   0.9059     0.662    1.369     0.171
# x1      2.4329     0.862    2.821     0.005

t or z? The logistic table reports exactly this page's $z$. The Gaussian table refines it: lm uses the df-corrected standard error ($0.2965$ rather than the ML $0.2915$ — the small print from the curvature page) and compares against a Student-$t$ with $n-2$ degrees of freedom, which fattens the tails a little at small $n$. Same idea, extra honesty about having estimated $\sigma$ too; by $n = 60$ the difference is already in the third decimal place.

When Wald wobbles

The Wald test inherits the curvature route's one assumption: that the peak is well described by its local quadratic. Where that fails, the test fails with it:

Boundaries. On the Bayesian page, the Wald interval for 2 events in 20 patients dipped below zero — the same defect makes Wald tests unreliable for parameters near their limits.

The Hauck–Donner effect. In logistic regression, as a predictor approaches perfect separation, its estimated effect grows — but its standard error grows faster. The ratio $z$ then shrinks as the evidence strengthens, and the Wald test can fail to reject precisely when the effect is overwhelming. R's glm will happily print the misleading row.

The sturdier alternative uses no local approximation at all: compare the heights of two peaks on the actual likelihood surface. That is the likelihood-ratio test, next.

On the sampling branch, the counterpart summary is the credible interval read directly from the walked posterior — see Bayesian uncertainty.