Why an F ratio instead of repeated t-tests

With three or more groups, comparing every pair with a separate t-test inflates the chance of a false positive — each test carries its own 5% risk — and produces overlapping results nobody can summarise. One-way analysis of variance replaces all of that with one question: is the variation between group means larger than the variation within them? The answer is the F ratio; both parts are variances, so F carries no units and no sign, and F near 1 means the means are spread out by about as much as the within-group noise predicts. Each participant here contributes one number to one group.

The decomposition

The total variation splits into two sums of squared deviations. SS_between = Σ nᵢ (x̄ᵢ − x̄_grand)² measures each group mean's distance from the grand mean, weighted by group size, while SS_within = Σ Σ (x − x̄ᵢ)² measures every value's distance from its own group mean — the noise, untouched by any differences between the means. Each sum is divided by its degrees of freedom, df_between = k − 1 and df_within = N − k, to give mean squares, and then F = MS_between / MS_within. The p-value is the upper tail of the F distribution with (k − 1, N − k) df, since only large values count against the null of equal population means. The effect size is η² = SS_between / (SS_between + SS_within).

Working through the default data

The page loads three groups of six: Control 5, 6, 4, 7, 5, 6; Low dose 7, 8, 6, 9, 7, 8; High dose 10, 9, 11, 8, 10, 12. Their means are 5.500, 7.500 and 10.000, and the grand mean is 138 ÷ 18 = 7.667. Weighting each squared deviation by group size gives SS_between = 6(4.694) + 6(0.028) + 6(5.444) = 61.0; each value's distance from its own group mean gives SS_within = 5.5 + 5.5 + 10.0 = 21.0. With 2 and 15 degrees of freedom the mean squares are 30.5 and 1.400, so F = 21.786 and p is about 0.000037, displayed as < 0.001, with η² = 61 / 82 = 0.744. The within-group SDs span 1.049 to 1.414, a ratio of about 1.35 — inside the "under about 2" guidance the page mentions.

Where the F test stops

A significant F does not identify the interesting group. It says at least one mean differs from another, but not which pairs, how many, or in which direction; for that you need a post-hoc procedure such as Tukey's HSD. Substituting unadjusted pairwise t-tests is a common shortcut and is wrong in a measurable way: with three groups there are 3 possible pairs, so the family-wise error rate becomes 1 − 0.95³ ≈ 14% rather than 5%, and with four groups 6 pairs push it to about 26%. Bonferroni is a blunt but honest repair, and no post-hoc follow-up of any kind is performed here.

Three further gaps are worth stating. Independence is assumed, so listing one person's three scores as three groups, or spreading matched participants across groups, invalidates the p. Normality of the residuals is assumed — the leftovers after removing group means, not the raw values — with no normality test run. Similar group variances are assumed, with no Levene's test, no Brown–Forsythe test and no Welch-corrected ANOVA here. η² is also biased upward, overstating the population share of variance; ω² is the less-biased alternative. And a significant F is a statement about data under a model, not a verdict about reality: with a large N, groups that differ trivially still produce a small p.

Questions and answers

What does η² of 0.744 mean?

That about 74% of the total variation lies between groups rather than within — a large effect by convention.

My result is not significant. Have I shown the groups are the same?

No — the data are merely compatible with equal means, which is weaker, since small samples leave real differences undetectable.

Two groups only — is ANOVA the right tool?

It gives the same p as a pooled-variance t-test, since F equals t², but the t-test states the direction directly.

Can I enter summary statistics instead of raw values?

No — the sums of squares come from the individual values, so means and SDs alone are not enough.