Compare Group Means With an ANOVA Python Example

Compare group means with Python using a one-way ANOVA, assumption checks, and follow-up tests.

Three groups have different sample averages. Is that evidence of a real difference, or could ordinary variation explain the gap? A one-way ANOVA with Python gives you a way to compare those averages in one test. We will calculate the result, examine its assumptions, and decide what to investigate next.

ANOVA in Python: build a small experiment

Imagine measuring completion time, in minutes, for people using three independent interface designs. Each person uses only one design. The question is whether the population mean completion times are equal across designs. Here is a reproducible ANOVA Python example using invented measurements:

import numpy as np
from scipy import stats

groups = {
    "A": np.array([8, 9, 6, 7, 10]),
    "B": np.array([10, 12, 9, 11, 13]),
    "C": np.array([14, 15, 13, 16, 12]),
}

for name, values in groups.items():
    print(f"{name}: n={len(values)}, mean={values.mean():.1f}")

result = stats.f_oneway(*groups.values())
print(f"F={result.statistic:.2f}, p={result.pvalue:.4f}")

Install the dependencies with python -m pip install numpy scipy, then run the script. It prints means of 8, 11, and 14 minutes, followed by F = 18.00 and p ≈ 0.0002. The asterisk passes each array as a separate argument. This Python example sends the group arrays to the test in their insertion order. The SciPy documentation for f_oneway describes the inputs and the standard equal-variance test used here.

This Python analysis compares all three means before any pairwise investigation. In this ANOVA Python workflow, the factor is interface design, and its three levels are A, B, and C. Completion time is the numerical response. One-way refers to the single factor, not the number of groups. The observations should represent independent participants selected or assigned in a way that supports the comparison.

What the statistic measures

For an ANOVA computed with Python, the null hypothesis says that all three population means are equal. The alternative says at least one differs. The test compares two estimates of variation: one based on the separation between group means and another based on observations’ spread within their own groups. More precisely, the F statistic is the between-group mean square divided by the within-group mean square; each mean square adjusts its sum of squares by its degrees of freedom. NIST’s ANOVA explanation lays out that calculation.

In this dataset, the group means are far apart compared with the variation within each group, so F is large. Under the null hypothesis and the model assumptions, the p-value is the probability of obtaining an F statistic at least as large as the observed one. It is not the probability that the null hypothesis is true.

Choose a significance threshold before looking at the result. At 0.05, this p-value leads you to reject equal population means. If the p-value were above 0.05, the result would not establish equality; it would mean the data did not provide enough evidence to reject the null at that threshold. Sample size and variability affect the test’s ability to detect a difference.

Check the conditions behind the calculation

Running ANOVA in Python takes one function call; judging whether the result answers your question takes more thought. Check the study design first. Measurements from the same person under all three designs are paired, so an independent-groups test ignores their relationship. A repeated-measures method would be more appropriate. Likewise, observations from people working together may share influences that make them dependent.

For a standard ANOVA run in Python, the errors within each population are assumed to be normally distributed with equal variances. In practice, inspect the observations and residuals: subtract each group’s mean from each value, then look for extreme outliers or strongly asymmetric patterns. A normal probability plot of residuals can reveal substantial departures. With only five observations per group, plots and formal tests have limited power, so use knowledge of how the data were collected as well.

You can also compare group spreads and use Levene’s test as one diagnostic:

for name, values in groups.items():
    print(f"{name}: sample SD={values.std(ddof=1):.2f}")

levene = stats.levene(*groups.values(), center="median")
print(f"Levene p={levene.pvalue:.4f}")

The sample standard deviations here are all about 1.58. Levene’s test examines a null hypothesis of equal population variances. A large p-value is not proof of equal variances, especially with small samples. If the spreads differ substantially, consider Welch’s one-way test: current SciPy versions provide it through stats.f_oneway(*groups.values(), equal_var=False). Independence remains necessary. Investigate data-entry errors and understand outliers before deciding how to handle them.

From an overall result to a comparison

ANOVA in Python tells you whether the data support a difference somewhere among the means. It does not identify the pairs responsible. Looking at the sample averages suggests A and C are far apart, but that observation alone does not account for uncertainty or the number of comparisons.

For all pairwise comparisons under the equal-variance model, you can run Tukey’s test after the overall analysis:

posthoc = stats.tukey_hsd(*groups.values())
print(posthoc)
print(posthoc.confidence_interval())

The output provides pairwise differences, adjusted p-values, and confidence intervals. Check the row and column labels against the order A, B, C; the sign of a difference depends on that order. SciPy’s Tukey documentation explains the adjustment for multiple pairs. If variances are unequal, choose a follow-up method suited to that condition, such as Games–Howell, rather than applying the equal-variance procedure automatically.

A third ANOVA Python result to report is the size of the observed differences. For example, C’s sample mean is six minutes higher than A’s; its confidence interval helps show the range of plausible population differences. Statistical significance and practical importance answer different questions: a small difference can matter in a high-volume process, while a larger one may be irrelevant if implementation costs are high.

Put the result to work

Before sharing an ANOVA in Python, verify the group labels, units, missing values, and independence of observations. Report the group sizes and means alongside F, its degrees of freedom, the p-value, and the chosen threshold. Here those degrees of freedom are 2 and 12. Include relevant assumption checks so another reader can understand the analysis.

Next, use Python to repeat the ANOVA on your own data and inspect the residuals and group spreads. If an overall difference matters to the decision and individual pairs are the question, run an appropriate multiple-comparison procedure and interpret the intervals in the units of the original measurement.