Hypothesis tests

Method

• Seven tests on one page: the “Test” switch shows one of them. Each keeps its own data, fields and results; the settings file saves all of them and the chosen one.
• Conclusions: the significance level is α = 1 − confidence level (95% gives α = 0.05). When p < α the conclusion states the difference found, with its size and interval. Otherwise it says there is not enough evidence: a large p-value never shows that means or variances are equal, only that the data cannot tell them apart; the interval shows the values the data cannot rule out. p-values below 0.0001 are shown as “< 0.0001” (the CSV has the full value).
• 1-sample t (Minitab’s defaults): t = (x̄ − μ₀) / (s / √n) with n − 1 degrees of freedom; p from the t distribution for ≠ (two-sided), < or >. Interval x̄ ± t₁₋α/₂ · s/√n; for < the upper bound x̄ + t₁₋α · s/√n, for > the lower bound, as Minitab.
• 2-sample t: both samples in one column with a Sample column (exactly two names; sample 1 is the first to appear). Difference = mean₁ − mean₂. Default as Minitab: equal variances not assumed, SE = √(s₁²/n₁ + s₂²/n₂) with the Welch–Satterthwaite degrees of freedom (s₁²/n₁ + s₂²/n₂)² / ((s₁²/n₁)²/(n₁ − 1) + (s₂²/n₂)²/(n₂ − 1)), truncated to a whole number before p and the interval are computed, as Minitab’s help describes (the untruncated value is shown in the table; scipy and R use it as it is, so their p-values differ slightly). “Assume equal variances”: pooled s_p = √(((n₁ − 1)s₁² + (n₂ − 1)s₂²) / (n₁ + n₂ − 2)), SE = s_p √(1/n₁ + 1/n₂), n₁ + n₂ − 2 degrees of freedom.
• Paired t: two columns, one row per pair; the 1-sample t on the differences d = First − Second against the hypothesized difference (default 0). The histogram shows the differences with the interval of their mean and H₀, as Minitab’s paired t graph.
• One-way ANOVA (equal variances assumed, as Minitab’s default): SS factor = Σ nᵢ (x̄ᵢ − x̄)², SS error = Σ Σ (x − x̄ᵢ)², F = MS factor / MS error with k − 1 and N − k degrees of freedom. S = √MS error (pooled StDev); R-sq = SS factor / SS total; R-sq(adj) = 1 − MS error / (SS total / (N − 1)); R-sq(pred) = 1 − PRESS / SS total with PRESS = Σ (x − x̄ᵢ)² / (1 − 1/nᵢ)² (not defined when a group has one value). Means with x̄ᵢ ± t₁₋α/₂(N − k) · S/√nᵢ. Welch’s ANOVA and Games-Howell comparisons for unequal variances are not included.
• Tukey comparisons (Tukey–Kramer for unequal sizes, as Minitab): for each pair, difference x̄ⱼ − x̄ᵢ, SE = S √(1/nᵢ + 1/nⱼ), simultaneous interval difference ± q₁₋α(k, N − k)/√2 · SE, T = difference / SE and adjusted p = P(Q > |T| √2), where Q has the studentized range distribution. Its CDF is computed here by numerical integration (P(Q ≤ q) = ∫ f(s) P(R ≤ q s) ds with the range distribution of k normal values, Gauss–Legendre quadrature) and checked against scipy.stats.studentized_range to 1e-7 (the quantile to 1e-6). Grouping letters by Piepho’s insert-and-absorb method: means that do not share a letter differ; letters are named in the order of the largest mean.
• Chi-square test of independence: a two-way table of counts (row labels, then one count per column) or long format (Row, Column and an optional Count; repeated combinations add up, missing ones are 0). Expected count = row total × column total / N; Pearson χ² = Σ (O − E)²/E; likelihood ratio G² = 2 Σ O ln(O/E); (r − 1)(c − 1) degrees of freedom; no continuity correction. Counts must be whole numbers; a row or column with a total of 0 is refused. Small expected counts: Cochran’s rule (every expected count at least 1 and at most 20% of them below 5) is checked and a warning given when it fails. For a 2 × 2 table Fisher’s exact two-sided p is shown (tables with the same margins and a probability no larger than the observed one, as scipy.stats.fisher_exact and R’s fisher.test); when Cochran’s rule fails for a 2 × 2 table the conclusion uses Fisher’s p.
• F test for two variances (normal data): F = (s₁²/s₂²) / ρ₀ with n₁ − 1 and n₂ − 1 degrees of freedom; two-sided p = 2 · min(P(F ≤ f), P(F ≥ f)). Interval for σ₁²/σ₂²: (s₁²/s₂²) / F₁₋α/₂ to (s₁²/s₂²) / Fα/₂ (one-sided bounds with α); the StDev ratio interval is its square root. Minitab’s current 2 Variances reports Bonett’s and Levene’s methods by default and this F test only with “Use test and confidence intervals based on normal distribution”. The F test is sensitive to non-normal data.
• Levene’s test for 2 or more groups in the Brown–Forsythe form that Minitab’s Test for Equal Variances uses: the one-way ANOVA F of the absolute deviations from the group medians, k − 1 and N − k degrees of freedom. Bartlett’s test (normal data): ((N − k) ln s_p² − Σ (nᵢ − 1) ln sᵢ²) / (1 + (Σ 1/(nᵢ − 1) − 1/(N − k)) / (3(k − 1))), chi-square with k − 1 degrees of freedom. The intervals for σ are individual chi-square intervals of each group, s √((n − 1)/χ²); they are not Minitab’s simultaneous (Bonferroni or multiple-comparison) intervals.
• Assumptions: the t-tests, ANOVA, the F test and Bartlett’s test assume independent values from normal distributions (the t-tests and ANOVA are robust to moderate non-normality with 15 or more values per group); the chi-square test assumes independent items. None of the tests tells whether a difference is large enough to matter: judge that from the interval.
• Checks: every statistic, p-value and interval is compared with an independent Python script (scipy.stats ttest_1samp, ttest_ind with equal_var True and False, ttest_rel, f_oneway, chi2_contingency without correction and with lambda_ “log-likelihood”, fisher_exact, levene with center “median”, bartlett, the t, F, chi-square and studentized range distributions, and statsmodels pairwise_tukeyhsd) for the examples and edge cases: one-sided alternatives, other confidence levels, unequal sizes, two-value samples, a group of one value, small expected counts. The distribution functions (chi-square, t and F quantiles, studentized range) are written here without libraries.
• Examples: synthetic data, not from a real plant: tightening torque against a 10 N·m target, wall thickness from two suppliers, position error of boards in an old and a new fixture, cycle times of four machines, defects by type and shift, bore deviations of an old and a new spindle, fill weights of four filling heads.
• Files: CSV, TSV and TXT are read as if pasted (UTF-8, UTF-16, otherwise Windows-1257). XLSX: choose the sheet and the columns or a cell range; formula cells use the value Excel last saved, dates are kept as text. Files up to 20 MB are read in this browser only and never uploaded.
• XLSX files are read with SheetJS Community Edition 0.20.3, Apache License 2.0 (license), loaded only when an Excel file is opened or created.