Regression

Method

• Data: one row per observation in long format, the response Y and up to 8 predictors X1 … X8 (X1 required). Every row needs a number in each chosen column. Columns are found by their header names (“X1”, “X 2”, “Predictor 3”, “Y”, “Response”; a plain “X” is X1); otherwise the number columns from the left are X1, X2 … and the last one is Y. Unused predictor roles are set to “(none)”.
• Models: with one predictor, as Minitab’s Fitted Line Plot, a linear (Y = b0 + b1 X), quadratic (+ b2 X²) or cubic (+ b3 X³) model; a quadratic needs at least 3 and a cubic at least 4 different X values. With several predictors, as Minitab’s Regression with main effects only: Y = b0 + b1 X1 + … + bk Xk (no squares or interactions). A model with p coefficients needs at least p + 1 observations.
• Least squares by QR decomposition: the predictors are standardized (z = (x − mean) / SD) and the model is fitted by Householder QR of the design matrix of z (the powers of z for a polynomial), never by inverting XᵀX. The coefficients and their covariance matrix are then transformed exactly (a linear change of basis) to the reported coding. A column whose part not explained by the columns before it is below 10⁻⁷ of its length is refused as collinear, with its name.
• Coding: “Uncoded” reports the coefficients of the raw predictors (X, X², X³), as Minitab’s fitted line plot. “Coded” reports them for the predictors minus their means (Minitab’s “Standardize continuous predictors: subtract the mean”), which lowers the VIF of polynomial terms and changes the tests of the lower-order terms; the regression equation is always given in raw units. The fit, S, R², the ANOVA of the regression, the residuals and the predictions are the same in both codings.
• Coefficients: SE Coef = √(MSE · cⱼⱼ) with C = (XᵀX)⁻¹ of the reported coding, T = Coef / SE Coef, two-sided P from the t distribution with n − p degrees of freedom. VIF = cⱼⱼ · Σ (xᵢⱼ − x̄ⱼ)² = 1 / (1 − Rⱼ²), shown when the model has two or more terms; above 10 it gets a warning.
• Model summary: S = √MSE, MSE = SSE / (n − p); R-sq = 1 − SSE / SST; R-sq(adj) = 1 − MSE / (SST / (n − 1)); R-sq(pred) = 1 − PRESS / SST with PRESS = Σ (eᵢ / (1 − hᵢ))² (the residual of each observation when it is left out; not defined when a leverage is 1). Durbin-Watson (optional, for rows in time order) = Σ (eᵢ − eᵢ₋₁)² / Σ eᵢ².
• Analysis of variance: Regression SS = SST − SSE with p − 1 degrees of freedom and F = MSR / MSE. Each term has one degree of freedom: sequential SS (the default, as the fitted line plot: each term added to the terms above it, so Linear then Quadratic then Cubic, or the predictors in the order X1, X2 …; they add up to the regression SS) or adjusted SS (each term entered last, Minitab’s Regression default; equal to T² · MSE). Both F-values use the MSE of the full model, as Minitab’s ANOVA table. Lack of fit needs replicates (rows with the same predictor values): pure error SS = Σ (y − mean of its setting)² with n − m degrees of freedom (m distinct settings), lack-of-fit SS = SSE − pure error SS with m − p degrees of freedom, F = MS lack of fit / MS pure error. A significant lack of fit means the model misses the shape of the data.
• Fits and diagnostics: leverage hᵢ = xᵢᵀ (XᵀX)⁻¹ xᵢ, standardized residual eᵢ / (S √(1 − hᵢ)), Cook’s distance = (standardized residual² / p) · hᵢ / (1 − hᵢ). Unusual observations as Minitab: R when |standardized residual| > 2, X when the leverage is above 3p/n or 0.99, whichever is smaller.
• Prediction: fit ŷ₀, SE Fit = S √(x₀ᵀ (XᵀX)⁻¹ x₀), confidence interval for the mean ŷ₀ ± t₁₋α/₂(n − p) · SE Fit and prediction interval for a new observation ŷ₀ ± t₁₋α/₂(n − p) · √(S² + SE Fit²), as Minitab’s Predict (confidence level default 95%). A point outside the range of a predictor is marked “Outside the X range”; with several predictors a point inside every range but with a leverage above the largest leverage of the data is marked “Beyond the data” (hidden extrapolation, after Montgomery, Peck & Vining; FactoKit’s own mark, not Minitab’s). The fitted line plot draws the same intervals over the range of X.
• Residual plots (“four in one”): normal probability plot (residuals against the normal percentiles of the median ranks (i − 0.3) / (n + 0.4), with the normal line of the residuals’ mean and standard deviation), residuals versus fits, histogram (the bins of the Cpk tool) and residuals versus observation order. Regular residuals by default, as the fitted line plot; standardized residuals on request.
• Summary sentence: α = 1 − confidence level. For one predictor it reads the highest-order term (is the curve needed?), for several predictors each predictor given the others; then the lack-of-fit test and the unusual observations. Warnings: perfect fit, fewer than 5 error degrees of freedom, VIF above 10, R-sq(pred) more than 15 percentage points below R-sq (a rule of thumb for overfitting), extrapolated predictions.
• Checks: every number is compared with an independent Python script using statsmodels OLS (params, bse, tvalues, pvalues, R², adjusted R², anova_lm with sequential and adjusted sums of squares, get_prediction for the intervals, OLSInfluence for leverages, standardized residuals and Cook’s distance, variance_inflation_factor, durbin_watson), lack of fit against the saturated model, and PRESS by refitting without each observation. Published check: the Longley data of the NIST Statistical Reference Datasets (6 predictors, highly collinear); the NIST certified coefficients, standard errors, residual SD, R² and sums of squares are reproduced (the reference script solves the data in exact rational arithmetic and agrees with the certified values to all 15 printed digits); this tool matches them to at least 12 significant digits (in practice about 13).
• Example: synthetic data, not from a real oven: the peak temperature of a reflow test board at nine conveyor speeds, two boards per speed and three at 0.8 and 1.2 m/min. The linear model shows a significant lack of fit; the quadratic model fits.
• Files: CSV, TSV and TXT are read as if pasted (UTF-8, UTF-16, otherwise Windows-1257). XLSX: choose the sheet and the columns or a cell range; formula cells use the value Excel last saved, dates are kept as text. Files up to 20 MB are read in this browser only and never uploaded.
• XLSX files are read with SheetJS Community Edition 0.20.3, Apache License 2.0 (license text at the end of this file).