Model Performance Calculator

Because accuracy alone can be badly misleading, especially when the classes are imbalanced. A spam filter that never flags anything as spam is highly "accurate" on a dataset where most email isn't spam, but useless at its actual job. Precision tells you how trustworthy a positive prediction is (of everything flagged positive, how much really was), while recall tells you how complete the model's positive predictions are (of everything actually positive, how much got caught) -- reporting them separately exposes tradeoffs that a single accuracy number hides entirely.

No -- it's estimated from RMSE using the relationship MAE = RMSE x sqrt(2/pi), which holds exactly only when prediction errors are normally distributed. This calculator doesn't take individual prediction errors as input, only a summary MSE and variance, so it can't measure MAE directly; it derives an estimate under that normality assumption instead. If your model's actual errors are skewed or heavy-tailed rather than normally distributed, the real MAE could differ meaningfully from this estimate.

No -- it plots a smooth, parametric curve shaped to match the AUC value you enter, using a simplified mathematical model rather than your classifier's real threshold-by-threshold true/false positive rates. It's useful for visualizing roughly what shape a given AUC typically corresponds to and for exploring how the optimal threshold and lift change with AUC, but it is not a substitute for computing an actual ROC curve from your model's real predictions and true labels.

It runs an approximate two-proportion statistical significance test on the accuracy difference between the two models, using a normal-approximation p-value based on the pooled accuracy and test set size, then checks it against the conventional p < 0.05 threshold. This test assumes the two accuracies are independent samples; if you actually evaluated both models on the SAME test set (the usual setup), the correct test is a paired one (like McNemar's test, which needs the count of examples the two models disagreed on) and would typically be more sensitive than this approximation, not less -- so a "not significant" result here can be a false negative for a genuinely different, same-test-set comparison. It also separately reports an efficiency comparison -- accuracy relative to parameter count -- because a model that's not statistically significantly more accurate but is far smaller may still be the better practical choice, which is why the recommendation logic weighs both significance and efficiency, not accuracy alone.

Because a confusion matrix -- counts of true positives, true negatives, false positives, and false negatives -- is a complete, compact summary of a binary classifier's performance at one decision threshold; every metric in this mode (accuracy, precision, recall, F1, MCC, specificity) is derivable from those four counts alone. You get those counts by comparing your model's predictions to the true labels on a test set and counting how each prediction falls into the four categories, which most ML libraries compute for you directly.