AI Safety Evaluation Calculator

They're loosely inspired by the NIST AI Risk Management Framework's general emphasis, not a formula pulled directly from the NIST document -- harmful content (25%) and bias plus hallucination (20% each) are weighted most heavily as higher-stakes risk categories, while robustness and transparency (10% each) count for less in the composite. Treat these as a reasonable general-purpose starting weighting for prioritizing remediation effort, not an official or universally mandated scoring standard.

No -- it only combines scores you provide yourself, typically from your own red-team testing, benchmark evaluation, or internal audit process. The calculator's role is to weight and combine those existing scores into one composite figure, a risk classification, and a rough test-coverage estimate; it has no way to independently verify the six input scores are accurate, so the output is only as trustworthy as the evaluation process that produced your inputs.

It's a statistical estimate of how likely your stated number of test cases is to have surfaced the residual defect rate implied by your composite safety score, using a simple probability-of-detection model. It is NOT a measurement of what your test suite actually found -- it's a theoretical estimate of test-suite adequacy given your safety score, so a low coverage number is a signal to run more tests, not a report of results from tests you already ran.

A high self-reported safety score backed by only a handful of test cases is less trustworthy than the same score backed by thousands of test cases, because a small test suite has a real chance of simply missing failure modes that exist. By blending the composite safety score (70% weight) with the test-coverage estimate (30% weight), deployment readiness rewards both a genuinely safe model AND enough testing to have real confidence in that safety assessment.

It sums the gap between each of the six dimension scores and a perfect 100, then applies a rough rule-of-thumb multiplier of about half an engineering hour per gap point. This is a coarse planning estimate meant to give a rough sense of scale for prioritizing remediation work, not a real engineering estimate -- actual time to close a safety gap depends heavily on what's actually causing it (a data problem, a model architecture limitation, or a guardrail engineering task all take very different amounts of effort for the same point improvement).