Inter-Rater Reliability Calculator

Not at the calculator's default values, though it isn't inert in general. Rater 2 Positive Rate defaults to exactly 50%, and 0.5 × x + 0.5 × (1 − x) always equals 0.5 regardless of x (lines 14-16), so Expected Agreement — and therefore Kappa — doesn't move when Rater 1 Positive Rate changes only because Rater 2's rate happens to sit at that exact midpoint.

Yes. Expected Agreement is rater1PositivePct × rater2PositivePct plus their complements' product (lines 14-16); that expression only collapses to a constant when rater2PositivePct is exactly 0.5. At any other Rater 2 Positive Rate, Rater 1 Positive Rate does change Expected Agreement and, through it, Kappa.

The engine labels it "Moderate" agreement (line 28, kappa ≥ 0.41) — meaningfully better than chance-level agreement, but short of the "Substantial" (≥0.61) or "Almost Perfect" (≥0.81) bands used for higher scores under the standard Landis-Koch interpretation scale. Kappa itself comes from Jacob Cohen's 1960 paper "A Coefficient of Agreement for Nominal Scales" (Educational and Psychological Measurement), which defined it specifically to correct raw percent agreement for the agreement two raters would rack up by chance alone.

Exactly two raters and a simple positive/negative classification — the formulas for Observed Agreement, Expected Agreement, and Kappa (lines 10-21) are all built around that two-rater, two-category structure and don't generalize to comparisons among three or more raters or to ordinal/multi-category rating scales.