Paste per-model answers plus the truth label. Get the effective independent vote count n_eff. No upload, no server — this runs entirely in your browser.
n_eff = N / (1 + (N−1)·r̄), where r̄ is the mean pairwise correlation of each model's error indicators (wrong = 1). It estimates how many independent voters a panel behaves like. If every model failed on the same items, r̄ → 1 and n_eff → 1 no matter how many models you added. It is not the panel's accuracy; it is how much independent evidence the panel carries.
The trap: adding weak, near-random models can raise n_eff, because random mistakes don't agree. Report n_eff for a competent panel (drop models near chance), or n_eff will flatter the headcount.
A second trap: two error indicators cannot correlate above a ceiling set by their error rates (r̄max = √(p(1−q)/q(1−p))). So when models differ in skill, the raw r̄ understates how much they really depend on each other. The tool also reports a ceiling-corrected n_eff, which divides each pair's correlation by its maximum possible. It usually lands below the raw number, and it is the more honest read.
On HELM openbookqa (multiple_choice_joint, 500 items), 31 models from 11+ families:
r̄ = 0.251, n_eff = 3.63 of 31. Restricting to models with accuracy ≥ 0.40 (26 models):
r̄ = 0.353, n_eff = 2.64 of 26. One best model per family (11): n_eff = 2.95 of 11.
Same regime as the published literature (Nine Judges ≈ 2 of 9; Wavering Oracles 2.58 of 7).
Ceiling-corrected, the competent panel reads r̄ = 0.537, n_eff = 1.80 of 26; raw and corrected
together bracket the truth at roughly 1.8 to 3.4 depending on slice.