A fixed Elo K-factor stops learning at a noise floor you can predict

I simulated 16 players with fixed true strengths and rated them with Elo. With K=32, the error in their ratings stopped shrinking after about 100 games each and stayed at 52 Elo points through game 1,000. A back-of-envelope formula, error ≈ 9.3·√K, predicts 53. Refitting all the games at once (a Bradley–Terry fit) kept improving the whole way, down to 13 points. After 1,000 games each, K=32 still put only 69% of neighbouring players in the right order, against 88% for the refit.

The test

Each trial draws 16 true ratings from a normal distribution with a standard deviation of 200 Elo points, then plays 8,000 games between random pairs, about 1,000 per player. The stronger player wins with the usual Elo logistic probability. Everyone starts at 0. I ran 20,000 trials per setting, and every setting saw the same players and the same coin flips. Error is the root-mean-square gap between the ratings and the truth, both centred on zero; standard errors are under 0.2 points.

Large K learns fast and then stalls high; small K learns slowly and ends low. K=4 is the worst setting in the table for the first 200 or so games and the best fixed K by game 1,000. The batch fit beats every fixed K at every checkpoint.

Neighbouring players were about 50 points apart on average, so a 52-point error scrambles them. The share of adjacent pairs in the right order after 1,000 games per player: K=64 65%, K=32 69%, K=16 74%, K=4 83%, batch fit 88%. Those numbers had already stopped moving by game 400 for K=16 and above.

Why the floor is √K

After each game, Elo moves a rating by K times (score − expected score). That kick is random, with variance K²·p(1−p) where p is the win chance. Its average is a pull back toward the truth of about K·(ln 10/400)·p(1−p) times the current error. The rating settles where the kicks and the pull balance, at a variance of K²·p(1−p) divided by twice the pull. The p(1−p) cancels, leaving 400K/(2 ln 10), or about 87K: a standard deviation of 9.3·√K whoever the opponents are.

K=64 overshoots the prediction by 4%. The derivation assumes the rating moves little per game, and at K=64 a single game can move it by 30 points or more.

A shrinking K nearly matches the batch fit

Letting each player's K fall with the games they have played removes the floor. I first tried K = 400/(games + 10). It ordered players almost as well as the batch fit (86% of adjacent pairs) but its error was still 48. The ratings had come out squashed: a regression of rating on truth gave a slope of 0.77, because the steps had shrunk before the ratings had spread out from zero. Order doesn't care about scale; error does.

The best step for this kind of running estimate is one over the information gathered so far. For Elo that works out to K ≈ 400 / (ln 10 · p(1−p) · games), and with p(1−p) around 0.2 here that is about 870/games. K = 870/(games + 20) gave a slope of 0.91, an error of 26 and 87% of adjacent pairs right, within a point of the batch fit on order.

So what

A fixed K is a choice of noise floor. If you rate a league with K=32 and the players' skill doesn't change, every rating stays about ±50 points off forever, and no number of extra games fixes it. If you want ratings that converge, shrink K with games played (about 870/games on this scale), or refit everything in one go now and then.

The case for a fixed K is skill that changes. A player who improves needs ratings that can move, and a shrinking K or a refit of the whole history would be slow to follow them. None of my players changed, so this test only shows what a fixed K costs, not what it buys.

Method: NumPy, vectorised over trials, on one machine. 16 players, random pairings, no draws, no rating changes over time. The shrinking-K slopes come from a separate run of 2,000 trials. The batch fit is a Bradley–Terry maximum-likelihood fit with one virtual draw per player against an average opponent, so a player who has never lost still gets a finite rating.