Twelve years of committee-model backtests

Two models, one honest gap

MMX-F says how good teams are. The committee model predicts what the 13 people on the selection committee will do. The gap between the two is the most interesting thing on this site every week in November, and it is only worth reading if the second model actually works. So before the 2026 committee publishes a single poll, here is the model's full record on the twelve seasons that already happened.

What the committee model is

It is a gradient-boosted tree model (LightGBM) trained on every College Football Playoff poll from 2014 through 2025. The target is committee points: 26 minus a team's rank, or zero for the unranked. Each week the model scores every FBS team and the top 25 scores become the projected poll.

It looks at 20 features, all computed as of each poll date from our own tables so the same code runs live:

• Record: wins, losses, winning percentage.
• Resume: Q1 wins, Q1 plus Q2 wins, bad losses, best win by opponent rating, strength of schedule, strength of record.
• Quality: MMX-F rating and rank.
• Recency: last margin, weeks since the last loss, wins in the last three games.
• Anchors: the previous week's committee rank and the current AP rank.
• Priors the committee smuggles in: a brand score (trailing ten years of AP poll appearances), a power-conference flag, and a Notre Dame flag.
• Flags: conference champion, for the final poll.

One rule sits on top of the trees. When two teams land within three projected spots of each other and their scores are within 0.35 points, the one that won the head-to-head meeting goes ahead. The committee almost always does that, so the model does too.

How it was tested

Leave-one-season-out. Twelve rounds: train on eleven seasons, grade the twelfth, repeat until every season has been graded by a model that never saw it. Three metrics per season.

• Top-15 MAE: for the teams the committee actually ranked in its top 15, how far off the model's rank was, on average. Lower is better; zero would be a perfect ordering.
• Spearman: rank correlation across the full top 25. A value of 1.0 means the identical order.
• Final top-12 overlap: of the twelve teams in the committee's final poll, how many the model also had in its final twelve. This is the metric that decides playoff fields.

The targets were set before training and are published alongside the model: top-15 MAE of 2.0 or better, overlap of at least 10 of 12, Spearman of at least 0.93.

Reading the table

Across all twelve seasons the model averages a top-15 MAE of 1.44, a Spearman of 0.950, and 11.1 of 12 final-playoff teams. All three targets are met, and every season since 2015 clears the MAE target on its own.

Then there is 2014, the one row that looks like it belongs to a different model. A top-15 MAE of 4.79 and a Spearman of 0.779 are the worst marks in the table by a wide margin. That was the first season of the playoff. The committee was inventing its norms in public across seven polls, and the model's single most important input, where the committee had a team the week before, did not exist for the opening poll. We could drop 2014 and print a prettier average. It stays.

The other soft spot is 2020, the pandemic season: five polls, wildly uneven schedules, a top-15 MAE of 1.76 and only 10 of 12 on the final field. Everything from 2015 onward lands between 0.86 and 1.76 on MAE, and the model called the full final twelve in 2016, 2022, 2023, and 2024.

The uncomfortable part

The model's feature importances are not flattering, and they are published anyway. The current AP rank and the previous week's committee rank dominate everything else by more than an order of magnitude. The resume inputs, strength of record, strength of schedule, and best win, matter at the margins.

In plain language: the model is an excellent student of the committee's inertia. Most of its accuracy comes from knowing where the committee and the AP voters had a team last week, plus what happened on Saturday. That is also the honest description of how the committee works. The resume features earn their keep exactly where the interesting decisions live, at the edge of the top 12, which is why the model tracks them at all.

What the model does not do is predict upsets. It predicts how the committee reacts to results that have already happened. Playoff odds come from the simulation, which uses MMX-F to play the games and the committee model to rank the outcomes.

The live era starts at zero

The 2026 ledger is empty on purpose. The committee model activates around week 4, once roughly 200 games are final and the resume features mean something. The first projection publishes before the first CFP rankings in early November, and the moment the real rankings drop, the projection is graded in public: rank correlation, top-12 overlap, average error, and the single worst miss. Nothing gets taken down.