LLM Leaderboard

I kept asking myself the same question, which was which model is actually worth sending a job to right now, and the answer was spread over more than a dozen sites that each measure something different. This page pulls them together. A job on my VPS fetches every source each night and rebuilds one static page: the frontier at a glance, then a table you can sort and filter, and a set of charts. The sort, the filters, and the columns you turn on all live in the address, so a link carries the exact view.

It is one Python file that uses nothing outside the standard library, and the page it prints is plain JavaScript with no framework and no build step.

The hard part

Every source names models its own way, and some of them list each reasoning setting of one model as a separate model. Join them carelessly and a row ends up with one setting’s rating beside another setting’s test score and a third’s price. Each number is real, and the row as a whole describes a model that does not exist. A leaderboard is only useful if a row can be trusted to be about one thing.

How I solved it

The join keeps track of how it was made. Each row records how loosely its sources were paired, and hovering a value shows where it came from. When a rating was measured on a different setting from the one the row is named for, the page still shows it, marked with a star, and never sorts it with the rest. Where a cost had to come from another setting, both halves of the ratio come from the same test run, and the tooltip names the run.

The same rule covers a bad night. When a source fails, the page serves its last good copy and says so in its header line, so a short table is never read as a whole one.