On 5th March, I will conduct a 'fun' experiment. A World Cup of rankings (or better, a World League of rankings). The idea was inspired by a comparison I saw on CTR's ranking page...
Yes indeed. On
CTR's ranking page there is a comparison among ranking systems. But correctly
and for reasons of fairness and seriousness: only games are considered played
on neutral ground.
Only in these cases can be derived from the ranking positions on the ‘predictions’
immediately. If I consider only the ranking positions in games with a host, so
there will be false claims about what a system (really) predicts.
Example: in eloratings Cyprus
is ranked on #121, Northern
Ireland #113. But eloratings ‘predict’ win
expectancies 61% (.610) for Cyprus
and .39 for Northern Ireland.
So – if Cyprus
wins, most (or all) private persons are right. Eloratings predicts the same as
these persons … But – in your ‘League system’ is assumed, eloratings predicted
false (which is not true).
In the games of March 5, there are very many games in which the minimal weaker
classified teams play at home. These teams very often are nonetheless (‘predicted’)
favorites due their home advantage.
In some cases a wrong prediction will change in a ‘correct’ one (if Northern Ireland wins in Cyprus …). In
the very most cases but we will see an absurd inaccuracy of ranking systems
which doesn’t exist.
The only (reliable) measure of the accuracy of a ranking system is the standard
deviation, based on the squared deviations. (Example: Cyprus wins,
actual result = 1, prediction eloratings .61, deviation = .39, squared
deviation for this match = 0.392).
On the other hand: in the long run the numerical value of the standard
deviation ‘correlates’ with the proportion of games in which the ‘predicted’
winners (favorites) lose. In order to make
a meaningful statement at all, the sample must be large enough (to minimize the
impact of randomness).
For lay people the correct measurements (standard deviation, etc.) seem too complicated
… Therefore, I published on my site a simple, easily comprehensible example. In
contrast to the game here, in my example grossly distorting tendencies do not
occur (perhaps the only problem is the size of the sample – 154 match results).
I understand that you have simplified the game (Prediction League). For many
ranking systems is not to see how they calculate the home advantage, some
others consider no home advantage. But - I think it's a bad solution that now,
in many cases the predictions of ranking systems will reversed in the exact
One additional point: I’m sure it is possible that members of this forum could
predict better than ranking systems or bookies. Therefore bookies ‘cheat’ on
odds. Tipico at the moment offers for Japan
v New Zealand
1.15 – 7 – 20. If you are adding the reciprocals of the odds, so you do not get
1 (=100%), but 1,06 ... (the value bigger 1 is the calculated profit)
In terms of eloratings Tipico calculates including the home advantage a win
expectancy of .886.
And – btw, the simplification (here, for the ‘Prediction League’) is in a
higher sense one of the reasons for the existence of the FIFA rankings in the
Simplify, simplify, simplify – that’s the motto of FIFA ranking and of the
crowd. Great guys like Voros McCracken (or Martien Maas): away with you!