It might be an interesting exercise (for a bored nerd) to find a correlation between margins of victory in different sports.
That's essentially what I did when I had my rankings operational - I had to calibrate the margins of victory before applying the modified Elo formula.
I'll drag up my old rankings from 2012 to re-illustrate what Neil has already illustrated without trying, with the addition of a few figures for fun (well, to illustrate the sharp divides within the Top 20 between football and rugby union).
As we can see, the difference between 3 and 14 (South Africa and Japan in rugby) is 639 Elo points.
The difference between 3 and 14 in football (Brazil and Italy) is a mere 128 Elo points.
If we go down 639 Elo points below Brazil in football, we would arrive at Bermuda. Bermuda beating Brazil would certainly be a shock.
The difference between 1st and 5th in rugby is roughly the same as between 1st and 20th in football.
The other example given by ice hockey - Great Britain beating Canada would not be such a big shock - 417 Elo points only. An equivalent shock in ice hockey would be Estonia beating Canada.
In other sports, equivalent shocks to Japan beating South Africa at rugby would be:
CRICKET: Scotland beating India
BASKETBALL: Bulgaria beating USA
FIELD HOCKEY: Poland beating Australia
WOMEN'S FOOTBALL: Austria beating USA
WATER POLO: Brazil beating Serbia
HANDBALL: Saudi Arabia beating Spain
CURLING: Russia beating Canada
FUTSAL: Lithuania beating Brazil
BEACH SOCCER: Latvia beating Russia
FLOORBALL: Estonia beating Sweden
BANDY: Norway beating Russia
Kenya beating West Indies at the 1996 Cricket World Cup was of similar shockingness to Japan beating South Africa, except the margin of victory was even greater (73 runs - probably the equivalent of Japan beating South Africa at rugby by 15 or 20 points).
If we look further down the FOOTBALL rankings, San Marino beating Russia or Sweden would also be a shock of similar magnitude, or Afghanistan beating Scotland.
(PS - Neil's objective fudging turned out to have the same effect in establishing exactly the same Top 18 in rugby, and showing the big gap between those and everyone else.)