What’s Wrong with the FIFA Rankings?

The Official FIFA World Rankings often receive puzzled looks and sometimes severe criticism too. I am going to explore the calculations and whether there might be improvements available

I first wrote about the FIFA Rankings in 2014 in response to the surprising presence of Switzerland in Pot 1 for the World Cup (top 7 teams in the world + hosts Brazil) with well-considered teams such as France, Italy, Netherlands and England left in the generic unseeded Europe pot.

Let’s start by decomposing the calculations used, explaining what the intention of each part is, and pointing out some of the issues.

 

How does FIFA calculate the Rankings?

Each international match you get a certain number of points. The ranking system is ordered by the average number of points over the last 4 years, with a greater weighting on more recent years. The points that you get for the match is as follows:

Points = M x I x T x C

M = Points for match result
3 points for a win, 1 for a draw, 0 for a loss.

Intention:
A win is a better result than a draw, which in turn is better than a loss.

Seems OK on the surface of things, but they appear to have ignored some basic yet vital match data – the scoreline, and the location of the match.

Flaws:
M1: If you lose, you get 0 points for the entire match and the other factors in sections below don’t even play a part. Losing 8-0 to American Samoa is equivalent to losing 4-3 to Spain.
M2: If you win, it doesn’t matter how much you win by. 8-0 against Spain is the same as 1-0 against Spain.
M3: There is no difference whether you play at home or away. Home is clearly an advantage but this isn’t accounted for.

I = Importance of match
Friendly = 1, Qualifier = 2.5, Continental/Confederation Cup = 3, World Cup = 4

Intention:
A team’s true worth can be seen in the most important matches, while friendlies are often played with experimental sides.

Unfortunately, a scaling factor is clearly not the way to go here, considering that your end result (as explained later) is an average across all your matches. The intention here is to weight the score towards the important matches, but all they’ve managed to do is tell you that your result is worse if its a friendly match…

Flaws:
I1: The more friendlies you play, the more of a disadvantage you have because your average will be so much lower!
I2: Saying that the European Championship is as prestigious as the Oceania Cup is a very contentious decision.
I3: Different teams naturally play more friendlies than others. The most notable example being hosts of major tournaments, who do not need to participate in the qualifying tournament, thereby ensuring that they plummet down the rankings in the 2 years preceding the tournament. (Look at most recent examples Brazil and France in the 2013 and 2015 rankings respectively)

T = Strength of opposing team
This depends on the World Ranking of the opposition team, with 1st = 200, 2nd = 199 down to a minimum of 50 for any team ranked 150th or below.

Intention:
Playing against better opponents is harder, so you should be rewarded with extra points if the team is ranked higher.

The intention here is excellent, but the execution isn’t great. This is really badly scaled against the 2 factors I’ve already decomposed.

Flaws:
T1: Let’s compare how to balance T with I in 3 matches:

  1. Beating Spain in a friendly. T x I = 200 x 1 = 200
  2. Beating American Samoa in the World Cup (if they made it). T x I = 50 x 4 = 200
  3. Beating Tajikistan in a qualifier. T x I = 85 x 2.5 = 212.5

Match 1 is worth the same or less than the other 2, which doesn’t seem quite right.

T2: Let’s compare how to balance T with M in 3 matches:

  1. Drawing against Spain 2-2. T x M = 200 x 1 = 200
  2. Beating Rwanda 1-0. T x M = 74 x 3 = 222

Scraping a win against Rwanda is worth more than drawing with Spain. Also doesn’t seem great.

C = Strength of confederation
Scaling factor according to the opposition team. This was worked out using performance at World Cups
Europe/South America: 1.0
North/Central America: 0.88
Asia/Africa: 0.86
Oceania: 0.85

Intention:
Some continents are better than others at football so they put in a multiplier depending on your opposition.

The intention is not good to start with. There is no need for this part…

Flaws:
C1: They already scaled by opposition strength in T. I’m not sure why it’s being done again.
C2: Why do Guyana and Brazil get the same scaling? Or Spain and Andorra?
C3: Why are they using the World Cup to figure out the scaling? Only the best teams from each continent are there. For example, you’re basing the strength of Oceania on Australia only!

 

Example

Now that you understand the rankings breakdown a lot better, here’s a ludicrous made-up example:

We have 2 countries, A and B. Here are their results from this year:

A beat Spain 7-0 in a friendly in Madrid. 3 x 1 x 200 x 1 = 600
A beat American Samoa 14-0 in a friendly. 3 x 1 x 50 x 0.85 = 127.5
A beat Tonga 12-0 in a friendly. 3 x 1 x 50 x 0.85 = 127.5
A beat Guyana 15-0 in a friendly. 3 x 1 x 50 x 1 = 150
A drew with Brazil 4-4 in a qualifier in Rio. 1 x 2.5 x 191 x 1 = 477.5
A lost to Germany on penalties in the World Cup after a 3-3 draw. 0 x 4 x 199 x 1 = 0

You thrashed Spain away from home, embarrassed a load of other nations, and unfortunately lost on penalties to Germany. Your average is 266.5.

B lost 9-0 to American Samoa in a friendly. 0 x 1 x 50 x 0.85 = 0
B lost 9-0 to Tonga in a friendly. 0 x 1 x 50 x 0.85 = 0
B lost 9-0 to Vanuatu in a friendly. 0 x 1 x 50 x 0.85 = 0
B beat Antigua and Barbuda on penalties in the World Cup after a 0-0 draw. 3 x 4 x 92 x 1 = 1104

You were totally emabarrassed by the worst nations in the world 3 times. You scraped a win in the World Cup on penalties against an island with less people than many UK towns. You haven’t even scored this year. Your average is 276. You are better than team A!

 

What alternatives are there?

I’m certainly not the first person to suggest that FIFA should consider another ranking system, so alternative ranking systems have already been developed and analysed.

One option would be to improve the current system by removing the useless parts, re-scaling the bad parts, and using more of the basic match data.

A complete switch in system is perhaps the best route to follow though. If any of you are chess fans, you’ll be aware of, or even familiar with the ELO rankings system. This can easily be translated to football, as both are 1v1 games that can end in a win, draw or loss. Add in some factors for home advantage and the scorelines and you have http://www.eloratings.net/.

Some university researchers also investigated other rankings methods and provided a comparison based on their predictive power. Their paper is here: http://lasek.rexamine.com/football_rankings.pdf.
If a ranking system can predict the outcome of a match more accurately than another, it could be considered superior. Here’s the final table of methods and how good they were at predicting results. A lower score is better.

FIFA ranking daily – 1.3681
FIFA ranking release – 1.3705
Elo WWR 1500 – 1.3698
Elo WWR FIFA06 – 1.2674
Elo WWR FIFA06 WDL – 1.2934
EloRatings.net – 1.2634
EloRatings.net 1500 – 1.3265
EloRatings.net FIFA06 – 1.2811
Elo++ – 1.3062
Least squares – 1.2786
Least squares home team – 1.2681
Network-based ratings – 1.4268
Markovian ratings wins – 1.3605
Markovian ratings goals – 1.3557
The Power Rank – 1.2735
Ensemble – 1.2358
All draws – 1.5960
Home team – 4.1733

It can be seen that the FIFA rankings were quite poor at predicting any results (doesn’t surprise me). EloRatings.net was the best single method, so I’d encourage you to read about that on their website if you’re interested.

Interestingly the very best method (“Ensemble”) was to take a few of the individual methods and average their predictions out! This can’t be considered for a ranking system unfortunately.

I shall finish with a particularly pertinent line from their conclusion:

“Our experiments has shown that it is possible to outperform the official ranking procedure by relatively simple algorithms, which is surprising given the high influence of this ranking on football competitions”