Daniel Caro, University of Oxford and Jenny Lenkeit, University of Oxford

Country rankings in international education tests – such as PISA and TIMSS – are often used to compare and contrast education systems across a range of countries. But it isn’t always an even playing field.

This is because countries with very different social and economic realities participate, so countries such as Norway, Russia, Chile, Lebanon and Thailand are all being compared against each other. And this is without the difference in socio-economic backgrounds of these different countries being taken into account.

If the latest world education rankings are anything to go by, Turkey and Thailand perform poorly when it comes to their students’ achievement in science. But our analysis shows that if you look at the rankings differently (from an even starting point), both Thailand and Turkey may in fact be just as good as some of the high performing Asian countries.

Our analysis is a much fairer comparison, as it allows for the differences in wealth and social development in which students learn and teachers teach. It builds upon our previous work, where we produced and analysed an indicator of “effectiveness”. The effectiveness indicator ranks performance of countries as if they all had similar socio-economic conditions – thus levelling the playing field.

This makes it easier to see which countries are actually the most effective at educating their students, with social economic factors like wealth taken into account.

New style rankings

The graph below shows how countries are ranked in their effectiveness. At the top of the effectiveness ranking, we find education systems such as Singapore and Japan, which are also generally high performing in PISA and TIMSS.

But our analysis also revealed that countries such as Turkey and Thailand are actually highly effective and perform above expectations in terms of education. This is despite both countries having an overall lower performance score in the global education rankings.

As the graph shows, the performance of education systems in Turkey and Thailand is underestimated if guided by country rankings alone. This is because although these countries perform below average and rather poorly in PISA, they are as effective as high performing Asian countries.

This means that Turkey and Thailand would be ranked among the highest performing countries in the world – if there was no socio-economic differences between countries.

Our analysis also shows education systems in Norway are ineffective, and the same was found to be true of Australia. So while these countries are ranked highest in human development in the world, they are not among the highest performing in these international tests when we level the playing field.

On the lower end of the effectiveness ranking, we find Qatar and the United Arab Emirates. These countries are also among the lowest performing in PISA rankings and could be doing much better for their high income per capita levels.

Fair comparison

It is clear then that overall performance rankings alone do not make a fair comparison when it comes to judging the quality of education in different countries. And our analysis shows how the socioeconomic conditions of a country are vitally important when comparing global performance in education rankings.

Using our data, there would certainly be a case for countries like Chile, Malta, Georgia, the United Arab Emirates and Qatar to look to Turkey and Thailand to work out how to improve their education systems. And as our analysis shows, global education rankings are probably not the best measure of educational performance after all.

Daniel Caro, Research Fellow, Oxford University Centre for Educational Assessment, University of Oxford and Jenny Lenkeit, Research Fellow, Oxford University Centre for Educational Assessment, University of Oxford

This article was originally published on The Conversation. Read the original article.