Why should the 2008 THES-QS world university rankings not be taken seriously?

For an alumnus of a national university —the University of the Philippines (UP), it is not that easy to accept the fact that the Ateneo de Manila University (ADMU) outranked UP in the 2008 university rankings recently released by the Times Higher Education Supplement and Quacquarelli Symonds (THES-QS). ADMU rose from 451 ranking to 254 while UP landed to 276 from last year’s 398.

Two other local universities made it to the rankings. Quoting blackshama on his Filipino Voices article On university ranking: views from both sides of Katipunan Road:

“De La Salle is at 415th and our only ancient university, the Royal and Pontifical University of Santo Tomas is at 470th.”

As early as July 2007, UP announced already its refusal to participate in the said survey because the THES-QS could not explain where it got the figures on which UP’s rank was based in the year 2006.

The UP Newsletter article entitled Only two RP universities made it to the world’s 500 best; but must the rankings be taken seriously? written by Alicor L. Panao continued:

“In response to UP’s objection, however, Quacquarelli Symonds research assistant Saad Shabbir simply wrote back that if it did not receive the information by the deadline then it would be 'forced to use last year’s data or some form of average.'”

Just last Friday, October 17, UP announced again in its website that it did not participate in the THES-QS university rankings and President Emerlinda R. Roman did not even receive any invitation to be a part of it.

To give you an overview of how the THES-QS came up with ranking of the top universities of the world, here are the six indicators they used:

• 40% - Academic Peer Review - Composite score drawn from peer review survey (which is divided into five subject areas). 6,354 responses in 2008.
• 10% - Employer Review - Score based on responses to employer survey. 2,339 responses in 2008.
• 20% - Faculty Student Ratio - Score based on student faculty ratio
• 20% - Citations per Faculty - Score based on research performance factored against the size of the research body
• 5% - International Faculty - Score based on proportion of international faculty
• 5% - International Students - Score based on proportion of international students

Apparently, if you don’t submit proper documentation in the abovementioned criteria, old data will do for the THES-QS researchers. Sounds rational enough, isn’t it? Okay, not really, am just being sarcastic.

To boot, Wikipedia’s THES-QS World University Rankings article offered several other criticisms about the said survey including too much emphasis on peer review, errors have also been reported on the faculty-student ratio used in the ranking, the pool of responses is heavily weighted in favor of academic ‘peers’ from nations where the Times is well-known, the results have been highly volatile for other universities, and others.

Panao’s article in the UP Newsletter also offered flaws in the THES-QS survey in other higher education institutions in many countries just like what happened in Duke University whose rank rose drastically in 2005 when it was listed to have 6,244 faculty members. It turned out that this figure was actually the number of undergraduate students enrolled at Duke in the fall of 2005.

ADMU’s official comment in the 2007 THES-QS World University Rankings questioned the criteria used in the survey as not congruent to a Catholic school’s mission and vision. Hence, the results became their benchmarks to strive more “to grow in excellence and in linkages with peer institutions internationally… and continue to be the formation of a next generation of leaders who will not only be the best in their chosen professions, but who will place this excellence at the service of others.”

True to a fact, they deserve congratulations for emerging first among Philippine schools included in the ranking.

However, I just don’t get the basic idea in a certain part of the article Rankings and caution by Ramon Miguel Samson posted in the ADMU website saying:

“Self-professed experts would have provided their own analyses of the rankings, ranging from intelligent and pragmatic examinations of the methodology, to delusional theories that are on the same level of saying “nobody called us,” which all make reference to karma or some other supernatural cause.”