The National Institutional Ranking Framework (NIRF), launched in 2015 by the Ministry of Education as a standardised metric for evaluating Higher Educational Institutions (HEIs), serves as a measure for evaluating higher education institutions in India. Since its launch in 2015, NIRF has become a benchmark of excellence, ranking institutions across the country.
Or have they?
While various rankings by online platforms and media outlets exist, they may lack proper accreditation.
Furthermore, international rankings like Times Higher Education (THE) and Quacquarelli Symonds (QS) World University Rankings consider only a limited number of institutions.
NIRF reflects the trust placed in an institution by the public, industry, and academia. These qualitative surveys evaluate how an institution is perceived by industry professionals, peers, and the public, offering insights into its academic reputation and standing in the industry.
But do these rankings genuinely reflect the reality of academic excellence, or do they fall short of capturing the true essence of India’s diverse educational landscape? EdexLive speaks to experts to understand the objectives of NIRF, the metrics behind it, methodologies, and anomalies to measure the performance of higher educational institutions.
“When a measure becomes a target, it ceases to be a good measure”
The above-mentioned statement by British economist Charles Goodhart becomes particularly relevant to the NIRF when the pursuit of a high rank becomes the primary focus, it can distort the integrity of the metrics themselves.
Quoting Goodhart, a senior research analyst at the Department of Science and Technology — Centre for Policy Research (DST-CPR), Indian Institute of Science (IISc), Bengaluru, Moumita Koley challenges the emphasis on rankings. While metrics are intended to provide benchmarks and insights, she questions whether they adequately address the unique needs of India’s academic institutions or if they simply serve as an arbitrary goal.
“Rankings are essential to identify promising institutions. Metrics act as a valuable benchmark for students. However, if these metrics are not thoughtfully designed to address the needs of a diverse demographic, its effectiveness diminishes,” said Koley, further adding that it’s crucial to understand that universities play a complex and multifaceted role, making it challenging to capture their contributions in a single score.
Unpacking perception scores: A case of discrepancy
In a country like India, many factors influence the perception of educational institutions beyond just rankings. When asked which institutions they consider the best, a student responded:
“For engineering, Indian Institutes of Technology (IIT) Bombay, Delhi, and Kharagpur are the top choices. Medical students often aim for public institutions. And for an MBA, everyone knows the Indian Institutes of Management (IIMs) in Ahmedabad, Lucknow, Bangalore, and Kolkata are the best.”
This suggests that opinions about institutions are shaped not only by rankings but also by a broader perception influenced by reputation, tradition, and public awareness.
In university rankings, perception serves as a metric that assesses how an institute is viewed by the academic community, industry, and research sector. Beyond the data, perception is highly subjective and plays a significant role in shaping the institution’s reputation.
As per the outline framed by NIRF, the ranking framework evaluates institutions on five broad categories namely,
— Teaching, Learning and Resources
— Research and Professional Practice
— Graduation Outcome
— Outreach and Inclusivity
And lastly, Perception.
Akhil Pruthi, Head of Ranking & Accreditation, Birla Institute of Technology and Science (BITS), Pilani, identifies perception as a major drawback in the NIRF rankings. He stresses the importance of aligning perception scores with an institution’s actual achievements.
Pruthi points out that despite BITS Pilani’s world-class facilities, distinguished alumni, and global recognition, its NIRF perception score is 28.11 out of 100 — a significant shortfall.
“BITS Pilani, renowned for its exceptional facilities, illustrious alumni, and global reputation, is often lauded as a premier institution. Despite this high regard and its substantial contributions to academia and industry, the NIRF perception scores narrate a different story,” said Pruthi.
Here are a few institutions that received significantly higher scores compared to BITS Pilani:
The low perception scores of BITS Pilani, a prestigious private institution that has been operating since the year 1964, raises questions about how such a longstanding and well-established institution, along with others of similar renown, could score poorly despite its rich legacy. These discrepancies jeopardise the prospects of these institutions.
Pruthi, questioning the methodology behind the rankings, told EdexLive, “When there is a significant mismatch between perception and reality, it risks undermining the credibility of both the rankings and the institutions they represent. We must consider that if perception and reality are so disconnected, what does this imply about the reliability of the metrics we use to gauge institutional excellence?”
Vivek Thakur, an IIT Kharagpur alumnus and Managing Director of Scholars Den, a platform mentoring IIT-JEE aspirants, has extensive experience working with several IITs. He spoke to EdexLive about the critical role perception plays in shaping an institution’s academic credibility and the opportunities it offers.
Thakur pointed out a stark example regarding the opening and closing ranks of the IITs. While IIT Madras tops NIRF charts, IIT Bombay is often considered the most sought-after institution.
“One can easily look into these ranks for these IITs after the Joint Entrance Examination (JEE) Mains and Advanced examinations. This offers a clear indication of their standing, which contrasts sharply with the official rankings,” he asserted.
Opening and closing ranks for BTech in Computer Science and Engineering for 2024:
IIT Bombay - 66
IIT Delhi - 116
IIT Madras - 159
Thakur further noted that students often consider these closing ranks before selecting their institution, focusing on the top courses by the IITs, effectively establishing how perception is an important parameter in decision-making that the NIRF Rankings cannot take into cognisance.
What does the NIRF state on perception?
As per an official document released by NIRF, here is a parallel drawn between the NIRF and other international rankings.
— Indian rankings emphasise the collection, verification, and use of reliable data
— Global rankings often give disproportionate weightage to perception and internalisation (measured by the international students and faculty)
— NIRF rankings are based on objective data as the higher education system is diverse and large, whereas the other rankings fail to address this
It further states that overemphasis on perception data can be “misleading” and susceptible to manipulation, a concern that is particularly relevant to global rankings.
Methodological flaws in NIRF data
Group Vice-Chancellor at BITS Pilani, Professor V Ramgopal Rao, and Abhishek Singh from BITS Pilani have identified several inconsistencies in NIRF, particularly concerning data transparency and accountability. Their paper titled “Unpacking inconsistencies in the NIRF Rankings”, published in Current Science, on June 10, 2024, shows how a detailed analysis was carried out to find out why the NIRF failed to account for transparency and accountability, by contrasting the QS World University Rankings and the NIRF.
The paper argues that the NIRF lacks mechanisms to correct anomalies and potential errors, impacting the stability of the rankings. The authors also observed significant shifts, both positive and negative, among institutions outside the top 20 over the past two years (2022 and 2023).
A. Fluctuations in ranks
Rao and Singh note that while temporary fluctuations are expected due to data variations over time, excessive variability may raise concerns about the reliability of the data.
A few examples that can be taken into account where the differences in the change in ranks are quite perceptible are:
B. QS World University rankings versus NIRF
The QS World University Rankings 2024, favoured by many academics, includes 46 Indian universities, making India the seventh most represented country worldwide and third in Asia after Japan and China.
Notably, IIT Bombay improved from 149th to 118th place, while IIT Delhi climbed 47 spots to reach 150th. Below is a comparison of the leading Indian institutions based on QS rankings and their positions in the NIRF.
Several universities and institutions that are highly ranked in the NIRF also feature prominently in the QS University World Rankings. The table below shows their names along with their NIRF and QS Rankings for 2024.
The table above certainly shows discrepancies when it comes to the two rankings, but can one ranking scheme discredit the other?
EdexLive inquired into this rising conundrum.
Different methodologies, different objectives?
BITS Pilani official Akhil Pruthi said that the differences between the rankings stem from their distinct methodologies.
“NIRF allocates 60% of its score to teaching, learning and resources, and research and professional practice. In contrast, QS assigns 45% of its weight to reputation, split between employer and academic feedback, and adds 15% for international metrics like faculty diversity and global research networks," he said, emphasising that differences in metrics do not invalidate the process.
He added, “The apparent discrepancies between them might reflect the inherent differences in their evaluation criteria and objectives rather than indicating outright errors.”
Even if validated, the NIRF ranking methodology still leaves room for doubt.
Vishal Vaibhav, a former professor at IIT Delhi, pointed out that the top five IITs (Madras, Delhi, Bombay, Kanpur, and Kharagpur) have been awarded full scores — 20 out of 20 — under the ‘Facilities for Physically Challenged Students (PCS)’ criterion. While IIT Delhi, Kanpur, and Bombay have instituted dedicated cells for assisting People with Disabilities (PwDs) students, IIT Madras and Kharagpur are trailing behind in this regard.
According to online sources, a few infrastructural deficiencies for PwD students may be found, such as the absence of elevators or specially designed toilets in most of the academic buildings.
Commenting on the discrepancies between the reported scores and actual facilities, Vishal Vaibhav remarked, “In the NIRF rankings, if the numbers are cooked, one should be able to identify at least one instance where the rating is demonstrably inaccurate. In the category of PCS, the top five institutions have 100% scores which is an infrastructural impossibility on our campuses, even if only the academic areas are considered.”
Further critiquing the ranking schemes in the Indian academic context, the professor added, “Applying this observation to other categories, a ranking framework on lines of QS or THE are less effective in India because implicit standards are non-existent and the integrity of the enforcing/auditing bodies is generally compromised. Just consider how often we follow zebra-crossing rules and apply that perspective to the broader context!”
The matrix of the metrics: Is it TRULY reliable?
While the NIRF awards scores based on performance metrics, V Ramgopal Rao and Abhishek Singh of BITS Pilani argue that this approach falls short in capturing aspects such as relevance, innovation, social impact, and contributions beyond traditional publishing. “Exacerbating these problems is the bibliometric methodology employed by the NIRF, which relies entirely on commercial databases for data collection,” they said.
The official NIRF document stated that due to the “Absence of a reliable and comprehensive database that could supply all relevant data required for computing the scores for ranking of HEIs, registered institutions were invited to submit the required data through an Online Data Capturing System (DCS)”.
Further, publications and citations of the research outputs of applicant institutions were retrieved from ‘third-party sources’ such as Scopus (Elsevier Science) and Web of Science (Clarivate Analytics).
However, here's an interesting discovery
Manipulated metrics and the sham of paper mills
While the data sourced may be verified, it does not eliminate the risk of breaches in academic integrity. EdexLive, in collaboration with India Research Watch (IRW), a platform launched by academics, researchers, and students, uncovered loopholes that allow institutions and several academicians to gain an undue advantage.
Manipulation of research metrics has become a prevalent method to enhance both national and international rankings. One way to achieve this involves the use of illegal ‘paper mills’. These entities, whether individuals or organisations, offer to write a manuscript on behalf of the researcher. These are then sold to individual(s) with fees depending on the order of authorship. To further inflate citation scores, the writer may include excessive references to highly ranked universities.
A LinkedIn post by IRW highlights a case where an author was listed with six affiliations in a single paper. This is not an isolated case and there are numerous instances of multiple affiliations. The universities involved in these cases are consistently the same, including several of the top-ranked private institutions in India.
Here is the list of the Indian institutions mentioned in the paper by the author of the article:
1) UPES, Dehradun
2) Chitkara University Institute of Engineering and Technology, Punjab
3) Chandigarh University, Mohali, Punjab
4) Lovely Professional University, Punjab
As per the official NIRF 2024, in the overall category, UPES ranked 59. Chitkara University bagged the 94th position in the Engineering category. Whereas Chandigarh University and LPU ranked 32nd and 50th in the Overall category, respectively.
What are ‘Paper mills’?
Achal Agrawal, IRW Founder, revealed to EdexLive that numerous channels on messaging platforms like WhatsApp and Telegram facilitate the transactions of paper-buying services.
“In lieu of money, these channels offer you to become a second or third author in a Q1 journal, which falls within the top 25%. You pay them a certain amount, and then the university compensates you for publishing under their affiliation. In this cyclical process, you not only earn money, but also gain authorship on an article. This is the core operation of paper mills,” explained Achal.
He pointed out that this type of fraud is becoming increasingly common in the field of science, driven by the incentives created by the ranking system.
Another case involves Saveetha Institute of Medical and Technical Sciences (SIMATS) and Saveetha Dental College in Chennai. According to Retraction Watch (The Center for Scientific Integrity), a reputed blog that tracks retracted scientific papers due to academic integrity issues, SIMATS allegedly committed a significant academic breach.
In a report from December 2023, a major academic breach committed by SIMATS was highlighted. An associate professor at Kebri Dehar University in Ethiopia revealed that an official from SIMATS allegedly proposed adding their affiliation to his papers in exchange for incentives. When the professor did not receive the promised payment, he exposed the scheme.
Further allegations against Saveetha institutions include a practice at Saveetha Dental School, where students are required to write manuscripts as part of their exams, which are then published post-review and added references in journals like Scopus. The report states that the institution terms this an exercise to help students gain experience in research.
In 2022-2023 alone, Saveetha Dental College published 4,277 articles.
This performance is reflected in their NIRF scores, where SIMATS received high marks for publications, surpassing even IIT Kanpur in a few categories. However, their low scores in professional practices and project footprints raise questions about the global repute of these rankings.
As per lens.org, SIMATS published a total of 5,037 scholarly works (2022-23), whereas, a public institution like IIT Kanpur's data for the same timeline, stood at 2,034.
The article highlights that SIMATS ranked:
22nd in the overall category
53rd in the engineering category
11th among the universities
20th in the research institution category, and
12th in the medical category.
Where does India stand globally?
In the Global Innovation Index (GHI) 2023, India was in the top three innovation economies, in the lower middle-income group in the Central and Southern Asian region. As per the innovation rankings, India stood 40th amongst middle-income economies and fourth globally in publication output, surpassing the United Kingdom in fifth position and close behind Germany in the third position.
India’s four cities — Bengaluru, Delhi, Chennai, and Mumbai — were named science and technology clusters globally, with India being one of the best performers in ‘Knowledge and Technology Output’. However, Achal warns that the increase in research output may be driven more by financial incentives than by genuine academic progress. Retraction Watch reported that SIMATS spent ₹4.8 crore (USD 584,000) on publication incentives in 2021, suggesting that such practices could tarnish India’s global standing.
Emphasising India's scientific reputation, he remarked that it is on the decline even if it seems to be booming. He further opined, “The publishing industry gains billions of dollars and makes more profitable margins than corporate tech giants. They are encouraging a business of fraudulence. All that output mostly contributes to scientific pollution.”
While research and development in India appear to be advancing rapidly, the increasing number of paper retractions suggests that the evidence of growth is indeed not so harmonious.
As of September 30, 2022, Retraction Watch has recorded a total of 1,541 retracted items originating from India between 1947 and 2021.
While this number is significantly lower than China (14,728) and the United States (4,908), it surpasses the UK (1,358), Japan (1,303), Germany (966), and South Korea (717) within the same period.
India’s R&D funding and ranking challenges
The IITs, IISc, and Indian Institutes of Science Education and Research (IISERs), in India, have been a beacon of the Research and Development (R&D) industry. The NIRF is mostly dominated by state-of-the-art public institutions, mainly the IITs. This gives a great impetus to these institutions and produces exemplary outputs.
However, the inequality when these institutions are pitted against several other public universities is palpable.
Manu Awasthi, an associate professor of computer science at Ashoka University, says that India may do better in the field if it extracts itself from the vicious cycle abetted by the rankings.
“The current circumstance is effectively a zero-sum game due to the government’s inadequate investment in R&D. With overall funding remaining relatively small, top-performing institutions receive a larger share of resources, enabling them to continue excelling and secure even more R&D funding. This limited expenditure diverts resources away from non-top-tier institutions, trapping them in a vicious cycle,” explained Awasthi to EdexLive.
Terming it as a long-winded argument about the improvement of rankings, it may require a distinct amount of time.
The emphasis on R&D should be a uniform affair rather than a sporadic one. Otherwise, over time, it might not make any distinctive difference in the rankings and also eliminate other worthy institutes due to their lower rankings.
Challenges and opportunities: India’s research culture & pathways for reform
With over 400 patents and 10,000 citations in top journals, Sasikanth Manipatruni, a researcher, and an engineer, while sharing his academic experience, told EdexLive that other countries such as the US have one of the best research cultures and an egalitarian ethos.
The culture of equality and meritocracy he experienced at Cornell was profoundly liberating for him. As a PhD student there, Manipatruni enjoyed a level of freedom that would have been challenging to earn in India.
Being an alumnus and topper of his batch at IIT Delhi, Manipatruni highlighted, “IITs do not really have a research emphasis for the teachers. There is way too much teaching and bureaucracy.”
When it came to policies to retain and incentivise academic talent, he said, “While India boasts of the world’s best methods to identify academic talent, it is decades (often half a century) behind in the autonomy of institutions, egalitarian culture for science and critical infrastructure to retain and groom the talent domestically.”
When asked if there are possibilities that such a landscape can change, the academician and entrepreneur mentioned that the Indian system can replicate the US, Japan or Singapore where the policymakers actively seek inputs from young world leaders, constantly refining their industrial policies.
Additionally, it may require:
a. Developing institutions with autonomy
b. Encouraging endowment funds through large donations for academic institutions
c. Immense investments in the research infrastructure
Underscoring the problems in governance, Manipatruni said, “It is very sad that British India produced more Nobel prizes than independent India.”
While it is essential for these changes to be implemented consistently at the grassroots level within institutions, one can only hope that these esteemed institutes will not only retain existing talent but also attract global talent. This would then help position India as a research powerhouse, transcending the commercialisation of education that is further influenced by rankings.
