With the Supreme Court resuming hearing petitions surrounding the National Eligibility-cum-Entrance Test for Undergraduate (NEET UG) medical admissions tomorrow, July 18, several aspects of the exam, allegations of misconduct, and the statements made by the National Testing Agency (NTA) trying to defend itself could be challenged.
One of these is an affidavit submitted by the central government on July 10 with the data analysis of the NEET UG results conducted by the Indian Institute of Technology (IIT) Madras.
This report, seemingly formed after Chief Justice of India (CJI) DY Chandrachud asked the NTA, “whether it would be feasible using data analytics” to identify suspicious cases, claims that there was “neither any indication of mass malpractice nor a localised set of candidates being benefitted leading to abnormal scores”.
Further, IIT Madras’ report showed data of all candidates as a bell-shaped curve, adding that such a graph can be seen in any large exam. Moreover, the institute also claims that no anomalies emerged in the data.
Report’s validity questionedChallenging these claims, several experts questioned the validity of the methodology employed by IIT Madras in reaching the conclusions it did.
For instance, Prof Dheeraj Sanghi, Vice-Chancellor at JK Lakshmipat University, Jaipur, and former professor at IIT Kanpur questioned how IIT Madras could infer an absence of anomalies just by looking at the graph, as quoted by The Indian Express.
In conversation with EdexLive, he also said that a bell-shaped curve, which represents a normal logarithmic distribution of marks, needed to clarify whether some candidates used unfair means to write the exam or not.
Taking this idea further, Prof Vishal Vaibhav of IIT Delhi issued an “experiment” to verify the soundness of IIT Madras’ research methodology.
He presents four graphs similar to the one created by IIT Madras, all of which include data from the scores of NEET UG 2023, but with a catch — some of these curves are doctored!
“The trick is that only we know which curves are doctored. For all it matters, it could be none of them or all of them,” he told EdexLive.
Prof Vaibhav plans to submit this test to the Supreme Court for IIT Madras to identify which of these graphs has anomalies. “If the institute could conclude that there were no anomalies in the NEET UG 2024 results just by the shape of the graphs, they should also identify anomalies in these graphs through similar means,” explains the Assistant Professor at Optics and Photonics Center, IIT Delhi.
“IIT Madras not quantifying its claims”
The biggest red flag in the analysis is the lack of a standard dataset against which this year’s NEET UG data could be compared, making the conclusions of the report more unfounded, says Prof Vaibhav
According to him, the absence of data from the 2023 NEET UG exam renders the conclusions of the report hard to believe. “The report states that it was confident in its inference of a lack of anomalies. In statistics, confidence in data carries a lot of weight, and I do not understand where IIT Madras’ confidence in its findings comes from,” he says.
Moreover, he finds the fact that there were no deviations in the report a little unsettling. “The findings are too perfect, too clean — and there is nothing to back them,” he says.
According to Prof Vaibhav, the NTA did not a provide comparative analysis of the 2023 and 2024 NEET UG data until this was pointed out, and even when it did, the data was insufficient.
“The NTA provided a graph in a pictorial form, and I had to extract data points from the image,” he says.
Therefore, through this test, Prof Vaibhav aims to demand that the NTA issues proper statistical data on the conduct of the exam and the scores of the students.
