OpenAI's AI chatbot ChatGPT was unable to pass the Union Public Service Commission's (UPSC) extremely difficult Indian Civil Services Examination, according to media reports on Saturday, March 4.
The chatbot, launched in November 2022, has garnered immense popularity and successfully passed various US tests including the US Medical Licensing Exam (USMLE), MBA exams, and the Level 3 Engineers Google Coding Interview.
The Bengaluru-based Analytics India Magazine put it through the civil service tests, which included questions about geography, the economy, history, ecology, general science, and current events, to assess its competence. The magazine quizzed ChatGPT on all 100 of the questions from Set A of Question Paper 1 from the 2022 UPSC Prelims. "Only 54 of them were correctly answered by ChatGPT," it reported. Despite ChatGPT's knowledge only going as far as September 2021, inquiries about current affairs were not adequately addressed. Unfortunately, ChatGPT also gave incorrect responses for subjects like geography and the economy that were not time-specific, stated a report by IANS.
ChatGPT uses word prediction to create writing that resembles that of a human. ChatGPT is unable to conduct online searches, unlike most chatbots. Instead, it produces text based on word relationships that are predicted by internal processes. "ChatGPT is incredibly limited but good enough at some things to create a misleading impression of greatness," as per Sam Altman, chief executive at OpenAI.
Besides the UPSC exams, ChatGPT reportedly also failed terribly in an examination designed for sixth graders in Singapore as reported by IANS.
