AI Chatbot ChatGPT Fails UPSC Prelims Exams With 54% Score

Cracking UPSC is so hard that even AI hasn't been able to figure it out yet!

(Photo Courtesy: Twitter; Altered by The Quint)

Since its launch in November 2022, ChatGPT, an AI chatbot, has gained immense popularity due to its impressive performance on various challenging exams.

It has showcased its abilities by acing the final exam of the Master of Business Administration (MBA) programme at the prestigious University of Pennsylvania's Wharton School, and also outperforming the brightest medical minds in the US medical exam.

Recently, it was made to attempt the Indian Civil Services Examination conducted by the Union Public Service Commission (UPSC), which is considered to be one of the toughest exams in the world.

Analytics India Magazine (AIM) was the driving force behind ChatGPT's participation in the UPSC exam, as reported by News18.

While the chatbot refrained from providing a definite answer regarding its ability to pass the exam, AIM proceeded to test its knowledge by posing all 100 questions from Question Paper 1 (Set A) from the UPSC Prelims 2022.

To the surprise of the magazine, ChatGPT could only answer 54 of them correctly - which meant that it did not meet the cutoff of 87.54 for general category candidates, and thus did not clear the UPSC examination.

The questions were diverse and spanned topics such as Geography, Economy, History, Ecology, General Science, and current affairs related to India.

It is noteworthy that ChatGPT's knowledge was limited to September 2021, and it provided incorrect answers to topics like Geography and Economy, despite the questions not being specific to any particular timeline.

Additionally, in some multiple-choice questions, the chatbot created an extra option that was not provided in the original question and labeled it as "None of the above."

It is worth mentioning that the Indian Civil Services Examination is attempted by approximately 11-12 lakh candidates every year, and only 5% of them pass.

Although ChatGPT's performance fell short of clearing the exam, its attempt is commendable, and it continues to be a popular AI chatbot among users.