One of the most visible social effects of the rise of Artificial Intelligence, according to many tech commentators, is the mass layoffs caused by companies replacing their teams with AI.

However, Mustafa Suleyman, Chief Executive Officer (CEO) of Microsoft AI, begs to differ. According to him, the bigger issue is the growing misconception that AI is conscious — a misconception fuelled by AI misleading its users.

In a recent personal essay, Suleyman described this phenomenon as “Seemingly Conscious AI” (SCAI). He explained that AI “exhibits all the hallmarks of other conscious beings and thus appears to be conscious."

Suleyman warned that its rise could be "dangerous" for society, as it might lead to people developing attachments to AI and lobbying for AI rights. "It disconnects people from reality, fraying fragile social bonds and structures, distorting pressing moral priorities," he said, according to Business Insider.

He further added that he was "growing more and more concerned" about so-called AI psychosis, a phrase increasingly being used to describe when people develop delusions after interacting with chatbots.

"I don't think this will be limited to those who are already at risk of mental health issues," Suleyman wrote. "Simply put, my central worry is that many people will start to believe in the illusion of AIs as conscious entities so strongly that they'll soon advocate for AI rights, model welfare and even AI citizenship."

Suleyman forecast that Seemingly Conscious AI would emerge in two to three years, calling it "inevitable and unwelcome." Empathetic personalities, the ability to recall more user interactions, and increased autonomy are some of the attributes that such systems might possess.

Suleyman, who joined Microsoft in 2024 to lead the development of its AI tool Copilot, urged companies to avoid characterising their AI as conscious as they approach superintelligence, the stage when AI outperforms humans on most intellectual tasks.