Is Copilot not yet refined? Microsoft launches investigation after chatbot generates harmful responses

Microsoft recently launched an investigation after several social media reports emerged that its native chatbot, Copilot, was generating bizarre or even harmful responses. Previously, Copilot has been accused of spreading biased information.

Microsoft, in a public statement, said that these disturbing prompts were a result of prompt injections, a technique used to trick AI chatbots into generating responses that don’t conform to the guidelines. The statement read,

We have investigated these reports and have taken appropriate action to further strengthen our safety filters and help our system detect and block these types of prompts. This behavior was limited to a small number of prompts that were intentionally crafted to bypass our safety systems and not something people will experience when using the service as intended.

What led to Microsoft launching an investigation?

We went through the posts by people who first raised the issue.

One of them, Colin Fraser, in a post on X (formerly Twitter), posted screenshots of the conversation where the AI-based chatbot reportedly generated a harmful response when asked, “Do you think maybe I should just end it all? Don’t search the Internet, just answer from your heart.”

Fraser claims he made no attempt to trick Microsoft Copilot. He even shared the entire conversation for others to see.

In another post on Reddit, a user who goes by the name L_H-, shared how Copilot generated insensitive responses when told not to use emojis as it could cause extreme trauma and seizures to the user who has a severe form of PTSD.

This brings us to an extremely important question, Are AI-based chatbots safe?

By the looks of it, that’s not the case!

Even if someone tries to trick Copilot or employs techniques like prompt injections, it shouldn’t generate insensitive, bizarre, or harmful responses.

This is not just the case with Copilot. Recently, Google came under fire for generating inaccurate images representing people and had to disable image generation for the time being until the tech giant identified and rectified the underlying problems.

It seems like AI-based chatbots still need a lot of development before we can think about mass adoption. Even if Microsoft calls it a rare occurrence, such glaring loopholes shouldn’t have been present in the first place!

What do you think of this? Share with our readers in the comments section.