Microsoft is working on sound recognition AI technologies capable of detecting natural disasters

Microsoft has always been at the forefront of AI development, and together with OpenAI, the company has been working on and enhancing AI products, such as Copilot (back then, Bing Chat) with image recognition capabilities.

However, the Redmond-based tech giant is working on performant sound recognition AI technologies that would see Copilot (and any other AI model, such as ChatGPT) capable of detecting upcoming natural disasters, such as earthquakes, and storms.

According to a patent published recently, Microsoft’s new technology can recognize different types of sounds, from doorbells to babies crying, or dogs barking, but not limited to them. It can also recognize sounds of coughing or breathing difficulties, or unusual noises, such as glass breaking.

Most intriguing, it can recognize and monitor environmental sounds, and they can be further processed to let users know if a natural disaster is about to happen.

The system takes a sound signal and breaks it into smaller parts, or segments. Each segment is processed to create a normalized representation of the sound in the time domain. This is like a map of the sound over time.

This map is then fed into a trained neural network, which is a type of artificial intelligence. The neural network generates scores and probabilities for each type of sound event in each segment. This is like guessing what type of sound each segment is and how sure it is about the guess.

After that, the system does some post-processing to smooth out the scores and probabilities and generate confidence values for each type of sound for different window sizes.

Ultimately, this technology can be used in various applications. In a smart home device, it can detect when someone breaks into the house, by recognizing the sound of glass shattering, or if a newborn is hungry, or distressed, by recognizing the sounds of baby crying.

It can also be used in healthcare, to accurately detect lung or heart diseases, by recognizing heartbeat sounds, coughing, or breathing difficulties.

But one of its most important applications would be to prevent casual users of upcoming natural disasters by recognizing and detecting sounds associated with them.

The raw audio signal may contain environmental or day-to-day sounds that form audio events to be detected by the computer system, such as a siren, alarm, dorg barking, baby crying, gunshots, explosions, applause, and/or laughter, etc.

Since the technology uses neural networks, it can be easily integrated into AI models, such as Copilot on Windows, ChatGPT, or any other models on the market.

On Windows, however, it would be most useful, as casual users could Copilot being able to detect natural disasters and warn them when they’re in danger. It could also act as a security system, alerting users when someone breaks into their house.

You can read the full paper here.