Microsoft and Novartis use Kinect to help doctors assess Multiple Sclerosis

It seems that Microsoft’s Kinect is definitely getting interesting use cases outside of gaming lately. For example, we told you last week how the technology allowed orangutans in an Australian zoo to play video games. Today, Microsoft is showcasing another innovative way to useKinect, in a partnership with the global healthcare company Novartis.

Novartis is a company that provides solutions to the evolving needs of patients, and its researchers are looking for better ways to evaluate treatments for multiple sclerosis, a disease that attacks parts of the nervous system. Multiple sclerosis can result in various complications such as physical, mental or psychiatric problems, and assessing whether treatments are effective in stabilizing a patient’s symptoms can be a challenge.

So, Novartis partnered with Microsoft to create the Assess MS Research Project, which aims to develop a standard set of tests that don’t rely on the subjective evaluation of human doctors. Abigail Sellen, a principal researcher in the Human Experience and Design group at Microsoft’s Cambridge UK lab explained:

The clinicians that we worked with really care about their patients. They really want what’s best for them, and even the best neurologist will admit that when they use these rating scales, it’s pretty coarse-grained. They know that there’s a lot of variability, even in their own judgments, over time.

The Assess MS researchers realized that the computer vision found in Microsoft’s Kinect could give them a more consistent and realistic evaluation of how patients performed on medical tests. Then, if doctors could better assess the progress of the disease, they could eventually give patients the right treatments faster.

For Microsoft, this Kinect use case was an opportunity to leverage its various machine learning tools, but building dedicated algorithms for the project presented a challenging task: to gather data about patients, the Kinect depth camera needed new algorithms to enable it to recognize the patient’s movements in a more subtle and precise way. As Antonio Criminisi, a principal researcher in machine intelligence and perception at Microsoft’s UK lab, points out:

“We had to think hard about better, more accurate algorithms that would really pick up on the subtle signals that differentiate between an MS patient and a healthy individual.”

Another challenge was to create a system that would be easy to use by doctors and patients in the real world and constraints of a doctor’s office. Morrison, a computer scientist whose expertise is in human-computer interaction, explained:

We needed to ask, right from the beginning, how do we make it work in the clinics? It needs to be a collaboration between the patient, the clinician and the technology. We designed the prototype to help the clinicians help the computer, so that they could get the data they want.”

But finally, after years of hard work, the Assess MS Research Project has finally delivered a reliable proof of concept that is waiting to be tested with a broader number of patients, while the team of researchers will continue to tune the machine learning algorithms. Vas Narasimhan, global head of development at Novartis Pharmaceuticals, already believes that Assess MS will be able to provide a faster diagnosis and treatment for multiple sclerosis and maybe other diseases:

“We are excited about our collaboration with Microsoft Research to develop Assess MS, a more consistent way to measure motor dysfunctions caused by multiple sclerosis, which could lead to the development of better therapies and care for patients.”

Microsoft’s Kinect may not be quite as popular a gaming accessory any longer, but it’s still good to see that this technology can find truly useful applications in the real world. And it’s even more interesting to learn about use cases that combine Kinect technology and machine learning algorithms.