Microsoft Research studies how to use web search data to diagnose serious illness

Pancreatic Adenocarcinoma, or Pancreatic Cancer as it’s commonly known, is the fourth leading cause of cancerous death in the United States. As with most illnesses, the most successful way to fight against it is through early intervention. Unfortunately, many do not even realize they are sick until it’s too late. That is why a small team from Microsoft Research has been researching into crowdsourcing information for medical research and early detection.

Eric Horvitz, Ryen White, and Microsoft intent John Paparrizos from Columbia University have been looking into using search engines to increase early detection for users. Similar to the story of Yom-Tov collecting data about flu cases and vaccines that we covered a few months ago, this Microsoft Research team has used a collection of anonymized data from Bing searches to identify symptoms of pancreatic cancer.

The cancer is easy to miss through a list of very subtle signs:

• Itchy Skin
• Weight Loss
• Light-Colored Stools
• Patterns of Back Pain
• Yellowing of Eyes and Skin

By following the list of symptoms closely, the team are able to find individuals that are asking questions or looking up symptoms. Tracing through the anonymous user’s prior searches makes it easier to understand more about the illness and to identify connections.

The results can be striking:

“We find that signals about patterns of queries in search logs can predict the future appearance of queries that are highly suggestive of a diagnosis of pancreatic adenocarcinoma,” – the medical term for pancreatic cancer, the authors wrote. “We show specifically that we can identify 5 to 15 percent of cases while preserving extremely low false positive rates” of as low as 1 in 100,000.

While Microsoft Research doesn’t have plans to move forward in developing products linked to the study, the team feels confident in their ability to make a change anyways. Horvitz particularly feels committed to the effort, given that he lost both a childhood friend and a close colleague to the pancreatic cancer. “People are being diagnosed too late,” he said. “We believe that these results frame a new approach to pre-screening or screening, but there’s work to do to go from the feasibility study to real-world fielding.”

“People are being diagnosed too late,” (Horvitz) said. “We believe that these results frame a new approach to pre-screening or screening, but there’s work to do to go from the feasibility study to real-world fielding.”

The three men sincerely hope that the medical community will work with this study and present it in a way that can significantly improve early detection to not just pancreatic cancer, but many illnesses going forward. The feasibility study publication can be read at the Journal of Oncology Practice.