Is personalized image search the future of Bing? Microsoft thinks so

Microsoft has always been trying to make Bing Search a search engine popular enough to compete with other similar products on the market, such as Google. But, ultimately, it always fails, and while Bing remains the second most popular search engine after Google, it is light-years behind it, according to the latest statistics.

Microsoft’s CEO, Satya Nadella, recently said that he’d do just about anything to make Bing better, according to a report from The Verge, and while the Redmond-based tech company should have done this for years, indeed, if it wanted to make Bing a leading force on the search engine market, it seems that it might start to take the matter into its hands and come up with a solution.

This solution is apparently a new way of searching: a personalized image search, which Microsoft filed a patent for back in 2022. However, it was earlier this year that the patent was published. The patent describes a system that uses a method that employs user profile information and activity to look for and suggest other images that the user might like and be interested in.

In shorter words, the method offers a highly personalized image search. Could this be implemented in Bing? The patent definitely points out clues in that direction.

Personalized image-based search on Bing: how would it work?

The patent, Web-Scale Personalized Visual Search Recommendation Service, describes a system that offers personalized image-based search results.

This is how it would work:

1. The system receives a search request that includes an image.
2. It creates a personalized item embedding (a mathematical representation of the image) based on the image and user profile information.
3. It gets a list of candidate images based on the personalized item embedding.
4. These candidate images are ranked according to how likely the user would engage with them.
5. The system also makes sure there’s visual diversity among these ranked images, so they don’t all look too similar.
6. Finally, some of these diversified images are returned as the search results.

The patent doesn’t specify if the personalized image-based search system is meant for Bing, but it’s definitely a technology that only a search engine can implement.

Visual Search is a popular research area and is of great interest in the industry. Given an image, a visual search system can return a ranked list of visually similar images. Accordingly, a personlized visual search system is needed for providing images to users in a more personalized manner.

It’s good to note that Bing already has a visual search option, but it’s nowhere near the level of complexity described in the document.

But if that is not enough, Bing Chat, which is the Bing AI tool, also has a fairly complex image-based search feature, and generator as well. Users are able to upload their own images, and Bing AI will generate its personal input, and even create similar images. Giving the context length, this feature could also be taken as personalized image-based search, but limited.

If Microsoft wants to compete with Google, the company needs to bring in new features that are actually useful to customers. This could be one of them, however, there would still be a long way to go before Bing Search catches up on Google Search.