Microsoft’s CaptionBot describes pictures so you don’t have to

By: Madeleine Dean
2 minute read

Microsoft has launched an image recognition tool that attempts to describe the content of a picture. This artificial intelligence tool is still in the development stage and is constantly learning from pictures uploaded by users.

As far as accuracy is concerned, sometimes the description is fairly accurate while sometimes CaptionBot offers descriptions that have nothing to do with what is being depicted. There are also cases when the app cannot provide a description at all, owing to its rough-around-the-edges state.

Microsoft designed CaptionBot to learn with more experience, with the expectation that its captions will become more accurate over time. The more pictures users upload, the better the app becomes, as CaptionBot describes itself:

I can understand the content of any image and I’ll try to describe it as well as any human. I’m still learning so I’ll hold onto your photo but no personal info.

CaptionBot uses three technologies to describe what is being depicted in a picture: Microsoft’s Computer Vision, Emotion and Bing Image. The Computer Vision API extracts rich information from images to categorize and process visual data alongside identifying and extracting text from an image. The Emotion API, as its name suggests, analyze faces to detect a range of feelings, everything from anger, contempt, disgust, fear, happiness, neutrality, sadness and surprise. Bing Image searches the web for images.

We tested CaptionBot and the results were accurate in 50% of cases. For example, we uploaded two pictures: one depicting a gaming mouse, the other a stack of card. In both cases, the tool suggested it was a cell phone. On the other hand, CaptionBot accurately detected humans and faces.

Apparently, CaptionBot has an obsession with cellphones. One Twitter user reported the app thought Michelle Obama was a cell phone. For more CaptionBot funny captions, check out this Twitter page.

You can also test CaptionBot here. Do give it a try: you’ll either help the tool improve or you’ll have a good laugh!

RELATED STORIES YOU NEED TO CHECK OUT:

For various PC problems, we recommend this tool.

This software will repair common computer errors, protect you from file loss, malware, hardware failure and optimize your PC for maximum performance. Fix PC issues now in 3 easy steps:

  1. Download this PC Repair Tool rated "Excellent" on TrustPilot.com.
  2. Click “Start Scan” to find Windows issues that could be causing PC problems.
  3. Click “Repair All” to fix all issues with Patended Technologies (requires upgrade).

Discussions

Next up

Best Windows 10 antivirus software to use in 2018

By: Radu Tyrsina
7 minute read

Update – 2018 will soon come to an end and we already have a guide on what is the best antivirus you should get in […]

Continue Reading

These features are out for good with Windows 10 version 1809

iamsovy@gmail.com' By: Sovan Mandal
2 minute read

Microsoft is all set to launch its next big update, Windows 10 version 1809 in October. While that should be a nice piece of news […]

Continue Reading

Windows 10 18H2 builds no longer receive new features

By: Matthew Adams
3 minute read

The Windows 10 October 2018 Update (otherwise 18H2) rollout might now be two to three weeks away. For the last few months, new build previews […]

Continue Reading