Microsoft outlines how face recognition can be integrated into a Universal Windows App

The developer-focused blog posts continue from the Windows App Team with their recent look into the FamilyNotes sample app. The application is a sticky note style, meant to be used between family members to keep everyone the know easily. While the app isn’t available on the Windows store, the GitHub repository sample can be accessed and toyed with by developers for a better understanding of the upcoming Windows Anniversary features such as ink, speech, and face recognition.

Admittedly, the introduction to face recognition through camera features introduces a list of new concerns, many of them centered around privacy and the utilization and storage of images in relevant apps. In today’s blog post, the Windows Apps Team went into more depth concerning the FamilyNotes sample app and how developers could use this feature in their own projects safely and effectively.

Privacy being the key element to any face recognition coding, FamilyNotes keeps the user informed anytime that the camera is about to capture an image or submit it to an external service such as the Microsoft Cognitive Services Face API.

For developers to begin building apps that enable the use of Microsoft Cognitive Services, they will need to sign up for a free trial subscription to receive the MCS Face API keys. Afterward, they will be able to set up a FaceServiceClient through a single line of code, enabling interaction with the API for user identification. For FamilyNotes, this means that every image will need to be collected and stored for each user, then the app will recognize when an identified user is in front of the camera, take the picture, and verify it across the stored images so that it can show the proper information.

The MCS Face API has three different methods of supporting this: identification, similarity, and verification. Through identification and similarity, the MCS compares the images to those that have been previously stored. However, with verification, the app will submit two images and then compare them to ensure they are the same. The sample FamilyNotes app uses the similarity method for less service transactions.

Once the collection has been established, the app can begin to determine if there is an identified user present in front of the camera. Much like the Microsoft Magic Mirror shown off at InnovFest Unbound 2016 in Singapore, the use of the camera is kept in the background to offer a seamless experience automatically as the users are identified. That is thanks to the face recognition not using the CameraCaptureUI, but instead using the MediaCapture API. The FaceDetectionEffect is created and added in the MediaCapture object through coding, essentially letting the camera capture a stream of images as fast as every 33 milliseconds to compare to the stored facial images.

Applications that implement face recognition should account for occurrences such as multiple users in front of the camera or how many submissions are available through the use of the free MCS Face API account. Simply put, the app shouldn’t constantly be streaming video for an attempt at face recognition. For FamilyNotes, the app limits the recognition only to when a user is in front of the camera and also applies a 10-second limit to the process. Adjusting filters and scenarios to determine the usage of the API has multiple solutions, where developers can truly implement the styles they prefer within reason.

Want to learn more about how to develop with the use of face recognition? Visit the full Windows blog post.