Microsoft Kinect and Xbox One let orangutans play video games

Did you know that animals can play video games? Well, YouTube has a large collection of videos of cats playing with iPads, and it’s pretty funny to see these poor kittens failing again and again to catch virtual fish or mice. But what if animals playing video games could actually be an interesting use case for scientific research?

That’s just what researchers from the University of Melbourne’s Microsoft Research Centre for Social Natural User Interfaces and Zoos Victoria have in mind. But sorry for our beloved little kittens, these researchers are working with orangutans, which not only are our close cousins (they share 97 percent of our DNA) but they’re also pretty ingenious and playful animals.

Around the world, many zoos have been experimenting for some years with computers and tablets to try unusual and fun activities with their primates. But if orangutans seem to really enjoy playing with tablets, the devices are not completely suitable as our curious furry friends could easily drop and break them.

However, Dr Marcus Carter, Human Computer Interaction Researcher at the University of Melbourne, had the idea to use Microsoft’s Kinect 3D sensor and the Xbox One to project an interactive digital space directly onto the floor inside an orangutan’s enclosure. The animal could simply activate the space by moving any part of its body across the projection, and then control any games or applications it would like to challenge their minds.

This “intelligent projection” technology is a world first experience, and in case you’re wondering about safety issues, researchers are actually projecting through enclosures protected by three panels of bulletproof glass. That’s really necessary, as orangutans are three times stronger than us.

The team is now actively developing games, picture galleries and painting applications specifically designed for orang-utans, but designing for animals is a real challenge for scientists accustomed to working with human subjects. Sarah Webber, PhD candidate working with Dr Carter, explains:

“As interactive game designers we use what we call participatory design. We work with the people who use the products and they provide input and feedback for prototypes. You can’t have those conversations with animals, so we have to find new ways to include them as participants in the design process.”

In fact, zookeepers who really know what orangutans are interested in can help researchers to design the perfect games.

“These individual differences are things we can use as inspiration to design something that is complex and motivates them to solve puzzles. We know apes can successfully use touch screens but they are very task orientated, so we want to see if we can devise experiences that are inherently engaging to them,” (added Ms. Webber).

To give you an example, Orangstagram is a selfie app which was designed for Kiani, a female orangutang who loves to look at pictures of herself. And because this app allows her to browse through her picture library, it is opening up the possibility of animal psychologists to ‘read’ the emotional health of orang-utans by analysing what pictures they select.

Could this intelligent projection technology be used with other species? Zoos Victoria’s Animal Welfare Specialist Sally Sherwen explains:

“We don’t just want to stop here. If the trial is successful we could utilise digital enhancement to benefit a range of species, but this will be different according to a species’ preferences.”

So, if you thought Kinect was a dead technology, some animals actually love it! But before we imagine Kinect games for animals on the Xbox Store, maybe the technology could create new opportunities for interaction between the orang-utans and zoo visitors. Wouldn’t it be fun if zoo visitors could play collaborative games with animals? We’ll have to wait for the results of these interesting research to find out.