Are you happy? Sad? Angry? Terrified? Microsoft knows…

By Bill Carmarda as written on nakedsecurity.sophos.com

Is something troubling you?

No, seriously, we saw you walking by, and you looked like something was on your mind. Something bad.

Well, it wasn’t exactly “us.” Our webcam caught you in a crowd. Then, our cloud-based face detection service flagged your mood. We don’t quite know what it is – we’re working on that! – but something’s bothering you.

Maybe we could help? To show you how much we care, here’s an ad for something to make you feel better…

Behold the next new power of the cloud!

This week, Microsoft announced the free public preview of Azure Media Face Detector, the latest in its growing set of speech and vision services running atop the Microsoft Azure cloud.

Microsoft’s core Face Detection service can detect and track up to 64 faces concurrently as they move around within an .MP4, .MOV, or .WMV video frame.

Once faces have been detected, Microsoft can pass them to its optional Emotion Detection component. That’ll analyse “multiple emotional attributes… including happiness, sadness, fear, anger, and more.”

As The Register noted, audio-based sentiment analysis tools are already in production helping call center reps figure out just how irate you are; now they’re coming to video, too.

Microsoft says Azure Media Face Detection can be used for “people counting, movement tracking, and even gauging audience participation and reaction via facial expressions.” If you’re using their neatly JSON-formatted data, you can instantly tell if a crowd likes what it’s seeing in your store window… or what it’s hearing from your political candidate.

Security isn’t mentioned as an application, but Microsoft may well be thinking about such matters. Just last week, Rudiger Dorn, Microsoft’s Director of Cloud Computing, told TechWeekEurope that Microsoft’s AI and machine learning Platform-as-a-Service (PaaS) offerings will offer significant competitive value in “allow[ing] you to track criminals and terrorists.”

As with most “early days” technologies, Microsoft’s service has plenty of limitations. It does best with faces looking straight at the camera. It can lose track of faces that slip in and out of frame. And perhaps most significantly, this specific “Media Processor” doesn’t put a name with a face.

For that, you’ll need to link Face Detection with Face Recognition. But, of course, those technologies are coming along quite nicely, too. Microsoft was recently promoting face recognition as part of the same Project Oxford research that’s being “productized” here.

Microsoft isn’t the only outfit diving into this space. Google’s Cloud Vision API also promises face detection and Image Sentiment Analysis to recognize emotions like joy, sorrow, and anger. Per Google:

Combine this with object detection and product logo detection, so you can assess how people feel about your logo.

(How they really feel – not what they told your focus group!)

Whether you’re a Microsoft or Google partisan, you’ll still be able to run these services at enterprise-scale.