Connecting with beer lovers, Kinect-style

A lost puppy and his Clydesdales stablemates may have commanded the advertising spotlight during Super Bowl XLIX, but for our money, the real marketing magic from brewer Anheuser-Busch was on display at the “House of Whatever,” a gigantic tent set up for three days outside the stadium. Inside was a huge bar, behind which hung a Kinect v2 sensor oriented toward the crowd of thirsty football (and beer) fans, with a large video screen above it.

As patrons walked into view of the sensor, the screen served up signage asking, “Can we can interest you in a drink?” Stepping up a little closer, the fan was presented onscreen options among the freshly poured, free glasses of Anheuser-Busch beers sitting on the bar. As the thirsty patron happily picked up a beverage, the screen displayed the choice, along with anonymous age and gender analytics of all visitors that day and a pie chart showing which beers had been the most popular.

The Kinect v2 sensor recognizes patrons as they approach the bar, prompting the application to offer them a beer.
The Kinect v2 sensor recognizes patrons as they approach the bar, prompting the application to
offer them a beer.

The patron was then offered a chance to raise the glass and say “cheers,” at which point the Kinect sensor captured the image and displayed it onscreen. The fan could then retrieve that photo using a QR code and Instagram it with hashtag #getkinected, in order to be in the running to win a new Microsoft Surface Pro. 

The Kinect-enabled system was developed by Microsoft and incorporates world-leading biometric technology from NEC, which uses face recognition and measures the age, gender, and total headcount of patrons. NEC and Microsoft have been working together closely on this new breed of interactive retail systems, which offers a compelling shopping experience for customers and invaluable backend demographic and engagement data for the retailer. A version of the system was displayed earlier in January at the National Retail Federation’s (NRF) annual event in New York City.

The system can even recognize previous customers—provided the retailer has obtained express permission to store the customer’s facial image, as, say, part of a loyalty program. This feature allows the retailer to serve up ads and offers that tie directly to that patron’s past purchases. (It also recognizes store employees, allowing the system to ignore their presence.)

Puppies and horses may engender warm feelings, but the customer analytics enabled by the Kinect-NEC system can generate on-the-spot sales and one-to-one customer interactions.

The Kinect for Windows Team

Key links