SceneTap’s patent-pending ID tech sounds even more like ‘Big Brother’

When you walk into a bar, it’s a given that the regulars are scoping you out. But you might feel differently if you knew that automated cameras were recording your apparent age, gender, ethnicity, and even level of attractiveness.

SceneTap is a controversial Chicago-based startup that helps people gauge the size, mood, and demographic makeup of the crowd at local bars. Now it’s aiming to patent fascinating and potentially invasive technology that can detect all the attributes mentioned above, plus your weight, height, and income level, among other things.

SceneTap first launched in July 2011 and makes sensors, cameras, and facial detection software that venues install to scan crowds and get back anonymized data. The data is repurposed for use by venue owners and would-be customers alike. SceneTap already has a presence at bars and restaurants in several U.S. markets, including San Francisco, Boston, Chicago, and Atlanta, where it records mostly benign, anonymized data, such as a crowd’s size, average age, and gender mix.

SceneTap markets its offering as a positive way for consumers to assess a venue’s scene before heading out for the night, and as an insightful tool for business owners looking to better understand their patrons. But SceneTap’s technology, specifically its mostly unknown detection capabilities, has come under fire from bar-goers unsettled by the implications of being digitally sized-up. Some bars have even pulled the system as a result of backlash from customers.

Those nervous patrons might be even more nervous once they read SceneTap’s patent application entitled, “Apparatus and method to record customer demographics in a venue or similar facility using cameras,” filed with the United States Patent Application on December 13, 2011 and published on June 14, 2012.

An anonymous tipster sent a copy of the patent application to VentureBeat, but the contents are publicly accessible from the USPTO’s website and are embedded below.

“The IP involved in these patents is merely about creating a barrier to entry for the business, not about what SceneTap will or will not do,” a SceneTap spokesperson told VentureBeat.

The patent, filed by SceneTap founders and inventors (Joseph) Cole Harper and Marc Scott Doering, outlines a very specific system capable of scanning a face to determine far more detail than the company has claimed so far. “We have no intention on doing 99 percent of what the IP provides in the near-future, if ever,” Harper said in a statement via email.

However, the patent also describes how data collected would be used for consumer-facing mobile applications, analysis products for venue owners, and marketing reports targeted at third-party advertisers.

In other words, after the cameras size you up, an offer for a discounted drink might pop up on your phone — but only if you meet a certain level of attractiveness.

The patent application describes and illustrates example processes for using video, audio, motion detection devices, laser-based, and radio frequency tracking devices, to determine the traffic flow and demographic breakdown of people at bars, restaurants, nightclubs, movie theaters, malls, schools, and so forth.

The demographic information may include a total number of people currently at a venue, a percentage of capacity filled for a venue, a ratio of males to females, an average age of males and females, a ratio of hair colors of customers, an approximate income level of customers, approximate percentages of race and/or ethnicity at a venue, approximate averages of height/weight, a percentage of people with glasses and/or facial hair, general descriptions of clothing type (e.g., jeans, skirts, sport coats, dresses), and/or general indicators of attractiveness.

How exactly the company will go about determining these characteristics is spelled out in the ensuing sections. Attractiveness, for instance, can be assessed by analyzing video images of customers to determine their measurements and then comparing those measurements against traditional standards of beauty.

“The local server 114a may use software that applies an array of measurements on geometry and symmetry to a face of the customer. The local server 114a measures proportions between eyes, nose, ears, and lips and references these proportions to a corresponding level of attractiveness,” according to the patent application. The numbers included are meant to signify servers in the associated illustrations. “The attractiveness levels are averaged and an overall or cumulative attractiveness grade is determined and displayed for the venue.”

The system may even include microphones or sensors to detect audio. The microphone can record crowd noise, loudness, laughter, talking, yelling, and music.

The document also addresses highly sensitive data types, such as a patron’s relationship status, intelligence, or education level, and explains that this data could be generalized for the venue but not displayed to the public. The inventors even suggest that some sensitive data collected, such as identifying people with criminal records, could be used to alert venue owners or authorities.

While Harper has previously insisted that SceneTap would not use facial recognition technology, the patent goes into great detail as to how the startup could use its detection technology in conjunction with identity networks and facial recognition software from companies such as Facebook and Google to “actually identify a person.”

In an embodiment, the facial detection software uses algorithms to determine what a customer looks like through physical characteristic analysis or through a matching program that utilizes existing data to match a recorded facial or body image to generic faces or body types stored in a database. The facial detection software determines, for example, that a customer is a 28-year-old male. The facial recognition software uses image databases (such as Facebook or government databases) to match a recorded image to an image in one of these databases to determine an identity of a customer in the image. In this example, the facial recognition determines that a customer is, for example, John Smith.

The patent application is embedded below.

Photo credit: penguinpanini/Flickr