Category Archives: Research

Snapchat wants to use image recognition to send ads, and it could be a little creepy

By | Filters, Image Ads, Image Recognition, Monetization, Research, Snapchat | No Comments
July 19, 2016

One of Snapchat’s best loved features is its photo filters, which use GPS data and augmented reality to add interactive “lenses” to your photos and videos. Now, the messaging startup wants to make that offering more powerful—and lucrative.

A patent application published on July 14, titled “Object Recognition Based Photo Filters,” describes lenses and filters that would be based on the picture you’re taking. For example, if you’re snapping a photo of the Empire State Building, you’d be given the option of a King Kong filter in which the ape climbs the building. The application also outlines how Snapchat could push you a free coffee offer after you post a photo of a hot cup of java.

Snapchat patent shows it'll recognize the Empire State Building, and push a King Kong filter to a user.
Snapchat patent shows it’ll recognize the Empire State Building, and push a King Kong filter to a user. (USPTO/Snapchat)
Snapchat would recognize your coffee in your photo, and send you an offer.
Snapchat would recognize your coffee in your photo, and send you an offer. (USPTO/Snapchat)

Snapchat has 150 million users who send 10 billion videos a day, and they’ve shown no resistance to using sponsored filters. One by Gatorade during this year’s Super Bowl generated 160 million impressions.

But the deep image recognition software needed for the capabilities described in the patent goes further than what’s been offered to date and could make users uncomfortable. Based on the application, Snapchat would be looking at what you’re sending, where you are, and send you advertisements based on that. Snapchat declined to comment on the application.

The tension between a user’s experience and building an advertising business has been a challenge faced by almost every social media company. Facebook and Twitter have had their ups and down, and so will Snapchat. The company is internally projecting sales of $250-$350 million in 2016, and between $500 million and $1 billion in 2017. Snapchat brought in just $59 million in 2015, according to TechCrunch.

Companies file patent applications that go unused all the time, and this patent has not yet been granted. But the bet is on whether or not consumers (especially young ones, like Snapchat’s core demographic) are willing to sacrifice their privacy for fun and potentially useful products. And, for Snapchat, the answer is the difference between being a hip trendy app and the next Facebook.


By | Filters, Research | No Comments
This is an excerpt written by Molly McHugh 
5/24/15 in Wired

IF YOU’RE LIKE most people on Instagram, you’ll scroll through all 22 filters, carefully consider the nuances of Inkwell vs. Lo-Fi vs. Hudson, and then settle on one of the filters you always use. Oh sure, there are  so many filters, but you always go back to your favorites “just because.”

Turns out it isn’t “just because.” There are some specific reasons you rely upon your old faithfuls, and a growing body of science examining how and why people choose filters and how those choices influence others’ reactions to the photo. According to a study out of Yahoo Labs, researchers looked at 7.6 million Flickr photos (many of which originated on Instagram and were uploaded to Flickr) and found “filtered photos are 21 percent more likely to be viewed and 45 percent more likely to be commented on.”

This study is but a drop in a fairly shallow pool: Despite mobile photography’s massive popularity, it’s been largely ignored by academics. “There is little work—scholarly or otherwise—around filters, their use, and their effect on photo-sharing communities,” the Yahoo Labs study explains. That’s due in part to photos being harder than text to analyze, but that shouldn’t be an excuse anymore, especially given the active commenting community on Instagram and other social media.

The Yahoo Labs team is not alone in its fascination. Researchers at Arizona State University have been studying Instagram and its filters since last year. “We were (and continue to be) motivated by the fact that Instagram has received very little attention from the research community,” says one of them, Subbarao Kambhampati “We believe that a careful analysis of Instagram can give us a valuable window into our collective online behavior.”

The Yahoo Study focused specifically on filters, and found people like higher contrast and corrected exposure, and find a warmer temperature more appealing than a cooler one. “Serious hobbyists” use filters only to correct a problem—say, correct the exposure. “More casual photographers” are more likely to manipulate their images with filters or adjustments that make them appear more “artificial,” according to the study.

Example of an original photo (top left photo) and many filtered variations. Some filters change contrast, brightness, saturdation; some add warmth or cool colors or change the borders.

One particularly interesting part of the research examined just who’s using Flickr. A few years ago, the “Flickr vs. Instagram” debate could be cast as “real camera vs smartphone camera.” That’s no longer the case. The iPhone rules all.

“The iPhone has been the most popular camera for years now,” says David Ayman Shamma, one of the Yahoo researchers. The proliferation of the iPhone and smartphones in general has lead to photography of all kinds becoming a creative outlet for millions. “It’s been the dream since the Kodak Brownie. With it, comes the creative space for many outlets and photographers, from people shooting on vintage film to food bloggers.”

Shamma says profiling Flickr users is a more complex task, because so many people use it for so many things. “Some people on Flickr only want to push their best portfolio pieces from DSLRs, while others publish their daily lives from their iPad camera, and many do a mix of cameras and content.”

No matter what you shoot or where you post it, you will be heartened to hear filter snobbery is dying. Pro shooters who once sneered at Instagram obsessives and their love of Rise, Mayfair, and X-Pro II aren’t quite as judgmental as they used to be (or, perhaps, as we only thought they were). These days, everyone uses filters.

“One of the surprising things to me was the pro set was talking fondly about the filters,” Shamma says. “Not that I thought they’d be snobby and elitist before the study, but I assumed they’d rather use their software tools of the trade over a one-click filter.” If you don’t want to say it, it’s OK, I will: I thought they would be snobby and elitist.

Of course, it’s impossible to talk about filters without talking about Instagram, because it is the world’s most popular photo app (it’s one of the world’s most popular apps, period). It seems there could be a difference between people who use Instagram to take and manipulate photos and those who do so with Flickr, or someone who uses Flickr to edit a photo and then cross-posts it to Instagram (or vice versa).

“There are similarities and differences for sure and we can see them by looking at what’s uploaded to Flickr via the Flickr app and what’s uploaded to Flickr through Instagram,” Shamma says. “The nature photos on Flickr from Instagram show more engagement when they are filtered, so it’s a function of what sub-community you’re speaking to.”

But Shamma says there are unifying factors when it comes to filters, regardless of platform or skill level. Could someone, therefore, use this research to design the perfect filter? In a word, no. “As awesome as something automatic might sound, there really is no silver bullet here,” Shamma says.

That’s because there’s more at play than how a filter looks. In many cases, the act of choosing the filter is equally important. “When we interviewed people for this study, we found that the photographers, regardless of skill level, enjoyed the process of selecting a filter,” he says. This explains why people painstakingly scroll through them all before invariably choosing a favorite. Even if someone could engineer the perfect filter, people wouldn’t want to lose out on seeing their photos transformed by all those filters. The element of choice, the function of looking and choosing, is one reason people so love filters in the first place.

Instagram filters

In case you were wondering, another study, in 2014 by Arizona State researcher, identified the most popular Instagram filters. They are, in descending order of popularity, no filter, Amaro, X-Pro II, Valencia, and Rise. Seeing “no filter” is a bit of a shock, given what Yahoo’s study says, but what’s most interesting is that the most popular filters may be the most popular because everyone thinks they’re popular. “These top five filters are actually present in the first seven filters of Instagram GUI at that time,” Kambhampati, explains. “This brings up the possibility that the (accidental?) placement of filters has more to do with their eventual popularity than any conscious photographic choice by the Instagram user.” The same study also found that there are generally only a few categories of Instagram photos: friends, food, gadgets, quote pics, pets, activities, selfies, and fashion.

Instagram categories

The researchers are still digging into Instagram, and plan to look at how the social network diffuses information and just what makes an image “go viral.” Such questions have been asked of other social networks, but not Instagram. “We have come up with a number of indirect measures to study diffusion (in particular, by studying the number of ‘likes’ and comments received in terms of the number of hops separating the liking and commenting user from the posting user.” The researchers soon will present a paper showing how we can glean sentiment from Instagram images with the help of “image features and the features from the textual comments.” Perhaps someday soon, we won’t simply know what Instagram filters are popular, but also how they make us feel.