blog post image

When there are no words…

As we continue to communicate with images more, the case for visual search is building, especially in retail. Recently, Amazon’s Firefly app has made the most impact in visual searching, using image recognition and search prompted by image-based queries. But other companies, most profoundly retailers and technology-based brands, are also exploring avenues to develop visual search. 

As Adi Phinas, founder of image search company Superfish said: “Visual search is for the cases in which I have no words to describe what I see … [it's] not here to replace the keyboard.”


While millions of people use their smartphone to search for products while shopping every day, text-based searching is not necessarily the best medium for mobile technology as it is smaller and without full-size keyboards.

That’s where apps like Amazon’s Firefly come in. With the launch of its Fire smartphone earlier this year, Amazon also debuted Firefly – an app that allows users to identify, and then buy, items seen in the real world. It works by utilising the phone’s camera to identify books, movies, food and games, among millions of other products, and then direct the user to Amazon to purchase the item. It also includes an audio-recognition component similar to the Shazam app that identifies sound bites. But Firefly takes things one step further: it can identify specific scenes from shows, access that scene and direct the user to Amazon to watch that scene. The user is then prompted to purchase the song, show or movie.

The app can also recognise art and then direct the user to the associated Wikipedia entry – a function, which at this stage is still unique to Firefly.

Screen Shot 2014-11-12 at 9.38.45 PM

Perhaps Amazon is the most advanced in the field of visual search because it understands that identification is the first step towards making a purchase, a fact vital to retailers as this technology develops further and comes closer to being adopted by the masses.

Google’s ‘Search by Image’ utilises a similar concept allowing users to either upload an image, or drag and drop one. It then identifies similar images from the web, including URLs to related images, and the same image in different sizes.

Superfish is a visual search company whose tagline is “making images the starting point to exploration and discovery”. “Through cutting-edge, patented technology we have developed a visual search engine that analyzes images algorithmically and transforms the way images are searched, seen, utilized and shared over the web. Quite Simply, we are opening up a new way to search,” its website states.

The company has developed various image search tools, including Like that Decor, that allows users to use their smartphone camera to identify household products and PetMatch, which uses visual search technology to identify adoptable pets in a local area.

Screen Shot 2014-11-12 at 9.43.46 PM

According to researcher eMarketer, the question around visual search for retailers is not if, but when, predicting it to become a mainstream tool for retailers within the next two years. “But are consumers clamoring for a way to find items based on pictures, instead of words? Retailer interest in visual search reflects their understanding of the changes brought on by mobile shopping,” eMarketer says. “Millions of Americans use their smartphones while shopping every day. But the current search experience is not ideally suited for mobile.”

“It’s not as easy to type on a mobile phone as it is on the desktop with keyboard layouts being the way they are,” said Brad Folkens, co-founder and CTO of image recognition firm Image Searcher.

However, the real potential for visual search reaches far beyond typing issues. It opens up a whole new realm of possibility for retailers.

Are you utilising any form of visual search? How would it aid your brand/business if you were to use it?