Visual search

At MozCon 2019, Joelle Irvine spoke about how optimizing images for visual search can improve the overall shopper experience. It was the first time that visual search was presented at the conference.

What is Visual Search?

It’s like “Shazam for images.” Instead of typing in a search query, the searcher provides an image and the search engine displays similar images results. Although not yet widely in use, demand for user-friendly visual search is there: 62% of Gen Z-ers and millennials would like to be able to search by image.

Who are the visual search innovators?

Many retailers, tech and social media companies have integrated visual search into their platforms. However, two stand out. Google and Pinterest have each introduced their own features that enhance discovery and even customer experience through visual search. While Google Lens is focusing on practical applications for everyday use, Pinterest Lens has had success using visual search to enhance discovery, engagement and conversion for fashion and home decor brands.

Pinterest Lens

Many think of Pinterest as a social media network, but the platform markets itself as “the world’s discovery engine.” While Pinterest’s audience may not be as vast as Google or Facebook’s, when it comes to buyer intent and visual search usage, the data can be surprising:

Pinterest visual search data

Why Pinterest makes sense for e-commerce brands

Also, in June 2019, Pinterest announced that they are working on a new feature called Complete the Look where their audience can find complementary items, like finding a handbag to go with a dress or the perfect rug for a living room. It leverages context like an outfit, season, body type, or aesthetics of a room to power taste-based recommendations, providing opportunities for improved online shopping experiences and upselling.

Pinterest Complete the Look

Pinterest’s forthcoming ‘Complete the Look’ feature

The Latest: Pinterest is actively working toward better understanding the content within their pins and the intent of their audience to provide a better overall experience. They are currently focusing on subjective attributes, which could get them one step closer to completing the look

Last month, Pinterest released the home feed tuner, providing its users with more control over what recommendations the algorithm serves up in their home feed. Their engineering team also recently shared that they’re developing a better understanding of the attributes of each pin so that they can match them with associated text queries accordingly.

Google Lens

As of December 2018, Google Lens could identify over 1 billion items, undoubtedly more now. Beyond their shopping, text and auto filters, Google announced two new filters for their Lens at their 2019 Google I/O conference:

  • Dining: scan a menu and get recommendations on best rated dishes; scan a bill to calculate tip and divide the total by the number of diners
  • Translate: scan content in a different language and translated text appears over the image in a matter of seconds; 100 languages are recognized
Google Lens

Google Lens filters

The Latest: Google Shopping is enhancing its platform to rival Pinterest’s discovery and conversion, with Google Lens at the core. Last month, they added a new feature in the US called “style ideas,” where searchers can search for style inspiration with a photo of an article of clothing. They are also adding a personalized home page based on recent purchase activity and interests, a price tracker with price drop notifications, an option to shop online or at local shops, a Google guarantee and customer support. Given the amount of data and images Google has access to, improving the shopping experience is clearly a power move. Not only are they competing with Pinterest, but also with Amazon.

What can visual search do for e-commerce brands?

While it’s not yet standard practice, optimizing for visual search can have major payoff for online retailers.

  1. Increase visibility for lesser-known brands

    It can be hard for people to discover new brands. 97% of Pinterest searches are unbranded and 70% of their audience is open to finding out about brands or products they’ve never heard of before. That’s great news for smaller e-commerce brands.

  2. Leverage existing platforms and partnerships

    If the technology exists, why not use it? Brands can optimize content for a platform where their audience is already like Google, Pinterest, Instagram or Amazon. Alternatively, brands can partner with image recognition tech providers (ViSenze, Slyce and Cortexica are the big players) and integrate visual search into their own platform: Levi’s and Pinterest created the “Find your Levi’s style” tool for personalized curated recommendations, while ASOS enlisted ViSenze to integrate visual search into their platform and provide an easy to use, enhanced customer experience.

    Visual search technology partnerships

    Visual search technology partnerships

  3. Capitalize on impulse buying

    72% of Pinners say that the platform inspires them to shop when they aren’t actually looking for anything. For brands whose audience intersects with the Pinterest demographic, ignoring this could be a major missed opportunity.

  4. Create a compelling in-store experience

    Physical shopping spaces have had a rough few years. Those that have found success have done so by augmenting in-store experiences in creative ways: Alibaba brought fashion AI mirrors into Guess dressing rooms to suggest accessories or different colour options to customers, Amazon created a tech-enabled mirror that projects clothing on customers so they can see what it would look like without trying it on, and Home Depot and Nordstrom have used Pinterest’s scannable Pincodes, programmed to take you to specific Pinterest boards, in-store.

    Visual search in-store experience

    Enhancing in-store experiences with image recognition

  5. Increase revenue

    Gartner predicts that by 2021 e-commerce brands who optimize for voice and visual search will increase profits up to 30%.

Read more: Visual Search: A New Opportunity for Retail Brands

What should retail marketers focus on?

Product image optimization is key. Image recognition technology is not yet perfect; lenses tend to auto-select the wrong focal point, categorize a person instead of a product or display items that are “currently out of stock,” leaving users frustrated. Technical optimization can help search engines index and categorize images through structured data and image sitemaps; leveraging integrations can sync product data via Pinterest Catalogues and Google Merchant Center; on the visual side, having uncluttered images with a clear focal point can help visual search engines identify specific products.

Topic and trend watching—which topics are relevant and of interest to the intended audience? Think beyond keywords: Pinterest’s monthly trend reports, Google Trends, Answer the Public, Google My Business Q&As and customer service reports are all starting points. Use insights to adapt images and associated text so they are timely and relevant to increase discovery and engagement.

Pinterest trend report October 2019

Pinterest trends report

And always be looking beyond the here and now, with an eye to what’s next: 

  • In May, Google announced the addition of augmented reality to Google Search; more recently Wayfair added an “Interactive Photo” feature to their mobile app, upgrading the AR functionality to be accessed for on-the-go use with an existing photo, in addition to its live version. They also improved the overall user experience with an AR powered 3D interactive room planner. Consider the possibilities for online shopping if these features were added to Google or Pinterest Lens. 
  • Several platforms have started monetizing the social experience by including e-commerce capabilities such as in-app checkouts and product catalogue integrations, (social commerce). The latest to venture into this area has been TikTok, which is testing features like “link in bio” (as seen on Instagram) as well as in-video e-commerce URLs, providing brands and creators an opportunity to capitalize on the platform. Image and video visual search are a clear tie-in. 
  • The development of visual sentiment analysis, a domain of AI that attempts to make sense of visual data by classifying it within binary frameworks, could be used to gain insights on the emotions of users on social media. 
  • Think about how visual search can enhance an experience or solve a problem in other industries, whether it be travel, events, real estate, food and beverage, automotive.

Although it’s early on, visual search is becoming more and more sophisticated and user-friendly, making it an increasingly powerful tool for brands and e-commerce platforms. The potential for visually-driven strategies to improve the overall shopper experience will only grow as we continue to see new integrations and applications.

Joelle’s MozCon Presentation:

Are you looking to integrate visual search into your marketing strategy? Bookmark works with brands to develop customized content programs that resonate with their audiences and drive engagement, turning every moment into a compelling and measurable experience. Check out Bookmark’s digital marketing services for more information.