In June 2018, Microsoft announced that it was launching new and improved intelligent Visual Search capabilities for its search application, Bing. The new technology allows users to more successfully search the internet, and the world wide web for information using just the images captured by their smartphone’s camera.

The widespread adoption of mobile devices by businesses has made photography, imagery, and other forms of visual information much more important for efficient collaborative communication within the workplace. It only seems logical that search engines be able to accept that visual information as input.

SEE: Digital transformation research report 2018: Strategy, returns on investment, and challenges (Tech Pro Research)

The new Visual Search capabilities are available on the Bing app for iOS and Android, and for Microsoft Launcher (Android only). The new capabilities began rolling out on Microsoft Edge for Android in June and will be available on Microsoft Edge for iOS and soon after that.

This tutorial shows you how to download and install the Bing app onto your mobile device and then how to use the built-in camera to input images as a search parameter. The example device is a Moto E 4 running Android 7.1.1, but the procedures outlined should be similar for any mobile device with a camera.

Visual search

To download and install the Bing app, navigate to the online store that is specific to your device’s operating system. For Android devices that would be Google Play. Search for the Bing app, and then click the Install button (Figure A). The application is available for free.

Figure A

Once installed, tap the icon displayed on your device to start the Bing app. You will likely have to answer a few prompts to allow Bing to know your location and grant access to your camera. On the main screen you will see the familiar search icon where you can enter text input. Next to that you should also see a camera icon (Figure B), tap that to start Visual Search.

Figure B

When you tap that camera icon, your device will switch to camera mode, and the next photo that you take will be used as search input for the Bing app. Take a picture of a product on the store shelf, a landmark, or some other recognizable object, and Bing will scan it and return related and similar imagery.

If you select barcode-mode, Bing will translate it into a product name and search for results. In Figure C you can see the results for a barcode search for a jar of horseradish sauce, complete with suggestions for where I can purchase a replacement.

Figure C

SEE: Mobile device computing policy (Tech Pro Research)

Intelligence limits

Take note that there are still some noticeable limits to the intelligence of Bing’s Visual Search feature. For example, I snapped a photo of my cat and used it as search input for the Bing app. The results showed me many images of cats with similar markings to my cat’s, but it did not return any results from my social media accounts, which surprised me since I had featured my cat in many past posts.

So, while using Bing Visual Search may be useful for many occasions, there is always the distinct possibility that you will have to resort to text-entry to find the most relevant information about a subject. While the technology of Visual Search is improving, it is far from perfect.


Your thoughts:

Does your business share more visual information than it used to? How are you managing that data? Share your thoughts and opinions with your peers at TechRepublic in the discussion thread below.