Another emerging area of mobile technology to watch out for is computer vision. According to Raja Bala, principal scientist, area manager of Xerox/PARC/Systems Lab, computer vision is capturing images or video with software trying to make sense out of them. Applications of such technology include the recognition of faces, objects, and landmarks with mobile devices. The emphasis is in building software algorithms that can operate fast and in real time on a smartphone.

“Computer vision is making sense out of visual data. It involves capturing and using machine learning to make sense of it,” Bala said. It’s not quite an out of the box solution. There’s a lot that takes place behind the scenes with training the machine-learning component of computer vision to understand objects and even faces.

Mobile device trends in computer vision

Mobile devices is the larger trend in computer vision, Bala said. The camera and sensors in mobile devices are improving with each new device released. Just point your tablet around a room, and it maps out the three-dimensional structure. Google’s Tango Project is one such example of computer vision.

Bala said that computer vision is the intelligent use of all the sensors on a smartphone. For example, a light sensor can be used to tell how much time someone is spending indoors versus outdoors. Other applications for computer vision include depression therapy, lighting correction in smartphone photography, and even an accelerometer for monitoring your activity throughout the day.


The promise of computer vision and mobile devices could have an impact across healthcare, transportation, and retail. Here’s a list of some typical applications of computer vision and mobile devices:

  • Take a picture of your meal and have an app monitor your diet and nutrition intake (Purdue University research).
  • Point a mobile device at your face and read vitals such heart rate and respiration (Phillips and Xerox PARC).
  • Use a mobile device to assist drivers during their journey (Xerox, NVidia, and auto manufacturers.
  • Take a picture of a pair of shoes you like, and an app suggests matching products at certain retail outlets.

Wearables and computer vision

Computer vision will play a part in the future of optical wearables (for example, Google Glass) especially in recognizing activities and interaction. Bala offered the following examples:

  • Use cellphone
  • Open door and go through
  • Handshake
  • Pass an object
  • Throw an object
  • Type on keyboard
  • Wave
  • Write on board or paper

He said, “One interesting application that caught my attention was using wearable vision to helping the visually impaired.” Bala further explained that a blind person goes up to a vending machine. The wearable takes a picture and recognizes objects in the machine. Now the blind person has information about the choices they can make for their next drink or snack.

Major players in computer vision

According to Bala, the major players in computer vision and mobility include big companies like Xerox, Google, Intel, Qualcomm, and NVidia. There are also small start-up firms building specialized software development kits (SDKs) doing interesting things like gesture recognition, and depth sensing. There’s also university-based research and development organizations doing work in the field.

When I asked about Xerox’s entry into computer vision on mobile device, Bala explained that Bala told me it started from work his company was doing in the trucking industry. Xerox created a mobile app for truck drivers to scan bills of lading to speed up the invoicing process. Another project focused on using a mobile device as a driver assistance device for long haul truck drivers. These transportation industry projects eventually grew into an important business for the company and an important technology focus for Xerox PARC in Webster, New York.

While you might not think about Xerox as a mobile company but they are, in the business of transactions which led them into computer vision and machine through using a mobile device as smart sensor to automate business processes.

This YouTube video gives an overview of Xerox’s Computer vision work:

Safe Courier Mobile Solution is a Xerox computer vision project focusing on secure scanning and exchange of documents using a mobile device. Bala gives an overview of the project in the You Tube video:

Towards a computer vision future for mobile devices

With tablets becoming part of more business workflows, we can expect computer vision showing up in future tablet designs. Bala also foresees more smartphone and wearables vendors baking computer vision into their future products.

“Everyone carries a smartphone or tablet,” says Bala. “We are going to see more and more wearable sensors.” He predicts many rich opportunities for computer vision – the problem of capturing visual data and making sense of the world around us from this visual data.