Security

How facial recognition software can track you in the offline world

Patrick Lambert looks at the increased use of facial recognition software.

mobilerecognitionscrenshot.jpg
 Credit: Viewdle
Most people are now fully aware of how much tracking is going on when you go online, and how easy it is for companies and governments to know where you go, what you do there, and who you talk to. But meanwhile, another piece of technology has slowly started being adopted by organizations all over the world, and is still relatively unknown by most people, yet could be the key to bringing the type of surveillance state that we currently experience online, to the offline world.

Let me give you a little anecdote of something that happened to me last month. I walked into a local bank branch near where I live in Montreal, a bank where I don't have any account, and never interacted in any way, in order to ask about some investment products. I get directed to one of the staff members and I talked to her for about 10 minutes, never saying my name or giving her any way to identify me. I ended up not being interested in what she was offering and walked away. But as I did, the last thing she said was "Au revoir, Mr. Lambert." That's nice but how did she know who I was, despite the fact that I never gave any hint of my identity?

I didn't think much of it at the time, but after getting back home I did some research and found articles from a few years back about some banks here in Canada contracting with a company to provide them with facial recognition cameras. Going over the incident in my head, this seems like the most likely explanation. Now one argument people could make is that banks already have cameras, lots of them, and so do stores and buildings all around our modern towns. But I would argue that facial recognition brings this to a whole new level. What we're basically seeing is the same thing that happened years ago with phone calls.

Back in the day, when you picked up a telephone and placed a call, it would go along an analog circuit and the wire would need to be physically plugged in. We knew it was possible for someone at the central office to listen in, but the chance was remote, and there was no way they could start listening to every phone call as they went on. This is the same way normal cameras function. The best refinement the industry has managed to do is divide several cameras into a single screen, and then place multiple screens in front of a single security guard. But facial recognition completely changes the rules of the game. Now, we have software programs able to recognize anyone who walks by instantly, and record that in a database.

Facial recognition systems can work in a number of ways. The traditional system uses a standard camera and software that can compare some features of your face to a database of photos. By looking at the size of your nose, your eye position, or the slope of your jaw, it may be able to figure out who you are. A newer and much more reliable way to do facial recognition is to use multiple cameras and do a 3D comparison. This way, many more features of your face can be compared with what's in the database. Finally, some systems can even identify skin patterns such as lines, moles, spots and other characteristics found on most faces.

But how widespread is it really? This technology is nothing new. Back in 2004 the US State Department began implementing a huge facial recognition system for visa applicants. The New York Times reported last month that the Department of Homeland Security is in the final stages of developing such a system as well. The FBI is also spending over $1 billion on its own facial recognition system. Recently, it was revealed that the Attorney General of Ohio already implemented such a system back in June, without telling anyone. The system relies on 10 years of information from driver's licenses, gun permit holders, known criminals, sex offenders and more, all stored in a central database. The technology is also widely used in advertising. This video is just one such system which can access 36 million users within 1 second.

So we know the technology exists, and has become very refined. We also know corporations and governments are hard at work to implement it for their own purposes. The problem is that as with most things in technology, the law is lagging behind. There is no legal framework as to how facial recognition can be done. We're all used to seeing cameras everywhere we go, but the question that needs to be answered sooner rather than later is whether we're alright with these cameras automatically identifying us, recording everywhere we go, and then acting on this information, providing anything from personalized ads on a nearby billboard to a complete tracking profile at your local government office.

Have you had any experience implementing or using this kind of software? 

About

Patrick Lambert has been working in the tech industry for over 15 years, both as an online freelancer and in companies around Montreal, Canada. A fan of Star Wars, gaming, technology, and art, he writes for several sites including the art news commun...

17 comments
jackee.c
jackee.c

Google is using it. To unlock my android tablet, all I have to do is look at it. Never mind that google tells me if the traffic is delayed on my way to work, or that the item I ordered has been shipped (they even know the item and provide a picture) despite the fact I didn't purchase it with my tablet or ask for any ofthat info. We are so busy watching NSA, who is watching Google?

LuisRDuran
LuisRDuran

I think this is not something new to the IT world as you mentioned. For many years Governments and organizations around the world have hidden information and technology so we do not know what information they get from us. I would say facial recognition has been used for a while however I am hesitant that it was used in this case. I am sure that this type of software is used in many ways by banks and other kind of organizations to detect who the person they are seeing is. Let's face it, the day when somebody with the right technology is able to see where we are and who we are has arrived.

H4CKN3T
H4CKN3T

I don't think facial recognition was used here either, it sounds like you probably had a bit too much to drink at last year's Christmas party and pulled a one night stand with this broad, she was going to ask for a repeat, but you left to quick because you were being paranoid about the cameras... As far as tracking goes, this is what I found on this particular site while reading this particular post:
ClickTale
Analytics
Google Adsense
Advertising
Media Optimizer (Adobe)
Beacons
NetRatings SiteCensus
Analytics
New Relic
Analytics
ScoreCard Research Beacon
Beacons

3rd Coast Geeks
3rd Coast Geeks

The proverbial double edged sword. It is awesome in its own right. The potential for abuse by government entities is down right scary.

ahanse
ahanse

How else are we to separate the sheep from the goats?

The Profiling that is done by governments and big businesses can be enhanced by facial recognition and is the way to go. What we need to do is encourage them with great gusto to have a central repository of collected data so individuals can see what is collected and challenge any discrepancies.

Photo IDs have been around for awhile now and no-one has reacted against them except they never like the result. Real time recognition just adds another dimension to the mix. It has its advantages and like all thing it can be abused.

 
dhubbard1500
dhubbard1500

Spooky...  just plain spooky.  Unfortunately, in this day and age of nobody trusts anybody...   et al ....  go with your gut...    

info
info

yawafrifa2000: You're thinking small. This is the age where people get fired because their employer found out they went out with some friends for a drink on a Friday night. This goes far beyond the bounds of 'criminal' behaviour...

jemorris
jemorris

I hope doesn't post a second time, browser seemed to stick in an endless loop...

After 9/11 there was and still is a big push for facial recognition on most public cameras. Seems like it was the second or third Super Bowl right after that where I read they had recognition software running and cameras watching every entrance. The prime reason was supposedly to watch for terrorists but they also used it to capture a large number of people with outstanding felony warrants. I don't remember what the the actual rate was then but to me it seemed phenomenal as to the number of faces per second that system was able to process and successfully recognize, I am sure the software has improved and we know that the hardware has definitely progressed. 

Rioch
Rioch

Since it was a local branch of the bank, is it possible that the staff member recognized you from the real world and was just being friendly?  Not all human interactions are technologically based, something we techies sometimes forget.

yawafrifa2000
yawafrifa2000

This technology has come to stay and will see much improvement over time. Whether we like or not, security agencies and other departments will continue to use it. But will it cause any worries to individuals? Well, I believe only the criminally-minded need feel uncomfortable. For if you have nothing to hide, you could be investigated, like EJK wrote, analyzed and discarded as a harmless person.

EJK
EJK

This technology is hardly as limited as you describe. It could easily be extended to search anything published on the web, or available as unsecured webcam (millions of those out there). You could be investigated, evaluated, and discarded as a potential employee without ever knowing the process ocurred. The same could apply to credit applications, promotions, etc.

alan.stanton
alan.stanton

Television and the movies offer us a glimpse into the near future if not the present. A few decades ago, the James Bond film "Goldfinger" showed a GPS-based device that allowed the user to track, via a moving blip on a video screen with a map, a vehicle that had a homing device attached. An issue of Product Engineering, a professional trade magazine, shortly after the movie came out reported that this device was no science fiction but in fact reality at that time. Now all of us can buy cars that have screens that show thier own whereabouts and how to get somewhere else. Star Trek's communicators became the cell phone, and NASA began working on creating their own versions of the medical tricorder and monitoring beds. The very popular U.S. television series N.C.I.S. features the use of facial recognition software in many of its episodes. And the current U.S. television series Person of Interest shows us the immense power of video monitoring and facial recognition systems. In fact, the second episode of the current season, which aired last night, featured a fictional company called LifeTrace which combines every bit of data it can amass from all of the commercial monitoring and tracking systems out there to create very intimate profiles of individuals. Do you like getting a grocery discount by using a shopper's card? You're paying with your discount for a profile of yourself.

alan.stanton
alan.stanton

I have no experience implementing or using, but I do believe that television and the movies offer us a glimpse into the near future if not the present. Going back a few decades for an example, the James Bond film "Goldfinger" showed a GPS-based device that allowed the user to track, via a moving blip on a video screen with a map, a vehicle that had a homing device attached. An issue of Product Engineering, a professional trade magazine, shortly after the movie came out reported that this device was no science fiction but in fact reality at that time. Now all of us can buy cars that have screens that show thier own whereabouts and how to get somewhere else. Star Trek's communicators became the cell phone, and NASA began working on creating their own versions of the medical tricorder and monitoring beds. The very popular U.S. television series N.C.I.S. features the use of facial recognition software in many of its episodes. And the current U.S. television series Person of Interest shows us the immense power of video monitoring and facial recognition systems. In fact, the second episode of the current season, which aired last night, featured a fictional company called LifeTrace which combines every bit of data it can amass from all of the commercial monitoring and tracking systems out there to create very intimate profiles of individuals.

Khulud Habaybeh
Khulud Habaybeh

security is preferable i think it will be widely spread thanks .