CXO

Gesture recognition: Wave if you think it's the future

At the moment the latest user interfaces are dominated by touch. But things are changing fast and technology is pointing us in new directions.

The history of interfaces has involved presses, clicks and swipes but the future will be gesture and natural language. Photo: MIT

Written in a Hong Kong coffee shop and despatched to TechRepublic over a free 100Mbps wi-fi service.

I have just watched a three-year-old walk up to a flat-screen TV and swipe. Nothing happened of course - so she then tapped it with a finger. When that too failed to produce any effect, she looked puzzled for a moment and then toddled off to find a controller.

Well, she would, wouldn't she? As a part of the iPad generation, she has been born into a multi-interface world where everything seems to be changing.

A few days ago I was in a friend's car where all the settings are controlled by a mouse. The driving position was delightfully clean and devoid of any obvious clutter and complexity - but that was because the controls were hidden in layers of drop-down menus.

In contrast, my car has a facsimile of a drive-mode selector - a stick connected to electronics and not a mechanical linkage. It also has real buttons and knobs and a touchscreen menu system that is only two layers deep and not six.

Some of our household appliances and IT devices have touchscreens and some don't. Usually I can remember which is which, but from time to time I find myself tapping and swiping a screen that isn't touch-sensitive.

And because I travel a lot, I also run into even more interface confusion in hotels, elevators and aircraft.

Overall, it seems that hotel and aircraft entertainment systems provide the biggest interface challenges for me and the rest of humanity.

No other sector has so many variants. And how frustrating when you can't even switch the lights on and off, navigate to the broadcast TV channels or find an on-demand movie.

But what really bugs me is that they mostly provide an online video tutorial on how to drive their unique interface. If there was ever an admission of bad design it is the operating manual, instruction book and video tutorial.

A few years ago someone presented me with a rather splendid multi-function wristwatch. It obviously cost a tidy penny, but it came with a 350-page handbook. I'm afraid it didn't last long. I don't read handbooks and the interface was in no way intuitive - so I just packed it all up and gave it to someone else.

I suspect that wristwatch is now continually circumnavigating the planet looking for an owner who likes the challenge of handbooks. I just hope it never catches up with me again.

The history of interfaces has involved the turn, press, click and swipe. What happens next? How about wave or gesture, facial recognition, body language and natural language?

The problems of speech recognition

Of all these natural modes, speech turns out to be the most difficult to engineer. While it's easy to realise in quiet environments, any form of background noise quickly degrades performance.

Try any form of voice recognition against the din of a busy city thoroughfare, such as Oxford Street in London, for example.

But to be fair, even holding a conversation with another human in those locations is tricky - and we have the advantage of a priori knowledge, context, cognition, and visual cues including subliminal lip reading to help us overcome acoustic masking and distractions. Our machines do not.

Will they get these additional inputs? Certainly, but not yet. I reckon we'll have to wait for a near 100 per cent cloud penetration to gain that facility. So it looks as if gesture space may well be the next big interface advance.

But there is a newcomer on the block that may well give you and me new capabilities. HTML5 is going to be a game-changer when it works in league with the cloud and artificial intelligence. That triumvirate of technology will enable you and me to design interfaces to our own apps and documents.

But best of all, it may just let us adjust interfaces to suit our individual preferences, as opposed to those of some anonymous designer or engineering team.

Personally, I'm a bit of a trekkie and I have always lusted after that Captain Kirk/Jean-Luc Picard interface - I just want to talk to machines. And with the current rate of progress in AI, it is conceivable that dream might become a reality before I expire.

In the meantime I suspect that people texting, speaking into free space, wearing headphones and Bluetooth earpieces while sitting, walking, running and driving will be replaced by people waving and pulling funny faces in front of invisible cameras.

About

Peter Cochrane is an engineer, scientist, entrepreneur, futurist and consultant. He is the former CTO and head of research at BT, with a career in telecoms and IT spanning more than 40 years.

26 comments
Slayer_
Slayer_

Not long now... Then next step, Holographic porn.

dford
dford

As long as I can see the screen without my glasses - to find out what I should be doing and if I've done it yet - I'll be OK.

cuulblu
cuulblu

Read the manual! I have many electronic devices. Without reading the manual, at least in part, I would not even be aware of some functions. If you have a device worth owning it's worth at least skimming the manual to learn how to use it properly. Scientific studies show we are losing our ability to read a book of any kind, even for pleasure, because of our need for instant gratification that is fed by our point and click culture.

CharlieSpencer
CharlieSpencer

I don't want to have to wave my arms, point, or gesture in order to control a device. I'm perfectly happy getting my workout at the gym or pool; I don't need to spend my workday getting repetitive stress injuries.

dal765
dal765

So many devices these days are designed to exclude the elderly, whether by size, form, complexity or not allowing for reduced dexterity or vision. Throw in loss of familiarity- a new learning curve- and confusion builds on confusion. Elderly people are the least likely to want to read an instruction manual.. if the print is large enough. New tech. needs to address this ever increasing sector of our population where it might be used by them. So intuitive is good for common items and the major functions. And let's not forget the value of feedback- audible, tactile. Classic example.. the silent electric car engine.

SKDTech
SKDTech

What is intuitive? Intuitive is when something works in an obvious fashion to everyone who attempts to use it regardless of prior knowledge. If you want intuitive then look at a manual hand tool, but even some of them are not intuitive. Computers are highly complex and the intuitiveness of the interface is dependent upon prior experience. People are always extolling the intuitiveness of the iOS interface and I will admit that it is not hard to pick up. But honestly, how many people with no prior exposure to any touch devices would have any idea what to do with one without being told or shown how to do so, even if it is only the owner swiping the unlock screen. Heck, even the unlock screen has explicit instructions [u]on the screen[/u].

beaverusiv
beaverusiv

I just have to say about manuals that not everything has to be intuitive, in fact I believe most things should NOT be intuitive. Being intuitive to me means being inefficient and feature starved. You know why the iPhone is gradually getting features that have been on Android for a long time? It's because they didn't want a manual, so everything has to dribble in so you're not learning very much at a time. I am in the process of changing my word processing over to vim. It is not intuitive, it does have a manual and yes it is helpful to have it's video tutorials. You know what? After learning how to navigate a document and shortcuts for HTML coding I am already twice as fast as I was, and I'm only just beginning.

beaverusiv
beaverusiv

Technically yes, I see it as the future. But right now gesture, voice and mind control is like tablets in their early years. I can't see it being mainstream for a while because they haven't matured as a technology. When gestures aren't something you have to do different, and instead are the computers acting on intent, then will it be accepted.

tjsobieski
tjsobieski

it must be the future, it was in a movie, with Tom Cruise. If it's in a movie it MUST be true

peter
peter

Those folks are always ahead of the game and have the best tech!

peter
peter

Poor sight, poor hearing, and stiffening fingers...all inherited form my gene pool...soon be time for a body upgrade :-)

peter
peter

I try not to waste my time reading books...the vast majority are pulp...and there is no search function...

peter
peter

I bet your not French or Italian :-)

peter
peter

Search out the apps for the old and disabled...there is a slew of really good stuff from bigger text and soft buttons to raster scanning interfaces.

peter
peter

Hands and fingers are intuitive - just watch little children - they are blank slates!

CharlieSpencer
CharlieSpencer

The Sandisk model had a small round button with a larger outer ring/button around it. The compass points of the outer ring were marked with a variety of symbols that generally indicated their function, but I couldn't adjust the volume. The insert that came with the device didn't mention the volume. It wasn't until I downloaded the full manual from the manufacturer that I learned the ring was also a wheel, and could be spun around the central button to adjust the volume. Intuitive perhaps to those who had used other MP3 players, but not to a first-time user.

dr-haim
dr-haim

I was gifted an iPad for my birthday a few months ago and I have to admit that as a long time Windows user I found my first experiences with the iOS interface to be anything but intuitive. I was extremely frustrated for weeks as I tried to figure out matters as simple as saving a bookmark in Safari (took me 20 minutes) or deleting songs from iTunes. I kept looking for an icon in iTunes to delete songs and then kept tapping and prodding the file name, which just played it again. I don't know too many people who would "intuitively" think to swipe left or right on a listing of a song and expect that that would allow them to delete it. When I posed this issue to my iOS savey friends, they agreed that they wouldn't have thought of that on their own if they hadn't learned it from someone else. Even common sense has to be someone's common sense. Prior experience is definitely an important factor. Easy to remember once you learn it is another thing!

peter
peter

Hmm, this seems to be a circular argument you are playing....video tutorials...

CharlieSpencer
CharlieSpencer

At least in the U.S., we have the expectation that we should be able to effectively use any device right out of the box, taking full advantage of all of its features. How will you know what those features are if you won't even crack the manual, much less how to use them? I don't enjoy reading manuals, and I certainly wouldn't bother with a 350-pager for a watch. But I'm amazed at the attitude people have that they will automatically know how to use a device just by osmosis, and that reading a manual is somehow demeaning.

peter
peter

Sometime change happens when enough people have died and moved out of the way, whilst at other times fashion just takes over and it all takes off. Easy to predict the how, but the when, that's far more difficult.

peter
peter

History would say that's the way it works....they seem to feed on each other

peter
peter

I just hate it when designers do this stuff!

maj37
maj37

To me interfaces are seldom really intuitive, I like the term "learned intuition". Like you said someone that has experience with similar devices and their interfaces can fugure it out pretty qucikly but the first time user needs help.

peter
peter

That is exactly like acquiring a bad golf swing or a bad habit...take ages to unlearn !

peter
peter

I'm with you :-)

peter
peter

Just like walking!