Hardware

Here is a touch-based interface that could make the mouse obsolete in less than a decade


This video is from last week's Gartner Symposium/ITxpo in San Francisco. The person talking is Eric Horvitz from Microsoft Research, and the demo is pretty impressive, especially since it is conducted on average desk surfaces and the data is manipulated very quickly and fluidly. Horvitz said that this technology could be available in the average office within 5-8 years. 

Will this eventually make the mouse obsolete? If it were combined with voice-recognition, could we potentially see a full system without a mouse and keyboard within a decade? I can't see a standard office going without keyboards anytime soon. Can you imagine the noise level if you had a whole floor of people in cubicles talking out loud to their computers? Everyone would need to have a private office in order to make that work.

However, in terms of replacing the mouse, this touch UI could be a big step forward, especially if the touch technology was combined with the forthcoming low-cost, ultra-thin OLED monitors to provide wide-view displays that can be easily manipulated with intuitive touch commands. What do you think? Is this the future? Join the discussion

About

Jason Hiner is Editor in Chief of TechRepublic and Long Form Editor of ZDNet. He writes about the people, products, and ideas changing how we live and work in the 21st century. He's co-author of the upcoming book, Follow the Geeks.

21 comments
Tony Hopkinson
Tony Hopkinson

Now big jammy finger prints on the screen so you can't even see what you are doing wrong. I've used touch screens, some have quite good niche uses, but as a general purpose robust and above all cheap interface, touch screens suck.

CharlieSpencer
CharlieSpencer

Look at the demo again. The user's hands are casting shadows where they block the data. The image is being projected from above or opposite the user onto what appears to be an ordinary flat surface. It does not appear to be displayed from below the surface. Am I misinterpreting what I'm seeing?

ITfor20+
ITfor20+

You are absolutely correct! There is (virtually) no touching involved in the video. The image is from above and the interaction is done using cameras and/or other motion sensors. In one part of the video there are TOKENS placed on the table ? they have patterns on them and the computer recognizes the position and orientation so it can project a virtual object around it. The researcher then manipulates the objects by moving and rotating the tokens. One of the ?tokens? is a cell phone that the computer ?sees? and connects with via Bluetooth ? extracting the photos and allowing them to be manipulated and even moved to the other phone on the table. Most of the rest is GESTURE BASED using machine vision. Not all that farfetched! We already see these in use ? a game console (playstation?) with a camera on top ?watches? the game player?s motions, allowing the player to interact within the game. I played a virtual volleyball game and a (lame) version of Pong by moving my hands around. The video mentions use of simple USB cameras to do this in the future. Go watch the movie MINORITY REPORT to see how this might work on a grander scale (they use gloves as part of the system in the movie). We already have limited gesture based interfaces in Vista with the tablet interface (which does involve touching). But the touch interface is WAY BETTER then the touch screens of yore. There are motion sensing ?virtual? keyboards out ? use your favorite search engine and look for ?virtual laser keyboard? for info ? they cost around $175 (US) if you want one.

Dr Dij
Dr Dij

think of the implications! programmers and data entry people would ENJOY going to work (excep t those with arthritis) currently you see people on the street nowadays talking to themselves and you realized, what with bluetooth earphones they are not crazy, just talking on their cell phone. Now we can have every cubicle filled with people standing, waving their arms wildly (with perhaps a flying Wii controller or two), and perhaps talking on their bluetooth cellphone, so they look like the crazy homeless person waving his arms in front of a store window crossed with the controller of the sky city on starwares with a blinky computer on his head! This will stop all sorts of people from thinking the workday is boring, and the young-uns (the know-it-alls) will think the workday is a big VIDEO GAME!

CharlieSpencer
CharlieSpencer

Or coffee cup, or sandwich? Seriously, how will the system interpret something I put down on the surface that isn't data? Food and drink, pens, paper I don't want treated as data, etc. That was a relatively large input area / display in the demos, and it may take a paradigm shift (a phrase I try to avoid using) before users accept having to leave that much of their desks otherwise unused. On the other hand, it will make military operations an absolute joy. Electronic situation maps with graphic markers to represent the units. Want D Company to relocate? Use your finger to drag their icon to the new position and the system will automatically generate a movement order. Want a minefield? Draw it in and the engineering battalion gets an obstacle overlay. The heck with keying in 8- and 10-digit map coordinates.

Wayne M.
Wayne M.

Remember Functions Keys? That old row of F1-F10/F12 that was printed across the bottom of the screen and was dynamically updated (i.e., context sensitive). I have concerns with the mechanics of the touch screen. These issues have long been the problem keeping touch screens from being more widely adopted. Two issues that spring to mind are the long term reliability of the sensor and what is the precision of the sensor? This latter issue has been hounding electronic voting machines for a while. What are the percentages of false reads? As the screen becomes more cluttered, how often do people misselect a neighboring selection. I am not a big fan of mice, but so far every alternative has been worse. Sure they may look good when a practiced presenter is demonstrating, but most are frustrating for the average user. Color me very cynical about the adoptation of touch-based interfaces for general computer work.

jasonhiner
jasonhiner

Those are some of my questions, too. Precision will be an important part of this discussion, because the new optical mice have gotten very, very precise. I wonder if the forefinger could become the new mouse in a touch-based interface. My fear is that will lead to simply making the whole workspace like a touchpad on a laptop, which I don't like.

00101111
00101111

Jeff Han looked at this technology at TED in 2006 and mentioned it has been played with over the last 10 years. Mutli touch display is not new, we are just simple hooked and faerful of leaving the mouse and keyboard behind. It sounds like M$ is buying out the little guy again, seriously...

mjd420nova
mjd420nova

Touch screens have been around for years. I serviced a system for resturants that had touch screen overlays that would fit any monitor and were only limited by how fancy the programer wanted to get. 265 different little areas to activate switches and up to 5 sequences from the same switch. This was with Intel 80286 based terminals in the later '80s. Bell and Howell had a setup that used IR sensors on the screen edge but was limited to an X - Y matrix, 15 by 15. The limitations are to numerous but are mostly based on finger size and viewers paralax.

jasonhiner
jasonhiner

Based on this interesting demo from Microsoft Research at the Gartner Symposium, this touch technology has a shot at making the mouse obsolete. If you haven't already seen it, take a look at my blog post to see a video of the demo: http://blogs.techrepublic.com.com/hiner/?p=452

techrepublic
techrepublic

It is not too often that we get to witness the creation of something that will result in a paradigm shift the size that this will. Every age has it's can opener, lighter, light bulb, saddle stirrups and least I even mention the personal computer and the Internet - but when you see what this technology brings to our attention I am sure you will agree that it is not something to simply interface with computers differently but that it is something that will fundamentally change how we interface with the Internet, media/data, work flow, recreation, conceptual and art-based activities, and even - dare I say - each other. If you pay attention to what the speaker is saying, you will realize that this is actually a grouping of various technologies that are already being used today and packaging them into a borderless bundle that works together within a seemingly ever expanding platform. At this very moment (5:45 AM Central) Eric (our head of server operations) and I are conversing (instant messaging) on a portion of the primary project we are developing for our main client. To have a 'common area' to relay information back and forth in 'real time' vs having to type out complex concepts - or even graphical layouts - that we find ourselves discussing quite frequently - would be nothing shy of a god-send. What they are demonstrating on this short video, when released, will forever change the way we think of computing and will open the door for these devices we play with day in and day out to become far more useful than even their original designers ever intended. I have my Uncle to thank for having opened my eyes to such a thing - such a possibility - so many years ago when he handed me 2 different books over a short span of time by the names of Power Shift and Killer App. What you are seeing in this video is to 'conference calling' as 'jumping really high' is to a 'rocket flight to the moon'. Here's to rocket flights and being able to sign loan docs online! When this is available to John Q. Public, drag-n-drop will never be thought of in the same light as it is today. Oh, and did I mention I want one? For everybody (grin)? _________________________ Max Laing, D. MP CEO / Project Development ActionCore, Inc. "Where We Plug In BOTH Ends Of The Cable!" Primary URLs... - http://AllowingSuccess.Com - http://AllowingSuccess.blogspot.com - http://PiddlePort.Com - http://LenderHost.Com - http://YourMortgageCareer.Com Other Involvements... [ to hear interviews with Max ] - http://beyondtheordinary.net/maxlaing.shtml [ to read about Max ] - http://spaces.msn.com/actioncore

Locrian_Lyric
Locrian_Lyric

"Gorilla Arm" The death of the early touch screens. Where your arms are comfortable and where your eyes look are two different places.

jasonhiner
jasonhiner

I don't think the workspace that was in the demo is the future of business. I just think it's a look at what will soon be possible. I'm glad that you brought up ergonomics, because that will have to be a key consideration in any revision of the current interface and workspace.

CharlieSpencer
CharlieSpencer

It's one thing to play chess seated, like in the demo. That only covers a relatively small area? But could you work effectively seated at a flat surface when you have to reach for the far side of it? Maybe something like tilted like an older drafting table would be more effective. Otherwise you'll have to stand all the time, and that won't cut it at all.

Dr_Zinj
Dr_Zinj

In fact, from an health and ergonomic stand point (no pun intended), standing at your workstation may be a healthier arrangement. Not to completely take away the office chair, but replace it with something more like a bar stool/workbench stool. Most knowledge workers are desk potatoes and getting fatter by the day.

CharlieSpencer
CharlieSpencer

I suspect that getting too small a control surface will result in a loss of pointer accuracy. I understand what you're saying about touchpads, but no one does CAD on a touchpad, and I suspect not many people game with one. I don't use a touchpad, period. If I have to work on someone's laptop, I bring along a mouse. And not one of those little-bitty "travel" mice; I'm talking a full-sized rodent.

cris.e
cris.e

When touchpad mice first came out they were smaller than I expected and I was surprised to notice that I rarely used more than the middle third of the space. I bet the same thing could be true here if you had the option to see your hands and workspace on the screen instead of the tabletop. In that case resizing an image or viewing your virtual pen output or whatever could happen in only the space your hands required instead of the larger space that legible docs might require. And you'd have an easier time of toggling to your "keyboard" if it were only a strip of plastic that corresonded to your home row or wrist rest so you could feel it when touch-typing). Anyway, I'd expect that the resolution in a normally lit office would be so bad that you'd need to stay with some sort of monitor to see, and at that point you could probably get away with a workspace not much over 6x12 inches.

Editor's Picks