Patents

Microsoft bringing touchscreens to walls, clothes, and more

Microsoft Research has unveiled a pair of new projects aimed at bringing touch technology to a lot more places. Will they become real products?

On Monday Microsoft showed off a pair of research projects that are aimed at the multitouch technology that the public has fallen in love with on their smartphones and tablets and taking it beyond LCD screens to everyday surfaces.

The innovations come from Microsoft Research and were unveiled at UIST 2012, a conference on User Interface Software and Technology this week in Santa Barbara, California.

The first is called OmniTouch (right) and it uses a combination of a laser-enabled pico projector and a depth-sensing camera to provide a wearable touch interface that a user can turn on and then use on any virtually any surface, but will obviously work best on a flat surface.

Here's how Microsoft Research explains it:

"The early phases of this work raised some metaphysical questions. If any surface can act as an interactive surface, then what does the user interact with and what is the user interacting on? The team also debated the notion of turning everything in the environment into a touch surface. Sensing touch on an arbitrary deformable surface is a difficult problem that no one has tackled before. Touch surfaces are usually highly engineered devices, and they wanted to turn walls, notepads, and hands into interactive surfaces—while enabling the user to move about."

The prototype for OmniTouch uses a big, bulky, shoulder-mounted camera that would be impractical for anyone to wear today, but with cameras and microprocessors continuing to shrink dramatically, we can obviously imagine a day where you could have something like this emerge in a much less intrusive form, such as tech-infused shirt or jacket.

Here's a video of OmniTouch:

The second innovation is called PocketTouch and it is aimed at making smartphones receptive to touch gestures through fabric, such as a pocket, purse, or jacket. The idea is that sometimes people want to quickly interact with a smartphone without having to pull it out. Microsoft refers to PocketTouch as an "eyes-free" solution. Here's the description:

PocketTouch enables a rich set of gesture interactions, ranging from simple touch strokes to full alphanumeric text entry. Our prototype device consists of a custom multitouch capacitive sensor mounted on the back of a smartphone. Similar capabilities could be enabled on most existing capacitive touchscreens through low-level access to the capacitive sensor... Our results suggest that PocketTouch will work reliably with a wide variety of fabrics used in today’s garments, and is a viable input method for quick eyes-free operation of devices in pockets.

And, here's the video clip of it:

The Microsoft Research team noted, "Both OmniTouch and PocketTouch are evolutionary steps of a larger effort by Microsoft Research to investigate the unconventional use of touch in devices to extend Microsoft’s vision of ubiquitous computing."

It's terrific to see Microsoft innovating on multitouch since it's obviously a critical element of the future of computing. However, Microsoft Research has a track record of showing off lots of cool stuff that never comes to market. They need to follow the lead of IBM's prolific research devision, which is much better at commercializing and productizing its best innovations -- or at least maybe IBM only shows off stuff that has a reasonable path to becoming a real product. Still, in this case, it looks like Microsoft has a couple innovations that aren't just cute ideas.

About

Jason Hiner is the Global Editor in Chief of TechRepublic and Global Long Form Editor of ZDNet. He is an award-winning journalist who writes about the people, products, and ideas that are revolutionizing the ways we live and work in the 21st century.

32 comments
trevorcsr
trevorcsr

"PocketTouch enables a rich set of gesture interactions, ranging from simple touch strokes to full alphanumeric text entry". This will not get approval in Europe as Apple have a sketch of someone writing with chalk on a blackboard.

Realvdude
Realvdude

Obviously the device size would have to be reduced for truely mobile use. Before watching the video, I thought that monitor glasses and retina detection (ala Canon camcorders) would make a lot more sense. After watching the video, I realize I could never look and blink faster than I could touch and input. Multi-touch is also out of the question with retina detection.

redbaron101
redbaron101

augmented reality / gestural interface is not a new idea - I've seen this in countless sci fi flicks, e.g. one of the bigger ones like minority report and countless smaller movies/series, all back in the late 90's. Even back when sixthsense started I'm sure I read about projected infra red keyboards being developed way before this. Its good though that technology is slowly getting there.

cgabriel7
cgabriel7

I can't see OmniTouch bulky hardware replacing tablets and even smartphones unless miniaturized small enough to attached to your eyeglasses/goggles. However, I could see many ways the military could use it attached to the already loaded helmet. Oh, and maybe C3PO can use it as a backup for his holographic system.

gpachello
gpachello

OmniTouch = Kinect + SixthSense. Next steep, make it smaller.

seanferd
seanferd

No wonder MS has such a hard time.

jayohem
jayohem

About 5 to 10 years ago the big news was a keyboard that was a display which folks could use with their various devices. Sounds as if Microsoft has done something along that line. Current term for it is projection keyboard. The Microsoft applications seem like the same principle using other instantiations. http://en.wikipedia.org/wiki/Projection_keyboard

Roc Riz
Roc Riz

Touch screen genitalia! You know that's something that WILL be exploited, and when it is, it will take off. Remember movies over the 'Net? Wasn't Pr0n the first thing to come out with movies that people actually WANTED to watch over the 'Net? Same thing with photos. Mock me if you will, but when the sex industry realizes what can be done with this sort of thing, there will be no gains. Not for a long time.

neldeeb
neldeeb

:)) Do we need women to misunderstand us (men) even more? Poor dum Microsoft. They will go to any length (even half a meter brick in your pocket) to immitate Apple :)) Even trying to innovate :)

cbutler
cbutler

Much of the technology necessary for the OmniTouch prototype is already in the Kinect sensor Looks like the brilliant and processor-nonintensive depth sensing method they came up with has some other legitimate uses besides Dance Dance Revolution :) I wonder if the Kinect was a development fork from THIS research project, or was it the other way around?

Slayer_
Slayer_

I want one of those when they are properly realized :). Just think of the video game potential.

wordsmithjr
wordsmithjr

Actually These thin plastic sheets are identical to one used in the Sci Fi film "Ultraviolet" phone she got from an ATM like machine and it was a sheet like disposable phone . I thought at the time that it would happen in future it is a great idea.

codepoke
codepoke

My gut reacts to it exactly the same as I did to Google's full body gesture system. Seriously? Alphanumeric control of a device in my pocket? We already have a pocket-dialing problem. Do we need to create pocket-emailing problems? I can't imagine a need for eyes-free control of such a device besides a quick gesture for "shut up whatever alarm you're currently embarassing me with." Maybe the video would make it all clear, but it's blocked at my company.

Roc Riz
Roc Riz

at "touch strokes." That just sounds obscene.

ianni67
ianni67

Actually, a similar application was presented at PUI2001, november 2001, Orlando, Florida. But the most similar ones which I've seen where presented in 2002 and 2003: http://visilab.unime.it/new/index.php?section=19 and http://visilab.unime.it/new/index.php?section=20. Please note that this is a VERY small lab at the university of Messina, in Sicily, south Italy. There have been hundreds of similar applications all around the world in the last 10 years. I wonder whether MS has a patent: It would be a nonsense.

neldeeb
neldeeb

Leave it for Apple to always create the Class, and the rest (all, and speciallt dum micros...) to make the object instantiations :))

seanferd
seanferd

I saw that many years ago. There are already quite a few interesting technologies for gestures, etc., out there. They just aren't consumer commercialized yet. But for all the apparent effort MS has been putting into these things (like Surface), they have very little to show for it. Except Kinect, which was originally based on third party technologies. (What else is new?)

auplater
auplater

Sigh...productizing... do we really need to keep univerbilizing all things as summarisations of technologized conceptualizations? Seems arbitrarily obfuscatorilized....

wizard57m-cnet
wizard57m-cnet

Virtual sex...the "real" thing having been outlawed and exchange of "bodily fluids" a crime as well!

adornoe
adornoe

What has Apple actually done by way of innovation? Most of what Apple has produced in the last 10 years is a re-working of others ideas or of products that had already existed in the marketplace. So, how is that innovation? When it comes to innovations, Microsoft has produced a lot more of them than Apple and Google combined. However, Microsoft hasn't taken advantage of their innovations to "product-ize" them, or hasn't been as successful at making their products as "fashionable" as Apple has with their marketing prowess. But, many of the ideas which don't come to market as products, are still money-making ventures in the form of IP. So, if Apple or Google or IBM or any other company were to develop or "innovate" with Microsoft's ideas, those companies would have to pay Microsoft for the use of those ideas. Google knows quite well what that means in the form of Android, where each device sold with Android has a Microsoft "fee" levied against it.

bhoffman
bhoffman

Microsoft, who sends many people to TED conferences, saw this technology at TED in 2006 or 2007. I suspect that this is where this technology came from. I believe it is from someone at MIT media Lab and that a prototype, with a bill of materials and software was listed at their site (component costs about $350).

matthew.clark
matthew.clark

Are you joking about being unable to imagine a need or use for this technology? I can see strong needs for this in military, medical, education, construction, delivery, warehouse, public kiosks and information systems, public transportation, automotive, sports, anything benefiting from heads up, natural and/or multimodal interaction, etc. etc. etc... Just think of the multitude of gaming possibilities... to say nothing of the porn industry...

belli_bettens
belli_bettens

First of all, if I remember right, the full body gesture system was an april fools joke, funny you bought it :-p (no offence) But I think that PocketTouche can have a real potential. The video demonstrated stuff like writing but I guess that that would be ridiculous to do in public on top of that it would be real cumbersome. But it might be of good use when you want to skip a song or mute an alarm (or call), so for small gestures. Question remains than, can it be produced cheap enough. Only then will a 'gimmick' be added to real devices

Slayer_
Slayer_

And I refuse to eat at Taco Bell.

neldeeb
neldeeb

The technology in question here can be used in all the fields you mentioned. Only if it wasn't dum MicroSh.. Only if that block you are supposed to put in your pocket wasn't that big. You know, women do exist in all those fields (the misunderstanding. It's inevitable. It's horriible ..)

neldeeb
neldeeb

I don't have a list of April's fool jokes that ever existed (nobody does). So I couldn't be offended by your remark (even if you wanted to) Plus, I didn't even refer to the body-gesture thing. It is the writer of the article who did :) I only referred to that stupid brick one is supposed to put in his (his) pocket (and be misunderstood by horny women) :) And it was actually my point, not only that it's rediculous to write on your pocket in public, but to put this piece of MS (Microsoft Stupidity) in your pocket, in public, in the first place :))

codepoke
codepoke

I'm sorry you missed it.

neldeeb
neldeeb

In the case of MS (Microsh..'s Stupidity) it could very well be not-a-joke) Sorry you can't get something like that, smarty :)

Editor's Picks