24 May 2013 by MacGregor Campbell
Read the signals
The elevator pitchIn the lobby of Microsoft Research in Redmond, Washington, there's an elevator that reads you like a book.
It is equipped with a camera that peers at people in front of its doors. When someone approaches, it will open – but only when it senses that the person is looking to use it. The system has processed many hours of video footage of people mingling in the lobby and has learned to distinguish between someone intending to use it and someone just walking by.
As computers come to recognise ever-more-detailed gestures (see main story), they will be able to infer more about us. Other researchers have programmed computers to use body language to infer a person's mood, be it happy, angry or sad. Such "emotionally intelligent" machines would better respond to our needs.
So if you are slouching in front of a screen, bear in mind that a computer may soon be watching.
Patently absurdIt's hard to believe that waving our arms, hands or fingers could spark heated patent litigation, but if the history of the two-dimensional gestures on touchscreens is any precedent, such a fate awaits 3D gesture interfaces, too.
A battle over 2D gestures began after a Silicon Valley party in the early 2000s, when Apple's CEO Steve Jobs got riled by a Microsoft engineer boasting about its stylus-controlled touchscreen Tablet PC. Jobs ordered his engineers to build their own touch interface, but he was adamant that it would use only hand gestures. They came up with specific moves like pinch-to-zoom, tap-to-zoom and swipe-to-unlock – which Apple quickly filed patents to protect.
This sparked a gestural-patent landgrab. For instance, Google applied for a patent on a text-recognition gesture involving underlining words in a picture with a swipe, while Nokia's gesture patent applications included circular or oval swipes, with the size of the circle or oval dictating the degree to which the screen zooms in on an image.
Attempts to patent 2D gestures ultimately failed. Apple recently took Samsung to court over the latter's use of the pinch-to-zoom and swipe-to-unlock movements. In the end, the US Patent and Trademark Office ruled Apple's patents invalid on the grounds that earlier inventions used the ideas.
An optimist might think that this would discourage a similar landgrab over 3D gestures. Alas, it hasn't. Microsoft holds patents for Kinect that cover flicks of the hand to scroll on screen, or gestures that call up a search box. And Intellectual Ventures of Bellevue, Washington, has filed a patent on a way to control a television that includes a raised "flat-hand" gesture to get its attention. The company is notorious for aggressively protecting its intellectual property too.
Litigation that tied the tech industry in knots for the past few years looks set to be repeated and, as usual, the only winners will be the lawyers.