Microsoft: 'Natural' UI Will Be the Wave of the Future

There's a new computing revolution coming and it's not based on a keyboard and a mouse. Instead, it will be based on touch, gestures, spoken language, and even painting -- what is becoming known as "natural user interface," or NUI.

Perhaps nowhere is that more visible than in Microsoft's (NASDAQ: MSFT) "Project Natal," an add-on device designed initially for the company's Xbox 360 gaming console that does away with physical controllers and instead turns the player's body into the controller through the use of 3D sensors and cameras. Gestures and movements determine how the game responds to the user, whether it is a racing game, tennis, or a shoot 'em up.

However, Microsoft see that as just the start. In the future, technologies like Natal, which is due for sale by the holidays, are likely to be anywhere and be used for many different tasks.

And that's just the start of where Microsoft sees NUI heading. Consider what happens when the office or the living room become the computer, when walls and other vertical surfaces become multitouch sensitive like Microsoft's Surface tabletop computer.

That's one of the questions being asked daily by Craig Mundie, Microsoft's chief research and strategy officer. When Microsoft co-founder Bill Gates retired, Mundie inherited half of Gates's duties, watching out for the long-term future of computing and how it will impact both the company and society at large.

The other half went to Lotus Development co-founder Ray Ozzie, who got the title of chief software architect. While Ozzie is the company's short- to medium-term planner, Mundie has been charged with formulating the long-term vision.

Some of Mundie's vision became apparent this week at Microsoft's annual TechFest, an internal science fair where the company's product mavens get to see what sorts of future technologies and trends are being studied at Microsoft Research, the company's research arm that falls under Mundie's purview. Although TechFest is closed to the press, Mundie and company provided a glimpse into what they've been thinking about -- and for them, the future is all about NUI.

In particular, Microsoft's researchers are delving into new possibilities surrounding accessing and controlling data through multitouch screens -- a concept that came up in 2007, when Microsoft introduced the Surface tabletop computer, and which was popularized soon after thanks to Apple's iPhone, which also debuted in 2007. Now, many mobile phones support multitouch input, including Microsoft's upcoming Windows Phone 7 Series.

At this week's TechFest, Microsoft researchers demonstrated a new spin on Surface. Called Mobile Surface, the technology uses a mobile device with a built-in camera projection system to turn anything flat, such as a desk or tabletop, into a multitouch sensitive display.

Another technology on display at TechFest is Project Gustav, what the researchers call "an immersive digital painting environment." Gustav lets users interact with a touchscreen using paint brushes and pastels in order to produce "convincingly real modeling for pastel and oil paintings," Microsoft said in a statement.

Although many technologies that Microsoft Research works on don't end up in products, some do -- Surface and Natal, for example, started out as Research projects. TechFest is designed to help with what can be a difficult process: technology transfer between researchers and product designers.

Microsoft Research was founded in 1991, with the goal of helping the company anticipate technology trends of the future. Today, Microsoft Research says it employs more than 850 computer scientists at eight locations in four countries.

Stuart J. Johnston is a contributing writer at InternetNews.com, the news service of Internet.com, the network for technology professionals.