Beyond Binary

After taking Natal for a test drive, CNET News' Ina Fried wants to know when the rubber will meet the road. One game maker suggests the answer is late next year.

(Credit: CNET News)

Now that I have gotten a chance to try out Project Natal, Microsoft's gesture recognition technology, I have the same question as everyone else. When is it going to be on the market?

While Microsoft isn't saying, one game maker has spilled the beans. In its recent earnings conference call, game maker THQ said to expect it late next year.

"We have for example, Natal from Microsoft, a platform addition coming late next year," THQ chief Brian Farrell said on last week's conference call.

In an interview with me, also from last week, entertainment unit President Robbie Bach declined to offer any more details on Natal timing.

"I'm not planning on being any more specific today," he said.

In addition to the comments from THQ, Electronic Arts also indicated on its earnings call that it plans to support Natal.

Microsoft has also said it has plans for Natal that stretch well beyond just gaming or the Xbox. Chairman Bill Gates told CNET that it is a technology he sees moving onto Windows, while Chief Research and Strategy Officer Craig Mundie has shown some of the same gesture, voice, and facial recognition technologies as being key parts of the office of the future.

Project Natal is both great gaming and a great workout, as CNET News' Ina Fried experienced firsthand when she got to try out the technology last week.

(Credit: CNET News)

REDMOND, Wash.--One of the reasons that Microsoft got such buzz for Project Natal is because it is so easy to see how the technology could change the face of gaming.

But it's even easier to appreciate once you get a chance to try the gesture recognition technology yourself. When I was in Redmond, Wash., last week, I got a chance to do just that.

Playing Ricochet, a 3D breakout-like game, I found myself wanting to do whatever I could to stop the balls from passing me. It felt less like a traditional video game and more like I was a soccer goalie and an entire team was firing shots at me. (For a firsthand look, check out the embedded video below.)

It was both a lot of fun and a bit of a workout. Apparently, I'm not the only one who has noticed that.

"Since I started working on this project, I've lost almost like 10 pounds," said Kudo Tsunoda, general manager of Microsoft Game Studios and the creative director for Project Natal. "We're going to have the most in-shape development team you've ever seen."

The effort is important to more than just the waistlines in Redmond. Microsoft is counting on Natal to give an important bump to the Xbox 360, which Microsoft has said is only mid-way through its lifecycle, even though it has been on the market since 2005.

After Ricochet, I tried my hand at an existing driving game that had been connected to the Natal interface. And while my steering hasn't gotten any better than when I checked out a set-up from GestureTek earlier this year, Microsoft's technology is quite impressive. The steering and other controls were both intuitive and responsive.

I moved my foot forward to accelerate and backward to slow down, brake, and eventually reverse the car. To steer, I simply used my hands like a steering wheel.

Although Microsoft demonstrated Natal at this year's E3 trade show, the software maker hasn't said when the technology will be available. The company has said that Natal, which incorporates face, voice, and gesture recognition technologies, will be sold as an add-on to the current Xbox 360 console.

The effort to turn Natal from concept to shipping product has been something of a mini Manhattan Project inside Microsoft, according to former Carnegie Mellon researcher Johnny Chung Lee, who is among those working on the effort.

And while smashing bricks and cars are some of the first ideas on how to use Natal, the vision clearly goes a lot further.

Inside Xbox, Tsunoda noted that Natal can be useful for more than gaming. He noted that for many first-time console users, the controller itself can be intimidating, even when trying to do things like navigate through menus. Oftentimes people get their first experience with the Xbox when they are at the house of a friend or family member who has an Xbox and they are handed a controller with lots of buttons.

"For a lot of people that can be intimidating," Tsunoda said. "You don't really know what to do and you're starting to feel stupid and everyone is looking at you and you are not being successful. That's really not a good first way to interact with our console."

Tsunoda and Entertainment Unit President Robbie Bach both said they are confident that Natal will also have great appeal for the core gamers already spending hours a week playing on the Xbox.

"Even the folks who are hard-core Halo or Splinter Cell players, they are also going to want to play Natal games," Bach said in an interview.

In an interview with CNET News last month, Bill Gates talked about how the technology has applications well beyond just gaming.

"I think the value is as great for if you're in the home, as you want to manage your movies, music, home system type stuff, it's very cool there," he said. "And I think there's incredible value as we use that in the office connected to a Windows PC. So Microsoft research and the product groups have a lot going on there, because you can use the cost reduction that will take place over the years to say, 'Why shouldn't that be in most office environments?'"

At last week's analyst meeting, Bach and Craig Mundie, chief research and strategy officer, also outlined the broad appeal of being able to interact more directly with computer interfaces. After Bach tried his hand at some Natal gaming, Mundie offered a demonstration of how gesture recognition might function in a work setting, saying that the desktop PC of the future could in fact encompass the entire office.

Microsoft Chief Research & Strategy Officer, Craig Mundie, demonstrates natural user interface technologies during Thursday's Financial Analyst Meeting in Redmond.

(Credit: Robert Sorbo/Microsoft)

REDMOND, Wash.--While gesture recognition, such as that seen in Project Natal can help gaming, Microsoft's Craig Mundie showed how it will also transform the office.

In a demo, Microsoft's top research and strategy officer showed how the desktop computer of the future will use an entire office as both display and input device, with voice and gestures augmenting a number of touch screens.

Mundie

"The real question is what killer apps (will mark the) new era and what will be the user interface that people use to get at them," Mundie said, speaking at Microsoft's financial analyst meeting here.

His demo included hologram-like video conferencing, a virtual digital assistant, and multiple surface computers along with voice, touch, and gesture recognition. The desk was a multitouch surface computer, and the office's walls were also a display that could easily switch from being a virtual window and collection of digital photos to being a corkboard of sticky notes to various workspaces.

In one case, Mundie also used Natal-like depth cameras to put himself in the middle of an architectural demo, essentially putting himself inside a building that was not yet built. His talk followed entertainment chief Robbie Bach demoing the gaming potential of Natal, playing a breakout-like game called Riccochet, where one uses their body to push, block, and kick balls at various bricks. Microsoft showed Natal at the E3 trade show earlier this year but hasn't said when the Xbox 360 add-on will be commercially available.

"I'm not playing the Riccochet game, but I am using these technologies," Mundie said. "This is our dream, but it is really not that far away. We see a pretty direct path to make this happen. We have all of the technologies to make this happen in our research labs."

In an interview earlier this month, Bill Gates told CNET News that Microsoft plans to use Natal far beyond the Xbox, including with Windows.

The demo was similar in some respects, but more advanced in others, than the one shown by Office chief Stephen Elop earlier this year.

It's been a year since Bill Gates left full-time work at Microsoft, but he's found plenty to keep him busy.

In between trying to eradicate polio, tame malaria, and fix the broken U.S. education system, Gates has managed to fulfill a dream of taking some classic physics lectures and making them available free over the Web. The lectures, done in 1964 by noted scientist (and Manhattan Project collaborator) Richard Feynman, take notions such as gravity and explains how they work and the broad implications they have in understanding the ways of the universe.

Gates first saw the series of lectures 20 years ago on vacation and dreamed of being able to make them broadly available. After spending years tracking down the rights--and spending some of his personal fortune--Gates has done just that. Tapping his colleagues in Redmond to create interactive software to accompany the videos, Gates is making the collection available free from the Microsoft Research Web site.

Gates said that he hoped his action would serve as a model for taking great educational content and making it broadly available for free.

"When a lecture is presented as well as this, it draws more people in to understanding science." Gates told CNET News. "And over time I hope there's more like this."

In a wide-ranging interview, Gates also reflected on the changes at Microsoft, spill the beans on the expansive vision for Product Natal and shared his thoughts on Google's just-introduced Chrome OS. Here's an edited transcript of that interview.

You first saw these videos on a vacation 20 years ago. Do you want to talk a little bit about how that happened, and what your reaction was to seeing those lectures?
Gates: Yes. I was in a period where, in order to learn new science, thought it would be a fun thing to see what films there were, and we went to some university catalogs, including University of California system had a catalog of films, and got a lot of health, biology, physics type films--those are those metal cans with big reels--and then we had a projector in a room that we made dark. So even (during) the day, you could thread these films. And there were a lot of interesting ones, but these Feynman lectures that he gave at Cornell...those were just unbelievably good.

After that, I got them put onto videotape, and I got rights to make a small number of videotapes. It was VHS tape at the time, and send it around to some friends who might be interested. But I always had in the back of my mind that it was kind of a crime that there wasn't broad availability of those things, particularly for young people thinking about science.

And so I sort of had this project in mind, and (have been) making some progress in understanding who had the rights, and eventually doing deals for the rights, and then getting these things scanned, and then getting Microsoft Research agreed to host the stuff and create some innovative software around it, which Curtis (Wong) has run. It's taken a long time, but with lots of PCs and the Internet, and my willingness to spend some money, now these things are just going to be out there.

What do you hope people get out of these videos? Who is your ideal audience for them?
Gates: Well, I didn't get to see these until I was about 30, and so I would love it if lots of young people saw them, and got a sense of the fun, and how science works, and what's complicated, and what's not. I hope some people who teach science are inspired by the way that Feynman managed to make it interesting without giving up the depth of how it works.

With super-high-quality material like this up there for free, I hope people see the potential, and that they'd benefit from this one in particular, and then it starts to push forward the idea if someone is great lecturer, then their work should be out there and available.

I've heard you talk about the way community college really should change, and really what we should be doing for some of these subjects that are somewhat universal is taking really the best explanations, the best lectures out there, and making those broadly available, and then focusing sort of the local learning around discussion and different sorts of things.
Gates: That's right. Education, particularly if you've got motivated students, the idea of specializing in the brilliant lecture and text being done in a very high-quality way, and shared by everyone, and then the sort of lab and discussion piece that's a different thing that you pick people who are very good at that.

Technology brings more to the lecture availability, in terms of sharing best practices and letting somebody have more resources to do amazing lectures. So, you'd hope that some schools would be open minded to this fitting in, and making them more effective.

But, you've also got this huge set of people who like to teach themselves and like to learn things, and yet find science kind of daunting. And when a lecture is presented as well as this, it draws more people in to understanding science. And over time I hope there's more like this, including some about science stuff that's changed since the time these were done.

How big an impact do you think these types of things can have in terms of the overall problem of getting people interested in math and science? Is this type of thing enough, or do we really need to fundamentally do more, younger?
Gates: Well, certainly in fifth grade through senior year, most students aren't yet motivated to want to learn a lot in general, and particularly about science and math. The big impact is anything that can help teachers do a better job, where teachers can either see other teachers doing it super-well, or they might incorporate some online things into the classroom experience. As you get older, and you've got people who are motivated more clearly, then it shifts where these online lectures can be a huge part of learning.

That's where Feynman with his clarity of explanation and simplicity of explanation, and love of the subject, and humor around it is such an exemplar.

You mentioned that you didn't get to see these until you were in your 30s. If you had seen them earlier in your career, maybe before you decided to start Microsoft, do you think you might have headed in a different direction?
Gates: I'm not sure. I've always liked physics, but I also want the equivalent lectures to be out there for biology, and computer science, and chemistry. Everybody has a level where you can bring in their interest. I mean, people care about animals, and disease, and food, but many of the sciences are so abstract, and the amount of things you have to learn before you start connecting to those practical issues can be very daunting. And yet with a teacher like Feynman they're out there in different fields, it's just that we haven't had a way to magnify their excellence, and make it broadly available.

One of the points that's made in the lectures is this idea that from the discovery of gravity there's basically been since then 400 years of just an avalanche of discoveries, and he sort of puts forth this notion of continuous progress. And I'm curious, do you see that having continued, or have we seen limits to sort of some of the full understanding that the basic sciences can give us? Are there things that are beyond sort of what basic science can teach us?
Gates: We're learning more about basic science today by a huge amount than ever before. You just take understanding materials, why they break, why they're strong, how you engineer them to have various properties, and a lot of that was black magic. And it's only now that we're able to say, okay, when we want to make batteries that charge really fast, okay, how do you make something with a lot of surface area that doesn't degrade.

Anyway, in material science, or basic medical things, or basic things about physics that are going to be important for cheap energy as just one example, this is the most interesting time. That's why it's partly an irony that you're not getting the best and the brightest particularly native born to go into science and math. And so you've got to look back and say, what is it we're doing about making it daunting, or abstract that holds that back so much.

There's an American physicist, Fritjof Capra, (who) wrote a lot of books in the '70s on ecology, and the limits of Cartesian thinking. Basically his thing was that by focusing on sort of the Cartesian reductionist approach to things that prioritizes sort of looking at the small parts--that type of thinking has contributed to not getting as deep an understanding of things like ecology, and really complex systems. Is that what's caused us to get into some of the problems we have, or do you think it's more just these are tough choices and require conserving, and things that are kind of hard for us as humans to do?
Gates: Well, the tough situation that we're in is that we have electricity, we have medicines, we have vaccines, those were all due to scientific understanding. And as we get new materials, new batteries, solar, nuclear energy that don't cause environmental things, it will be because of these scientific understandings. So, I think the incredible improvement in living standards, and life expectancy, and literacy, and all those things really do come back to the advanced scientific understanding. And when people look at history, that's the one thing that they always undervalue is how scientific progress has allowed us to do those big things.

It's true that as you go forward, you tackle more complex problems, but the tools of modeling and simulation and getting a lot of people who are mainly in politics, but know enough about science to be in the discussion, that's important. You know, there was a book written called Physics for Future Presidents, which took some of the basic notions of energy density and costs and dangers about radiation or nuclear weapons, and put that into a fairly straightforward thing.

We do have a problem if we don't draw a large part of society into at least some understanding of science and the tools of science. And so, having great lectures online, I have several goals--improve education, get more people into the sciences in a deep way, but also get a broader set of people into sciences in even a modest way.

When we talked a year ago, I asked kind of what you anticipated your life would be like once you stopped being at Microsoft full time. Now a year later what are some of your observations on how your time is different, and maybe what are some things that you hadn't expected about where you are today?
Gates: Well, the foundation work is very rewarding, and there's a lot of interesting complexity that comes with it. I'm pretty much doing what I expected to be doing, which is very different than what I was doing before my job changed. I do have about 20 percent involvement with Microsoft, where topics like their future of Office, of search, or various things that Steve (Ballmer) asked me to look into and help out with come along. So that's developed pretty much like I would expect.

It will be interesting as I get a year or two more out, and I know the activities and the people (at Microsoft) a little bit less, you know, how Steve and I make sure I stay fresh and connected and things like that. So, maybe the first year was always going to be the easiest. And it's at the level that we planned it for, which is giving me a massive amount of additional time to meet with scientists and go to the developing world and meet with various government partners.

For the last three months, up until two weeks ago, I was entirely in Europe, and actually based out of there. Our family had moved over there. So, I was up at Cambridge and Oxford. For that period I was particularly focused on the science and partners, both governments and companies, and things that happen to be based in Europe. That's done, but the kind of things I was doing there are exactly what my schedule looks like over the next six months, where I'm in India, I'm in Africa, going to meet with companies, doing things, meeting with scientists. So, you know, I'm thrilled by the foundation work, and fortunately I have Jeff Raikes running the foundation as CEO, and so my role at the foundation is a lot like it was in the period where Steve had already taken over as CEO, where I got to be more on the research side, the breakthroughs, the new ideas.

And you've been doing some stuff with Intellectual Ventures. I know every time you show up on a patent application that, folks get interested in what you're looking at, whether it's stopping hurricanes, or beer kegs, or what-have-you.
Gates: That's right. We're going to make the cows that don't fart. You name it, we've got it under control.

That's been really exciting to take this idea of gathering top scientists from a broad set of areas and think about problems that can be solved. And in the case of the foundation, you know, Nathan (Myhrvold) has used that ability to convene great scientists to look at things like how do you deliver vaccines without having to use as many refrigerators, or how do you pasteurize milk in a better way, some very interesting things. And then I also sit down with that group when they're looking at their rich world applications, including things around energy, and one of those has actually led to creating a company called TerraPower, which is focused on a new, very radically improved nuclear power plant design, which is a hard thing to get done, but extremely valuable if it comes through.

I'm curious of your thoughts of how Microsoft is doing as a company since you left. I'd also be remiss if I didn't ask you what you thought of Google's efforts to get in the OS arena.
Gates: Well, just to do the second part very succinctly, there's many, many forms of Linux operating systems out there, and packaged in different ways, and booted in different ways. So I don't know anything in particular about what Google is doing. But, in some ways I'm surprised people are acting like there's something new. I mean, you've got Android running on Netbooks; it's got a browser in it. In any case, you should make them be concrete about what they're doing. It is kind of a typical thing. When Google is doing anything it gets this--the more vague they are, the more interesting it is.

I guess there is the notion, though, and I know Microsoft Research had been looking at it, too, of whether the browser, because it's become so central to so much of our work, needs to take on more operating system-like characteristics.
Gates: It just shows the word browser has become a truly meaningless word. Anyway, what's a browser, what's not a browser? If you're playing a movie, is that a browser or not a browser? If you're doing annotations is that a browser or not a browser? If you're editing text, is that a browser or not a browser? In large part it's more an abuse of terminology than a real change.

What about on the question of how Microsoft is doing?
Gates: I'm always the one who thinks, gosh, why isn't Microsoft doing even more, because that's been my mindset, let's move fast, do new things very quickly. But, you have to say, whether it's Windows 7 that is a really excellent piece of work. I'd go so far as to say both compared to other operating systems, and compared to other generations of Windows, it's an extremely nice piece of work.

What they're doing in new versions of Office--I guess they showed a little bit of how the Web piece fits into it recently, but there's a lot about the new version that will get talked about in the next nine months or so. The work on search, where people see Bing as a nice piece of work, really see us in the game, hiring really top people, and willing to try to do things some different ways.

The part of Microsoft I stay up to date the most on is probably the research group. I was over at the Cambridge lab a few weeks ago, over at the India lab as part of a trip I take this month, and that's really the sort of crown jewel in terms of always feeding neat new things into Microsoft. I'd say a cool example of that, that you'll see is kind of stunning, in a little over a year, is this (depth-sensing) camera thing... Not just for games, but for media consumption as a whole... If they connect it up to Windows PCs for interacting in terms of meetings, and collaboration, and communication, you put the camera in now it's a cool thing, and it's just an example where Microsoft research did the original stuff to show, with the depth information, something great could be done. Then both the Xbox guys and the Windows guys latched onto that and now even since they latched onto it the idea of how it can be used in the office is getting much more concrete, and is pretty exciting.

So Microsoft is a very innovative company, but obviously in a hyper-competitive field, which is what makes it such a great field.

I'm not sure I understood that last point. You're talking about cameras, you were talking about like the depth sensing cameras that are in Natal?
Gates: Yes, exactly, Natal. The software libraries and applications we're doing around Natal.

And we'll basically see that in more than gaming? We'll see it in other scenarios, too?
Gates: Well, I think the value is as great for if you're in the home, as you want to manage your movies, music, home system type stuff, it's very cool there. And I think there's incredible value as we use that in the office connected to a Windows PC. So Microsoft research and the product groups have a lot going on there, because you can use the cost reduction that will take place over the years to say, "Why shouldn't that be in most office environments?"

Bill Gates may not be showing up at Microsoft headquarters every day, but he's certainly staying busy.

In an interview with CNET News, the Microsoft chairman talks about just a few of the things on his plate, including an effort to make a series of classic physics lectures available for free over the Internet.

Although it's unlikely to garner the audience of say, a sneezing panda, Gates said that putting great educational content online is an important part of getting people interested in science.

"When a lecture is presented as well as this, it draws more people in to understanding science." Gates said.

Gates also took on topics such as Google's Chrome OS and what things at Microsoft still have him excited.

One of those things, he said, is Project Natal, the technology shown at E3 this year that uses depth-sensing cameras to allow one's own hands to act as a video game controller. But Natal is not just for games, Gates said, noting that the technology is also being used by the Windows team, which sees uses both for controlling media at home as well as in a number of workplace scenarios.

"If they connect it up to Windows PCs for interacting in terms of meetings, and collaboration, and communication, you put the camera in now it's a cool thing, and it's just an example where Microsoft research did the original stuff to show, with the depth information, something great could be done," Gates said. "Then both the Xbox guys and the Windows guys latched onto that and now even since they latched onto it the idea of how it can be used in the office is getting much more concrete, and is pretty exciting."

Microsoft doesn't just want to bring gesture recognition to the Xbox with Project Natal. It also wants the technology in Windows, according to a very good source--Microsoft Chairman Bill Gates.

In an interview with CNET News this week, Gates talked about a world in which depth-sensing cameras such as the one allow people to control their PCs, game devices, and televisions. (See a video from the E3 conference below.)

Speaking about all of the technology Microsoft has cooking in its labs, Gates said: "I'd say a cool example of that, that you'll see... in a little over a year, is this (depth) camera thing." Gates said it was not just for games, "but for media consumption as a whole, and even if they connect it up to Windows PCs for interacting in terms of meetings, and collaboration, and communication."

Gates said it is an example where the project started in Microsoft research but is now being commercialized by both the Xbox and Windows units. "Both the Xbox guys and the Windows guys latched onto that and now even since they latched onto it the idea of how it can be used in the office is getting much more concrete, and is pretty exciting."

Using your body to control devices makes a lot of sense, Gates said. "I think the value is as great for if you're in the home, as you want to manage your movies, music, home system type stuff, it's very cool there," he said. "And I think there's incredible value as we use that in the office connected to a Windows PC. So Microsoft research and the product groups have a lot going on there, because you can use the cost reduction that will take place over the years to say, why shouldn't that be in most office environments."

Gates actually dropped the first hint of Natal during his joint appearance with Steve Jobs at the D: All Things Digital conference in 2007

"Imagine a game machine where you're just going to pick up the bat and swing it, or the tennis racket and swing it," Gates said.

Moderators Walt Mossberg and Kara Swisher mocked Gates, saying such a technology already exists and it's called the Wii. But Gates disagreed. "No, that's not it. You can't pick up your tennis racket."

He later added, "You can't sit there with your friends and do those natural things," he said. "That's a 3D positional device. This is video recognition. This is a camera seeing what's going on."

No matter what people thought they heard Steve Ballmer say in Chicago, Microsoft is not planning to introduce a successor to the Xbox 360 next year.

In a blog posting on Thursday night, Microsoft said that the Xbox 360 is "not even halfway" through its current lifecycle and will be here long into the next decade. The company is planning to introduce a new gesture-recognizing controller, code-named Project Natal, and that is expected next year, although Microsoft says it isn't talking timing with regards to Natal.

"As the Xbox team stated at E3 two weeks ago, we are not even halfway through the current console generation lifecycle and believe Xbox 360 will be the entertainment center in the home for long into the next decade," Microsoft's Major Nelson said in a blog post. "Project Natal will be an important part of this platform, but we have not confirmed a launch date at this time."

Of course, that is what has opened the door to all the speculation. In his posting, CNET colleague David Carnoy notes that Microsoft says it isn't planning an all-new Xbox 360 either.

That doesn't completely shut the door on new hardware for 2010. Although Microsoft has said it plans to offer Natal as an add-on to the Xbox 360, it seems reasonable that the company might well come out with some new bundle or set-up to accompany Natal's arrival as it looks to convert all of those Wii owners.

Johnny Chung Lee, the former Carnegie Mellon researcher known for finding creative ways to adapt the Wiimote, has revealed himself as one of the minions behind Project Natal, Microsoft's effort to add motion-sensing capabilities into the Xbox 360.

Lee, who is now a researcher at Microsoft, said in a blog posting that he has been working on the motion-sensing project.

"Now, I should preface by saying I don't deserve credit for anything that you saw at E3," Lee said in the blog, which he posted on Monday night. "A large team of very smart, very hard-working people were involved in building the demos you saw on stage. The part I am working on has much more to do with making sure this can transition from the E3 stage to your living room - for which there is an even larger team of very smart, very hard-working people involved."

Microsoft demoed Project Natal on Monday as part of its press conference at the E3 trade show. The technology allows a person to act as their own remote, with a depth-sensing camera capturing their motion, and software then translating it into actions.

Lee notes that he can't reveal anything beyond what Microsoft shared, but does talk a little about the technology that underlies Natal.

"The 3D sensor itself is a pretty incredible piece of equipment providing detailed 3D information about the environment similar to very expensive laser range finding systems but at a tiny fraction of the cost," Lee wrote. "Depth cameras provide you with a point cloud of the surface of objects that is fairly insensitive to various lighting conditions allowing you to do things that are simply impossible with a normal camera."

The hard work, he said, is then converting that cloud of points into human actions, something which requires some pretty sophisticated algorithms. That said, the work could lead in some even cooler directions.

"At times, working on this project has felt like a miniature 'Manhattan project' with developers and researchers from around the world to coming together to make this happen," Lee wrote. "We would all love to one day have our own personal holodeck. This is a pretty measurable step in that direction."

Before joining Microsoft, Lee gained attention for his projects using the sensor bar and remote of the Nintendo Wii to work as head-tracking devices, a multitouch user interface and more.

Lee has continued showing off his Wiimote projects since joining Microsoft, presenting at this year's Mix09 event in Las Vegas. (There's a video below, but it requires Silverlight.)

Update 2:25 p.m. I had a chance to chat with Lee briefly by phone.

For his part, Lee said he thinks Microsoft doesn't get the credit it deserves in the consumer arena.

"It's sort of a bummer that Microsoft gets kinds of a bad rap," Lee said. "It's a lot of very ambitious groups trying to do big things. Not everything makes it out the door."

Lee, who works in an applied science group that sits between the research and product arms at Microsoft, says that the company is working on some very cool stuff, though he could not go into a lot of details.

"I played a little bit with the depth cameras before Microsoft," he said. "The technology I have been able to play with since coming to Microsoft is a lot better."

The big news from Microsoft' E3 press conference was clearly its "Project Natal" motion sensing technology.

We're pretty sure that it is based on technology from 3DV systems, an Israeli start-up that we wrote about some time ago. I'm hearing that Microsoft has in fact, acquired the company, as has been reported. I have yet to get that 100 percent confirmed, however.

In any case, here's a video from Microsoft showing the technology in action.