'Wii hacker' part of Microsoft's Natal effort
Johnny Chung Lee, the former Carnegie Mellon researcher known for finding creative ways to adapt the Wiimote, has revealed himself as one of the minions behind Project Natal, Microsoft's effort to add motion-sensing capabilities into the Xbox 360.
Lee, who is now a researcher at Microsoft, said in a blog posting that he has been working on the motion-sensing project.
"Now, I should preface by saying I don't deserve credit for anything that you saw at E3," Lee said in the blog, which he posted on Monday night. "A large team of very smart, very hard-working people were involved in building the demos you saw on stage. The part I am working on has much more to do with making sure this can transition from the E3 stage to your living room - for which there is an even larger team of very smart, very hard-working people involved."
Microsoft demoed Project Natal on Monday as part of its press conference at the E3 trade show. The technology allows a person to act as their own remote, with a depth-sensing camera capturing their motion, and software then translating it into actions.
Lee notes that he can't reveal anything beyond what Microsoft shared, but does talk a little about the technology that underlies Natal.
"The 3D sensor itself is a pretty incredible piece of equipment providing detailed 3D information about the environment similar to very expensive laser range finding systems but at a tiny fraction of the cost," Lee wrote. "Depth cameras provide you with a point cloud of the surface of objects that is fairly insensitive to various lighting conditions allowing you to do things that are simply impossible with a normal camera."
The hard work, he said, is then converting that cloud of points into human actions, something which requires some pretty sophisticated algorithms. That said, the work could lead in some even cooler directions.
"At times, working on this project has felt like a miniature 'Manhattan project' with developers and researchers from around the world to coming together to make this happen," Lee wrote. "We would all love to one day have our own personal holodeck. This is a pretty measurable step in that direction."
Before joining Microsoft, Lee gained attention for his projects using the sensor bar and remote of the Nintendo Wii to work as head-tracking devices, a multitouch user interface and more.
Lee has continued showing off his Wiimote projects since joining Microsoft, presenting at this year's Mix09 event in Las Vegas. (There's a video below, but it requires Silverlight.)
Update 2:25 p.m. I had a chance to chat with Lee briefly by phone.
For his part, Lee said he thinks Microsoft doesn't get the credit it deserves in the consumer arena.
"It's sort of a bummer that Microsoft gets kinds of a bad rap," Lee said. "It's a lot of very ambitious groups trying to do big things. Not everything makes it out the door."
Lee, who works in an applied science group that sits between the research and product arms at Microsoft, says that the company is working on some very cool stuff, though he could not go into a lot of details.
"I played a little bit with the depth cameras before Microsoft," he said. "The technology I have been able to play with since coming to Microsoft is a lot better."
During her years at CNET News, Ina Fried has changed beats several times, changed genders once, and covered both of the Pirates of Silicon Valley. These days, most of her attention is focused on Microsoft. E-mail Ina. 
Nintendo must be kicking their self for not hiring this guy.
"If its by Ina Fried, I don't read it"
Your loss then. The reporter has a lot of interesting news to report about the happenings of Microsoft.
Fearghail plays the same hate games the GWB, Cheney, Osama, and the rest of the "run the world by fear and hate" do.
Am I the only person who doesn't want a motion-controlled video game? The point of the games I play is to relax and let my character do the work for me. Don't get me wrong, the Wii Sports games have a surprisingly efficient motion controller, but even so I've managed to learn the vital motions so that I can bowl a perfect game and get perfect scores in Target Practice for golf (the hard one...) without ever getting off the couch. I don't want to have to hold my arms at an awkward position to aim down the barrel of a digital gun. If I wanted to do so, I'd have gone to the range and shot a real gun.
The Wii seems like a novelty to me. There are only so many motions, so many games you can base around human-body movement, that the games are going to get old pretty fast. And then what? Smash Brothers, Mario, or Zelda comes out. Games that, by the grace of Dr.Robotnik, require buttons.
Play Metroid, Zack and Wiki or Eleedees to see great examples.
This microsoft system is brillent, but I think they need something else to complete it.
They need a basic wireless one-handed controller to deal with charecter movement.
As it stands you could aim at the screen with, say, your figures....but how would your charecter move?
As for limited motions, that applys certainly to gesture-mapping, but not to proper wii games that replicate the angles and intensitys of motion...which approach infinity. (or, if we want to get accurate, its 2axis of analogue sensing angle which are fairly accurate + 3 axis of displacement sensativity + all the buttons/stick + the same for the nunchuck + The 1024x1024 resolution of the IR pointer....thats a LOT of data!).
Motion plus is set to improve that by suppling the missing axis. (thus making 1:1 sword fighting, and sports games possible...and there is *a lot* more possibily actions in a real players motions then with buttons)
" I don't want to have to hold my arms at an awkward position to aim down the barrel of a digital gun"
You dont have to hold your arms in any positon, just point at the screen. Everyone hypes up/down the Wiis motion sensor, but the IR is the best bit and tis missunderstood.
Its not like a shotting range when playing FPS. Its akin to a mouse or stylus in how it feels, and the movements dont have to be big at all. Its not 1:1 like duckhunt.
Its a common missunderstanding that you need to hold your arms up to use a wiimote.
As long as your not pressed right upto the screen, you can lay your hands down pretty low in your lap and its very comfortable.
Also,
ps. Thank god someone else calls him Robotnik. I hate this eggman nonsense!
pps. Opera 9.6 here, and this site works fine.
Now if your tired of Winjunk all together use Ubuntu or OSX. UBUNTU=Free OSX=$
But, with the advanceness of the Natal tech....why all the boring demos to demo it?
Why not impliment his really cool headtracking techique? That would be utterly fantastic on nearly all first person Xbox games.
What console in history had a 33% failure rate? M$ released the POS early knowing of its flaws as well.
Nintendo should've hired this guy imo.
i dream of holodecks one day too, but dont think M$ is going to bring it to you, they are followers, not creators or innovators. find one M$ product thats original.
This is very interesting technology and kudos are deserved by all those who have been working on it.
But I see some big, pink elephants in the middle of Microsoft's living room that everyone seems to be ignoring:
- When the world is trending towards smaller, more personal technology such as cellphones and netbooks Microsoft seems to be focused on gargantuan consumers of living space such as "surface computing" and Natal ("not-at-all?"). It seems a little egotistical that they think people will sacrifice such a huge part of their homes because Microsoft products are so great.
- Did anyone else have a "WT*!" moment with the family in the video? Not really a point, just wanted to say it.
- At what point does the simulation get close enough to the real thing for the real thing to be an easier (and cheaper) option? I'm talking effort and physical activity here, and a real football is a much cheaper, healthier and more parentally attractive option than a holodeck in your living room.
- How many families have a 10'x10' open space in their living room for gaming? How many of these spaces also have a 52" HD TV in them? What percentage of families can afford this?
- How many people over the age of 12 will feel overly self-conscious when asked to crawl around on the floor, punch and kick imaginary things?
- It's all fun and games until someone breaks an arm or get lacerated on shards of a broken vase.
Then it's Lawyer Time!
you want to rag on a company because it's successful, well then why only rag on 1, because you are too stoopid to learn to use linux instead, why not rag on exxon, why not rag on ford or honda, they have had a far more detremental effect on this planet.
Grow up.
PS. don't quote me gigaflops and other cell processor BS, I know exactly what the hardware is capable of and that's why I KNOW the 360's design is better.
As far as everything else, that's more a question of how the technology is deployed, and while MS is probably going to make a few Natal games, their mostly releasing the software developer kit and trusting that other companies will be able to make something incredible with it.
Though I'm interested in weather or not Natal capability will be added to Windows 7 or 8. Their adding multitouch capabilities, but with a little tweaking this technology will allow a lot of that capability to be mimiced using a webcam rather than a touchscreen.
HOWEVER, if you noticed, another article mentioned that this system is smart enough to see bits of body parts and draw in the rest for accuracy. THEREFORE, I believe they will allow 3rd party vendors to play the role of the 'bad guy' by selling add-on accessories. For example, a lightweight oversized plastic gun case with attachments for Call of Duty (possibly configurable to represent different weapons) ... the market has already seen that the public will spend significant money on plastic guitars / drums ... so why not a plastic / foam rubber golf club, tennis racket, gun, etc ...
- by Schratboy July 15, 2009 6:39 AM PDT
- This is like the Matrix. Now the machines actually are the puppet masters. The dumbing-down of the world for commercial exploitation is nearly complete.
- Reply to this comment
-
(22 Comments)