Apple

I have proof from an expert that the iPhone interface really is better. Who's the expert? My 3-year-old son.

Over the years, I've seen countless newbies struggle to use the latest gadget, computer, or software. I like new technology, but it's been work hauling myself up learning curves.

But I'm convinced that after years stuck with only modest tweaks to the WIMP interface--windows, icons, menus, pointing device--real change is upon us. That's chiefly because the pointing devices now can be your own fingers.

Levi types random words on the iPhone's notepad application.

(Credit: Stephen Shankland/CNET News)

Within moments of his first crack at an iPhone, my son, Levi, had figured out how to flip from one photo to another by flicking his finger across the screen. He understood with no coaching how to steer the simulated steel ball around the holes in the Labyrinth game by tilting the phone. He loves to type nonsense words on the notepad application using the virtual keyboard, deleting them once they've been read. In the three months since I got the iPhone 3G, Levi has learned to take photos, browse them, change the phone's wallpaper, and, unfortunately, turn off Wi-Fi and switch on airplane mode.

My proudest moment came when Levi issued his first tweet, borrowing my account: "Eesfrrgjlphdvlksxnjjktwsdvnjmmkbvvnn." Though it was largely a matter of chance, of course, he could do it because he likes the cute bluebird icon of the Twitterific application, and touching it with his finger triggers entertaining interactions.

And I was intrigued when Levi tried unsuccessfully to use the phone's accelerometer to play JellyCar, trying to spur the car by tipping the iPhone so the car would "roll" downhill faster. Note to JellyCar developers: your user interface needs work.

Levi's first tweet

(Credit: CNET News)

As a parent, of course, it's tempting to assume that Levi's accomplishments are the result of his astounding intelligence. But of course much of the credit has to go to Apple and others who've advanced the state of the interface art.

"Human beings are a lot more programmed to manipulate things with our hands and fingers," said Dan Saffer, a founder of Kicker Studio and author of Designing Gestural Interfaces. "I was at a party with a Microsoft Surface table. There was an infant playing with it, not even a year old, pushing photos around and squealing. It's amazing how much it makes sophisticated computing power accessible to a hugely wide segment of the population."

Keyboards and mice aren't being replaced--they offer speed and precision for typing words, entering data, navigating documents, and issuing commands. But they are becoming just one of a host of mechanisms.

Touch screens, available on some Hewlett-Packard computers, are a big part of the revolution, letting people interact more directly rather than relying on a mouse, joystick, or other indirect pointing device. Multitouch sensors, which can detect multiple fingers simultaneously, add more sophistication, such as the ability to shrink a photo by making a pinching gesture on a trackpad. Newer Apple laptops offer more extensive use of multitouch, though at this stage only through the trackpad rather than a touch screen.

Levi took this picture of his feet with my iPhone while he was sitting in his car seat.

(Credit: Levi)

Computing devices also are getting ears and eyes. Speech recognition is available in rough form to power phone search on various phones with services from Yahoo, Google, Microsoft, and Vlingo. FluidTunes lets you control your iTunes library by waving your hands in front of a Mac's video camera.

Intuitive, physical interfaces aren't just for kids. I was stunned to see my technophobic mother-in-law gleefully bowling with a Wii last year. Sure, she couldn't have installed Nintendo's still-popular gaming device if we'd paid her, but using it was as easy as tossing a pebble in a pond.

"There are Wii bowling tournaments now for elders. It takes a sport they love, but there's no weight of the ball anymore. They can play it in a wheelchair. It's a huge hit at nursing homes," Saffer said.

It's not just that devices are easier to use when you can touch the interface, he said. It's that it's easier to learn by watching others use them.

"One interesting thing about touch screens is there's this whole realm of observation you don't have with standard computer setups, where the icons are smaller, and it's hard to tell what people are doing by watching," he said. "You can learn how to use an iPhone by watching people flip through it for a second. You can get it in a way you can't with a standard phone, where you're watching people push buttons to get through menus."

Of course, immersion helps, too. Levi's parents spend altogether too much time punching at keyboards and staring at screens, so he's got plenty of examples to emulate his elders. As a camera buff, I'm delighted when Levi pretends to take pictures--he made a toy camera out of Lego once.

A view of Microsoft's Surface device at the Sheraton Gateway Hotel shows a list of entertainment options for hotel guests.

(Credit: Stefanie Olsen/CNET News)

But I vacillate between pleasure that he's learning how to use technology and fear that he'll grow up ignorant of the non-electronic world. I'm prone to inordinate "screen time," a term heavily freighted with negative baggage in our household, and Levi's childhood will be far more digitally immersed than mine.

And perhaps worse, there's the prospect of losing my status as resident guru. There are plenty of more technically proficient people in my orbit, but none of them live in my house, and Levi doesn't ask any of them to read his typed nonsense words.

Most families come to some sort of reckoning when their son beats their dad in basketball. Ours will come when my wife asks Levi for technical support.

If you're intrigued by multitouch technology as much as I am, you've probably been following it pretty closely and are at least impressed by its potential.

But what could it do better? According to Apple, taking the "touch" out of multitouch would be a good first step.

According to an article on AppleInsider, Apple has issued a 30-page patent that touches on the implementation of proximity sensors into its multitouch technology on devices larger than the iPhone.

The potential innards of a multitouch panel with proximity sensors don't do much for me, but robots may feel differently.

(Credit: Apple)

The multitouch sensors combined with proximity sensors would let users interact with the given interface without actually having to touch the screen. Now, this seems a tad ridiculous to me, and is anyone really too lazy to move their finger an extra inch? Yeah they are, but that doesn't make it a good idea.

Apple sees some different applications for the technology. According to the company, users would have the capability to turn off the entire touch-screen panel, or just portions of it. In addition, users would able to power down one or more of the computer's systems by dimming or brightening the screen as they see fit.

Awesome, huh?! Alas, no. OK, I may be missing something, but why would you need a proximity sensor to do this? You could just move your finger another inch and accomplish the same thing. The only unique feature Apple cited from the filing was the idea that you could highlight virtual buttons on a display without touching them. This could prepare the button for actually being pushed. Again, how is this useful?

A new Apple patent application could provide ways to control a computer with a combination of input technologies.

(Credit: U.S. PTO)

Apple may be thinking about adding new ways to improve the multitouch interface work that is central to the company's plan for the future.

Unwired View unearthed a patent application filed by Apple (thanks, Gizmodo) containing ideas for a user interface system that builds on the multitouch input used on the iPhone by adding technology for voice recognition and even facial recognition.

Wayne Westerman and John Elias, the brains behind a multitouch interface company called Fingerworks, acquired by Apple in 2005, are listed as inventors on the patent application, as they have been for several other multitouch patents coming out of Apple.

The idea behind the latest patent application is to combine input from different sources, whether that's the now-familiar iPhone multitouch concept, voice recognition, or facial expressions.

Systems may have multiple input means. However, each input means is typically operated independently...in a nonseamless way. There is no synergy between them. They do not work together or cooperate for a common goal such as improving the input experience.

That's what Apple hopes to do with the system it's trying to patent--combine multiple forms of input in order to more efficiently control a computer. For example, you could select an object with a finger gesture, order the computer with a voice command to change that object's color to blue, and then tell the computer where you want to place the object by staring at the lower right-hand corner of the screen.

Patent applications, as a rule, are designed to cover as wide an array of possible applications for the technology as the author can think of, so don't expect to see a Mac or iPhone with all of that stuff just yet. Still, it does seem that Apple is putting an awful lot of time into user interface design these days.

Apple could be readying a notebook trackpad that lets you practice your Rock Band form while using your Mac.

A patent application unearthed by MacRumors.com seeks protection for multitouch technology similar to that introduced by Apple on the MacBook Air's trackpad. This time, however, MacBook users could use as many as four fingers positioned in "chords" to execute different tasks in Mac OS X, such as bringing all the application windows to the front or opening up the Dashboard.

A new Apple patent could bring new multitouch trackpad features to future Macs.

(Credit: MacRumors.com)

This could even be extended to individual tasks within applications, such as cut and paste. Those were the examples provided in the patent application, but it wouldn't be hard to imagine extending this to things like a browser, such as opening a link in a new tab or performing different tasks within iTunes.

A word of warning, as always, about patent applications: there's no guarantee that this technology will make it into a shipping product, or if it does, whether that will be out anytime soon. Still, it's clear that Apple has made developing advanced multitouch input methods a priority, as it continues to evolve the way people interact with their Macs and iPhones.