Webware

The World Wide Web Consortium has published a draft of an interface that browsers can use to manipulate files better, one of a series of steps aimed at gradually improving the sophistication and polish of Web site interfaces.

The draft File API (application programming interface) defines a number of ways that browsers and Web sites can handle files better. One big part of it: being able to select multiple files for upload, such as on photo-sharing sites or Web-based e-mail, a task that often relies on Adobe Systems' Flash today.

But there are other aspects, too. For example, the Files interface governs the use of "blobs," or packages of raw binary data such as video files. Google has touted blobs for its Gears browser plug-in as a way to divide large videos into small chunks so that uploads can be more easily resumed if a network problem interrupts the process.

Another benefit: files are handled asynchronously, which means the browser won't freeze up while a file is being uploaded or otherwise handled, and the browser reports progress on file transfers.

The technology is one example of work to transform the Web into a better foundation for interactive applications, a move that usurps some power from computer operating systems such as Windows and that's embodied most boldly in Google's Chrome OS project.

Here's one example of use of the Files interface provided by Mike Smith, who works for the W3C on matters relating to HTML--Hypertext Markup Language, the language used to describe Web pages:

A user uses a Web-based application for reading and sending e-mail. She wants to attach multiple files to particular messages. The Web application provides an user interface that allows her to select multiple files to attach at the same time. After she selects the files, they are uploaded to the Web application asynchronously, allowing the user to perform other actions while they are uploading (for example, finishing the rest of the message she was composing before you added the file attachments). As the attachments are uploaded, the Web applications shows progress bars to indicate how much of the contents of the files have uploaded thus far.

The interface can work in conjunction with various standards including the drag-and-drop support in the HTML 5 now under development and the Web Workers technology that lets browsers better perform multiple operations simultaneously.

The interface also can help Web applications process the contents of files. For example, Smith describes a lyrics finder:

A user has on her local file system a playlist file from her favorite desktop music player. The playlist contains a list of song titles and information, and she wants to be able to easily fetch the lyrics for particular songs without needing to manually search for the lyrics on the Web. So a site can provide a Web-based application that allows her to upload her playlist. The Web application then parses the file and then presents a user interface to her, show in the contents of the file as a hyperlinked, sortable list. She can then retrieve the lyrics for any given song just by clicking on a particular song title.

Arun Ranganathan, Mozilla's standards evangelist and chairman of the WebGL working group, wrote the specification, according to Chris Blizzard, Mozilla's director of developer relations.

Standards for the Web are advancing rapidly with W3C representatives including Microsoft working in conjunction with a parallel effort, WHATWG. New standards require actual implementation in browsers before they are accepted as finished, a fact that can lead to some chaos but that helps ensure the new ideas are tested in the real world.

Firefox 3.6, in beta testing now, will support most of the Files API, according to Blizzard.

Location awareness is hot, from gamelike social services such as Foursquare and Gowalla to platforms such as Google Latitude. Now one start-up is hoping to make it as easy for any company to integrate into a Web or mobile service as it is for retailers to use PayPal. Meet SimpleGeo, which on Thursday is launching into a private beta.

Boulder, Colo.-based SimpleGeo, co-founded by former Digg engineer Joe Stump and Socialthing founder Matt Galligan (who sold the would-be FriendFeed competitor to AOL), started out as a company called Crash Corp. earlier this year. The goal was to make augmented-reality applications for mobile devices like the iPhone, but the founders said that building the location-aware infrastructure for their first game took a whopping three months.

So they changed their company name and angle: SimpleGeo's purpose is to build that infrastructure for other companies to eliminate the development hell, hoping to do for "geo" apps what PayPal did for sites requiring payment systems or Facebook did for sites requiring logins and social-networking features. The complete offering, which can also build in augmented-reality features, encompasses storage, analytics, and a software development kit (SDK).

Three versions are available: free, $399 per month, and $2,499 per month. A public version is slated to launch in the spring.

Nobody's really doing this yet, though apparently a few other start-ups are toying with similar business plans, and SimpleGeo is still new enough that it has not yet closed a round of venture funding. Because it's in private beta, we also haven't yet seen just how powerful it is (though Galligan has posted some test video to Flickr) so it's not yet possible to answer the big, glaring question: what if Google makes a big, developer-focused Latitude push that could snuff out smaller competition?

Adobe Labs on Monday released test versions of two closely related foundations for Net-based applications, Flash Player 10.1 and AIR (Adobe Integrated Runtime) 2.

Flash is widely used to bring streaming video, interactive graphics, and games to browsers; AIR, with Flash built in, is a foundation for other desktop applications. Both are instrumental to Adobe's effort to stay ahead of the gradually broadening feature set of HTML and related Web standards.

Notable Flash Player 10.1 is support for not just Mac OS X, Windows, and Linux computers, but also a variety of smartphones, though that support isn't yet built in. What is available is hardware-based decoding of the popular H.264 video format, which Adobe said improves performance and saves battery life. It also supports HTTP streaming linked with Adobe's content protection technology.

A version of Flash Player 10.1 for Palm Pre smartphones is expected later this year, Adobe said, and the final version for all systems is due in the first half of 2010.

AIR 2.0, which includes Flash Player 10.1, brings tighter integration with desktop computers. For example, it can communicate with some USB storage devices, monitor multitouch user interfaces, tap into microphone audio data, render Web pages using HTML5 and CSS version 3, and use UDP networking useful for in-game chat.

The final version of AIR 2 also is due in the first half of 2010, Adobe has said.

Google, ever eager to renovate the computing industry for the benefit of the Web and its own business, is working to link two nascent but potentially significant projects, its experimental Go programming language and its Chrome Web browser.

Gordon, Go's gopher mascot

Specifically, the company is building a foundation to let programs written in Go run directly within a Web browser endowed with Google's Native Client software. Native Client is designed to let browser-based programs run faster than is possible with today's widely used JavaScript; though it's still in its early stages, it's built into Chrome and available as a plug-in for other browsers.

A little poking around the Go source code reveals a reference to NaCl, the abbreviated name for Native Client. And Native Client is indeed on the Go agenda, said Rob Pike, one of the five core members of the Go team, in a Wednesday interview.

"We have an embryonic implementation of the NaCl support for Go using 8g," a compiler that produces code for x86 chips such as Intel's Core line, Pike said. "It's restricted by a couple of details of NaCl's implementation, but we hope to see changes to NaCl one day that will make Go a full-fledged language in that environment."

The Native Client compiler--the tool that converts what people write into software a computer can run--is specially modified to screen out a variety of software instructions that could expose a computer to an attack from a Native Client module downloaded off the Web. And the Native Client software itself checks such modules before they run. The result, if the security approach stands up to security scrutiny, is browser-based software that runs close to the speed of ordinary software that runs natively on a PC.

Rob Pike discusses the Go programming language at a Google Talk

(Credit: Google)

Native Client has been maturing, the most recent stage being inclusion of NaCl within Google's Chrome browser, though disabled by default for now. Google is using Chrome as a vehicle to distribute other Web technology, too, including Gears, which can let people use Gmail while offline, and WebGL, which gives hardware acceleration to 3D graphics in the browser.

Go is only experimental at this stage, but Google hopes to use it to produce some of the software running on its vast array of servers. Google's scale makes even academic projects potentially commercially relevant, which is enviable to many companies who've tried to get projects off the ground.

Indeed, an episode earlier in the Go team's history is illustrative. Pike, Unix co-inventor Ken Thompson, and Russ Cox all worked on the Plan 9 operating system project that, like Unix, began at Bell Labs. (Yes, Plan 9 is named after Ed Wood's famously bad movie, "Plan 9 from Outer Space.")

Unlike Unix, Plan 9 didn't have much commercial success, although Vita Nuova does sell a version called Inferno. Getting a mainstream operating system off the ground is hard: you must convince programmers, software companies, and hardware makers to embrace it; you must convince people to use it in the real world; and you must keep pace with the evolution of entrenched operating systems.

A bit of Plan 9 lives on inside the Go project, with various Plan 9 tidbits appearing in the Go source code. Pike, though, says there's not much.

Glenda, the Plan 9 bunny mascot, looks similar to Gordon, Go's gopher mascot. Both were drawn by Rob Pike's wife, illustrator Renee French.

(Credit: Bell Labs)

"The 6g/8g/5g compilers are almost completely new but are tied to the open-source Plan 9 compiler suite's C compilers and linker," Pike said. "That's really about it except for the obvious historical connection for some of the protagonists: Ken, Russ, and myself."

Programming languages face similar challenges as operating systems in getting off the ground: A lot of interdependent elements in the ecosystem must all be built simultaneously. It's what's known in the trade as the chicken-and-egg problem: you can't make a chicken without an egg or vice versa.

But Google makes things different for Go. It's devoting real resources to the project and believes it could be useful on its own servers to run software such as the Gmail service Web browsers tap into. It's got the chicken and the egg under its own roof.

And with the money Google could save by increasing the performance or efficiency of its servers even just a fraction of a percent, it has abundant financial incentive to make things work.

Marrying Go to browsers is just another aspect of the same issue.

Assuming Go and Native Client mature enough to be useful, Google can't mandate that Web developers embrace them; indeed, they generally haven't embraced Gears even though it can help with some Web site matters. But again, Google has a browser and some awfully big Web sites it can use to get the ball rolling.

(Credit: Mozilla)

It's been just under four months since Mozilla launched its pilot program for contributions, a way for users to donate to add-on developers for their time and effort.

The program was launched in tandem with a redesign of Mozilla's add-ons site that gave developers their own profile pages. Many add-on makers were already running donation programs through their own sites, but wanted the option to show up in Mozilla's catalog too.

Already it appears to be working, but on a smaller scale than some developers might have hoped. For the half dozen developers that CNET News talked to, none has made enough from it to, say, quit their day job. While Mozilla would not reveal specifics on which developers are getting the most contributions, it did provide us with the total amount given: around $20,000. An organization spokesperson said that most of that came in September and October.

Of the 500 or so developers who are participating in the program, the average contribution falls somewhere between $5 and $6, with the largest thus far being $150. All have gone through PayPal, which is the sole way to pay through Mozilla's add-on site. PayPal then gets a small fee out of each transaction, something that comes out of the developer's pocket, although this varies based on how much the user gives.

Other ways to make money

Some developers believe Mozilla has gone about the payment problem in reverse. With the current contributions program developers are given the chance to ask for money before the user even downloads the free add-on. So why not give them a way to ask for a contribution after a user has downloaded and installed it?... Read more

The industry P.R. frenzy over scams in ads and offers on social networks goes on: Facebook announced on Thursday evening in a post on its developer blog that since it updated its developer platform terms of service this summer, it has disabled two ad networks that it says were running deceptive advertisements.

This comes in the wake of allegations that some companies that power offer- and survey-related moneymaking operations for social-gaming applications on platforms like Facebook's have effectively been scamming users into paying for services without disclosing those costs. One of them, Offerpal Media, has been particularly visible in the crosshairs.

"This battle is not new and it's far from over," the post by Facebook's Nick Giano wrote. "We faced stimulus scam ads on our own system earlier this year and pushed them off the site with rigorous enforcement. We did the same months later when deceptive ads from third-party ad networks appeared in applications. We're doing that again now as we see them appear in the form of offers."

Additionally, Facebook--which has said for quite some time that many of the activities highlighted in the "app scam" controversy are already banned by its terms of service--included in the post that more than 100 developer applications have been either "suspended or brought into compliance" over advertising issues, and that more than half of them were used by at least 1 million Facebook members per month. It's not clear whether these were all related to scams, or to other advertising-related infringements like the Burger King marketing campaign that encouraged users to "unfriend" their contacts in exchange for a free cheeseburger.

Facebook representatives declined to name which ad networks or applications it has banned. But the company did ban two companies in June, Social Hour and Social Reach, citing ad network policy violations. It's possible that the two ad networks mentioned in Facebook's blog post were banned months ago, given the "since July" language.

Earlier this week, MySpace--another big destination for social-network apps--announced that it had updated its terms of service to ban app scams. Prior to that, several prominent application manufacturers announced that they had banned potentially deceptive offers, despite the fact that they are responsible for a big chunk of virtual-goods revenues.

An update was made to this post at 7:51 a.m. PT on November 6 to note that Facebook banned two ad networks in June.

With a project called Closure Tools, Google plans on Thursday to start helping developers who aspire to match the company's proficiency in creating Web sites and Web applications.

Google is a strong proponent of using JavaScript to write Web-based programs, part of its Web-centric ethos. Indeed, the company has pushed the language to its limits with services such as Gmail and Google Docs, and it developed its Chrome browser in part to enable JavaScript programs to run faster.

But writing, debugging, and optimizing heavy-duty JavaScript can be difficult--in part because a given JavaScript program sometimes works differently on different browsers. Google's open-source Closure Tools project is an attempt to help with some of these challenges.

The first in the suite of tools is the Closure Compiler, a software package designed to boil down a JavaScript program so it's smaller and runs faster. For example, a function named DisplayAddress() could be replaced with just a().

Along with the compiler come some extra tools that run in the Firefox browser. One, Closure Inspector, is an extension for Firefox's Firebug add-on designed to help programmers understand and debug the rewritten JavaScript--linking a() back to DisplayAddress(), for example. Another add-on for the Google Page Speed extension lets programmers see how much the compiler helped.

Google also plans to make the compiler available as a Web application hosted on its Google App Engine service.

The second element is called the Closure Library, a collection of prebuilt JavaScript code that lets programmers handle relatively sophisticated technology--arrays and string manipulation, for example.

Last are Closure Templates, more prewritten code to ease creation of JavaScript and HTML user interfaces.

In an earlier era, programming tools were expensive packages bought by a select few, but open-source software, new marketing strategies, and new business methods have made that approach the exception rather than the rule these days. Now programming tools are often a means to another end--encouraging programmers to produce the software that will make Windows or the Palm Pre useful and therefore popular, for example.

In Google's case, the objective is often to make the Web more popular because it sees more activity on the Web as corresponding directly with more activity on its revenue-generating search site. Among the high-profile projects to this end are Chrome, Chrome OS, and Android, all subsidized by Google's powerful search-advertising business.

One interesting contrast to Closure is another Google project called Google Web Toolkit. It's designed to accomplish some of the same goals as Closure, including paving over browser incompatibilities and producing high-performance JavaScript. But with GWT, coders write programs in Java that gets translated into JavaScript.

So one last question: why the name?

Google's reply: "Being a functional language, the concept of a function closure is fundamental to the JavaScript language."

Betting that the benefits of the move will outweigh the risks, Yahoo has released the source code underlying in-house software called Traffic Server that can speed up Web site operations.

The software works by moving some data and operations closer on the Internet to the people trying using those services. Yahoo released it as an "incubator" project under the auspices of the Apache Software Foundation, a seasoned organization for managing open-source projects and also the site that houses the Hadoop open-source project Yahoo favors for large-scale data-processing challenges.

Shelton Shugar, Yahoo's senior vice president of cloud computing, plans to announce the move at the Cloud Computing Expo in Santa Clara, Calif., on Tuesday in a keynote speech, but the software actually arrived at Apache last week.

Shelton Shugar, Yahoo's senior vice president of cloud computing

(Credit: Screenshot by Stephen Shankland/CNET)

"We've donated Traffic Server to Apache because we think it's a great piece of code, and we want to build a community around that in the same manner we built a community out of Hadoop," Shugar said in an interview.

Traffic Server is a battle-hardened package with more than 200,000 lines of C++ code. Yahoo originally got the software through its acquisition of Inktomi earlier this decade, and it's been using it ever since. Today, the software delivers 30 billion Web objects and 400 terabytes of data each day.

And Yahoo can rightly be proud of Traffic Server's performance: that comes from a surprisingly small number of Yahoo servers--between 100 and 150, said Chuck Neerdaels, vice president of data services at Yahoo. The software is set up particularly to run multiple tasks at the same time, a design well-suited to today's servers with multicore, multithreaded processors.

Source code is what humans write in a higher-level programming language; only after it's been translated into binary machine code can a computer actually run that program. When associated with an open-source project, this software is available for anyone to see, modify, and distribute, in contrast to the locked-down world of proprietary software such as Microsoft Windows. So in effect, Yahoo is allowing others not only to use Traffic Server for their own ends, but also to modify it--for example, by taking advantage of its ability at to accept plug-ins that can adapt it for different tasks.

Giving away the farm?
So isn't there a risk that Yahoo is giving away some pretty important technology that's central to its business? Plenty of start-ups today are trying to grow to Yahoo's scale, and many of them are competitors.

Some Yahoo rival might very well gain as a result, but on balance, the company thinks that it'll come out ahead. For one thing, Traffic Server in isolation is not as powerful as Traffic Server woven into Yahoo's computing fabric, the company argues.

"What we're giving up is a generic building block. What makes it really interesting at Yahoo is how we've connected it with other things to make a bigger service," Neerdaels said. As for Yahoo's major rivals: "We suspect our larger competitors already have some solution they're happy with."

Yahoo expects a number of benefits from broader development and use of Traffic Server.

"We think a lot of folks can benefit from this, and by raising the tide, we think we can benefit as well," Shugar said.

For one thing, making Traffic Server open-source software will mean that people will grow familiar in its use, making it easier for Yahoo to hire engineers who already are up to speed.

"By virtue of basing services on open-source software, we attract people who want to work on open source. They like it, and they like the idea of it. It's a skill they can take with them from one place to another," Shugar added.

For another, Yahoo can benefit from others adapting the software to a broader range of uses, he said.

Gaining influence among developers
There are intangible benefits, as well, when it comes to recognition among programmers, whose influence in some ways makes them the digital elite. Microsoft long ago learned that much of its power comes from developer allies, and Google is trying to put that lesson to good use as well by releasing many open-source projects--Google Chrome being one recent example.

Yahoo isn't in the business of selling technology to others in the manner of Amazon Web Services, Microsoft Azure, or Google App Engine. But having solid technology is essential to Yahoo. While it's willing to sell its search business and engineering skills to Microsoft, it still needs in-house expertise to power its many Web properties and to reduce its operating costs.

Here, Traffic Server is important. For example, one area where Yahoo uses Traffic Server was at Yahoo Sports for handling scores. A regular Web server sends out the Web page to a person's browser, but Traffic Server handles the JavaScript technology that periodically refreshes the contents of a scoreboard element on that page.

It's only a "trickle" of data, but at Yahoo's scale, that can be some pretty heavy work. "When they moved to using the Traffic Server front end, they shaved something like 200 machines off their back end because session management was more efficient," Neerdaels said.

Another part of Yahoo operations retrofitted with the software is Yahoo Mail, he said. Traffic Server can be used to process the cookie text files on a person's browser to figure out whether that person can be logged in automatically or the person needs to authenticate anew. It also can route traffic appropriately when, for example, a person who is "homed" to Yahoo's servers in India visits the site while in the United States.

Traffic Server also manages a lot of more nuts-and-bolts tasks. For example, it can cache Web data closer to browsers so the original Web servers that house the data aren't as overtaxed. And it can store a Web address stored in the Domain Name System to speed up network speeds.

What's it good for?
Some of these chores can be handled by existing software, such as Squid, which is already open source. But Yahoo is on a roll with its open-source work, as the company seeks to advance its internal cloud-computing infrastructure. Expect more to come.

"As various pieces of our cloud get to a point of maturity, we will open-source specific pieces," Shugar said. Future candidates include Yahoo's foundation for hosting its Web applications on a virtualized, more flexible foundation, and its Sherpa and Mobstor services for storing data.

Winning open-source allies can be difficult, and Neerdaels said it takes an engineer a good six months to fully comprehend all Traffic Server's code, so immediate gains beyond fostering goodwill are unlikely.

But in the long run, Yahoo's program could pay significant dividends. Building a series of significant open-source packages could lead to a Yahoo infrastructure that's high-power but more standard than custom-made.

It's not every day that large, significant software packages arrive on the Net in open-source form--much less a series of them that are increasingly relevant to a competitive market of large-scale Web sites.

In this case, Yahoo's gift may indeed become Yahoo's gain.

iStockphoto's Kelly Thompson

(Credit: Stephen Shankland/CNET)

Google's Gears technology may not have caught on widely in the world of Web programming, but operators of the iStockphoto photo sales site have become believers.

Among other things, Gears enables browsers to store data on a local computer, which most notably means that Web applications can be adapted to work even while offline. But for iStockphoto's purposes, it primarily means better performance for people using the site and secondarily lower operating costs for the Getty Images photo sales subsidiary.

"We're not requiring anyone to install Google Gears," the company said on an explanatory Web site. "If you do install Google Gears, though, iStock will work much faster."

Google launched the open-source Gears software in 2007, but so far, the sites that use it--among them Gmail, Google Reader, WordPress, and MySpace--are the exception rather than the rule.

Speed and money
The main motivation for the change was getting a faster site, which benefits iStockphoto's financial results, said Kelly Thompson, iStockphoto's chief operating officer.

"It was 95 percent performance and end-user experience, but let's face it: if I can get more pictures pumped out faster, with more searches, we sell more," Thompson said. "Cutting down a page load time for a user is more valuable to me than the money I'll save on bandwidth."

The company adopted Gears with no prompting from Google, he added. "We did this on our own," with Web programmers jumping on the project because "it's sexy for them to work on it."

iStockphoto activated its Gears support September 30, Thompson said. In the first 16 days of use, Gears saved the company from paying for the transfer of 132GB of data over the network and lightened its Web servers by 8.7 million communication requests--and that's with only 19,000 Gears-installed users, a "tiny portion of our traffic," he said. Those without Gears benefit, too, since iStock's Web servers are unburdened somewhat by those who do use it.

The technology works by locally storing various Web page ingredients--photo thumbnails, JavaScript program code, Cascading Style Sheet formatting files, for example. Older files are flushed periodically so the users' hard drives don't get too cluttered.

"It's a pretty basic implementation right now: the second time a user sees any image or requests a JavaScript file, it loads instantly," Thompson said. One of his developers described it as "the opposite of a drug dealer: the first hit isn't free, (but) every subsequent hit is."

Google is trying to propagate Gears, which is available as a browser plug-in. In a more aggressive move, it built Gears into its Chrome browser. And in the longer term, the HTML5 standard under development reproduces the local storage abilities of Gears, a move that stands to spread the technology more widely.

HTML5 good, IE 6 bad
Thompson is a fan of another HTML5 technology: built-in video. iStock licenses video content, as well as photos and other content, and currently streams it with Adobe Systems' Flash technology.

"We'd love to be able to ditch Flash on the video side, but it's probably a ways out," Thompson said, citing widespread use of Internet Explorer.

IE is widely loathed among Web developers for its slow performance and lack of standards compliance, and even Microsoft wishes that people would upgrade from IE 6, but it's still the single most widely used browser out there, even though Microsoft released it in 2001, just before Windows XP arrived. Microsoft released IE 7 in 2006, and it tried to improve standards compliance and security with the release of IE 8 this March.

People are gradually shifting away from IE 6, but not fast enough for Thompson's taste--or plans.

"We announced we'd drop official support for IE 6 in 2010 back at the beginning of the year. I'm not sure we're going to be able to it: the percentage of users is dropping--just not quite fast enough," he said.

From August (top) to September (below), Internet Explorer lost a bit of usage share, compared with rival browsers.

(Credit: Net Applications)

According to Net Applications statistics, IE 6 is used by 24.4 percent of people on the Web today, followed by IE 7, IE 8, Firefox 3.5, and Firefox 3, in descending order of popularity. Overall, IE has 65.7 percent share of usage.

iStockphoto has more early adopters in its population and therefore different browser preferences. The top five browsers on the site are Firefox, with 37.8 percent; IE, with 34.4 percent; Apple's Safari, with 22.3 percent; Google's Chrome, with 3.4 percent; and Opera, with 1.7 percent.

Among iStockphoto's IE traffic, the majority of people use version 7, but the tide is turning.

"We've seen an almost 2 percent migration of (IE) 6 to 8 in the last 60 days alone. We're hoping Windows 7 will push it even more quickly," Thompson said. "For us, even though it's a shrinking percentage, it still represents over 1 million visits per month, so I can't cut them off at the knees."

"I think we're dominated by geeks, designers, and small businesses, all who move more quickly than the enterprise--not to mention we're 35 percent Mac, with the iPhone about to overtake Linux for third place" among operating systems, Thompson said.