XML, it seems, has run out of steam for Google.
Google said Monday that it has created an open-source project for a data interchange format called Protocol Buffers.
The software is meant to solve the problem of sharing information in a wide range of formats between servers at high speed. It's also designed to let companies like Google upgrade software on a network of connected servers without causing hiccups.
Google thought of using XML as a lingua franca to send messages between its different servers. But XML can be complicated to work with and, more significantly, creates large files that can slow application performance.
Protocol Buffers is an alternative way of describing the format of data that is being sent over the network or stored to a hard drive. Unlike XML, it's a compact format and is designed to be simple to use, according to Kenton Varda of Google's Software Engineering Team.
Varda wrote in the company's open-source blog:
Protocol Buffers allow you to define simple data structures in a special definition language, then compile them to produce classes to represent those structures in the language of your choice. These classes come complete with heavily-optimized code to parse and serialize your message in an extremely compact format. Best of all, the classes are easy to use: each field has simple "get" and "set" methods, and once you're ready, serializing the whole thing to--or parsing it from--a byte array or an I/O stream just takes a single method call.
Matt Cutts, a software engineer who heads Google's Webspam team, said in a blog late Monday that Protocol Buffers automatically generates Java, Python, or C++ code:
Think of Protocol Buffers as a very compact way of encoding data in a binary format. A programmer can write a simple description of a protocol or structured data and Google's code will autogenerate a class in C++, Java, or Python to read, write, and parse the protocol. Given a protocol buffer, you can write it to disk, send it over the network wire, and do any number of interesting tricks. Any medium-sized company (and quite a few startups!) should find Protocol Buffers very handy.
The software is available the Apache Software License 2.0.
Like a lot of what Google's engineering team does, this seems to make sense. XML has long been criticized as being too slow, which has led to controversial efforts to standardize XML compression.
But given the huge investment in XML, it doesn't look like Protocol Buffers will replace it. Instead, it will be used--certainly by Google and likely others--for Web applications that need a very efficient way to move around data in multiple formats.
Members of an industry group called the Open Geospatial Consortium have approved Google's KML technology as an open standard for describing some geographic data.
KML is used to manage the display of geospatial information in Google Earth, the company's software for flying over the surface of a virtual globe. With its 3D coordinate-based system, people can create models of city buildings, draw a line showing where they hiked, or overlay their own custom place names on a generic map.
Google hopes standardizing KML will help mean broader use for the map description language, but already, even rivals such as Microsoft have embraced it. This view shows Microsoft's Live Maps with a KML overly describing Glenn Canyon National Recreation Area.
(Credit: Microsoft)Google already shared its KML format openly, and others had used it in software products, but Google now hopes that its status as an official standard will decrease barriers to further adoption.
"What OGC brings to the table is...everyone has confidence we won't take advantage of the format or change it in a way that will harm anyone," said Michael Weiss-Malik, Google's KML product manager. "The goal is to prevent market fragmentation," in which different technology uses different standards.
File formats may sound mundane, but they can give strategic value to those who control them as a gateway to the data held by people and companies. In one high-profile example, open-source allies launched an attack on Microsoft's Office stronghold with the OpenOffice.org software, which could mostly read Microsoft's file formats.
One front in that war was an effort to set OpenOffice's file formats as an industry standard called ODF (OpenDocument Format), a move Microsoft countered with its own OOMXL effort, which Google opposed.
It didn't seem like there was powerful reluctance to use KML. For example, the latest Virtual Earth and Live Maps technology from Google rival Microsoft can use KML to let users export user information to navigation devices. And the Microsoft site can overlay KML files from the Internet onto its Live Maps--here's a (slow-loading) link to one from the National Resources Defense Council that describes expected effects from global warming to various national parks, along with the park boundaries.
But standardization will make KML more palatable, Weiss-Malik said. "Governments like to say they can publish to OGC KML instead of Google KML," he said.
And he expects to see a new era blossom of personal map publishing, all powered by KML. "We're just starting to see the birth of map publishing," he said.
KML stands for Keyhole Markup Language. It initially was developed by Keyhole, the satellite imagery company Google acquired in 2004. Keyhole's technology was built into the Google Maps site and the Google Earth software.
The standard, which geographic information system (GIS) software specialist Galdos Systems helped bring to the standardization process, is based on KML 2.2. The official KML standard can be downloaded from the OGC Web site.
Correction done to Alex Brown's affiliation.
The ISO has taken over control of the Open XML specification and started a committee to consider harmonization with the OpenDocument Format (ODF).
Wednesday was the last day that all resolutions to the new standard, called ISO/IEC 29500, were accepted, according to Brian Jones, a program manager for office at Microsoft who has been involved in the standardization process.
Last week, the ISO (International Organization for Standardization) announced that Office Open XML (OOXML) gained enough votes to pass as a standard.
Alex Brown, a member of the U.K.'s national standards body who led the Ballot Resolution Meeting in February, laid out what happens next now that Open XML is a standard in his blog.
Three committees have been formed to usher development of the standard going forward, two of which deal with handling Open XML.
The third is dealing specifically with interoperability between Open XML and ODF.
The most extreme voices in this debate are unhappy about Open XML's standardization.
But people dealing with exchanging documents and writing document-oriented software should take note that interoperability is now part of the ISO's digital documents charter. How that work will interact with other ongoing projects is unclear, but it is now be addressed at the level of national standards bodies.
"The DIN Delegate (DIN is the German national standards body) presented an update on the work that they have been doing around translation between the Open XML formats and ODF. I've discussed this a number of times before as being a key piece of the harmonization work," according to Jones.
Meanwhile, the protested Norway vote did not go down without a demonstration.
On Wednesday, sympathizers of the Norwegian committee that was essentially overruled by Norway's standards body to vote yes on the Open XML ballads staged a protest.
If anyone thinks that standards don't matter, take a look at these photos on Flickr of the event.
These people are protesting the fact that the technical committee tasked with forming Norway's position voted against Open XML ISO certification. Yet, Standards Norway decided to vote for it even though the country's issues were not sufficiently addressed.
One committee member, Geir Isene, wrote a quick summary of the drizzly event in his blog, saying that it was only the second IT-related protest in Norway.
European antitrust regulators are investigating whether Microsoft abused its desktop software market dominance in its effort to standardize the Office Open XML file formats.
The European Commission's antitrust regulatory body sent queries to several European countries to see how the standards-setting process was working, a spokesperson confirmed on Wednesday.
The investigation is still ongoing, he added.
The Wall Street Journal in February reported that the investigation had started.
In a letter seen by CNET News.com, European regulators queried the national standards body in Norway to gain details into the local standardization process. Specifically, the European Commission sought information on attempts to influence the debate or vote over the standards proposal.In response, Standards Norway said there was heated debate but not any "inappropriate behavior that endangered our process," according to a document seen by CNET News.com.
Office Open XML (OOXML) is a technical specification that describes the inner workings of how to read and create Microsoft Office documents. In 2005, Microsoft started a process to standardize Open XML in an effort to appeal to government customers who favor standards-based software and improve interoperability with third-party products.
On Wednesday, the International Organization for Standardization (ISO) announced that Open XML received enough for votes for it to be ratified as a standard.
The Commission query to Standards Norway came in February, before the results of the ISO ballot were published. The approval of Open XML reversed a previous attempt in September at standards approval that failed.
There was intense lobbying by Microsoft, IBM, and their partners to influence the international delegates from national standards bodies who participated in the voting.
In some cases, people favorable to Microsoft's pro-Open XML position joined standards bodies late in the process, bringing protests from Microsoft foes.
In addition, there have been reports of irregularities in the run-up to the most recent voting, which ended Saturday.
The head of the committee established to form Norway's position on Open XML wrote a letter to the ISO, complaining that the country's changed yes vote did not represent the views of most committee members.
Standards Norway, however, issued a statement indicating that its position will remain yes.
Microsoft's director of corporate standards, Jason Matusow, said on Tuesday that he expects IBM and its allies will launch "an orchestrated process attack in the hopes of overturning the ratification of Open XML, or at least to discredit what has come out of this long, global process."
In response, an IBM spokesperson on Wednesday said: "As always, the sentiment has to be organic. It will be up to people and organizations in individual countries to decide whether they want to try to appeal this."
Update 3:47 p.m. Pacific: Microsoft's general manager of standards and interoperability Tom Robertson said that Microsoft, too, has been queried as part of the investigation.
He said that Microsoft will "fully cooperate" with any investigation from the Commission.
In response to the accusations of stacking committees, Robertson said that IBM and other competitors have done exactly what Microsoft is accused of doing. For example, an employee from Google, which opposed Open XML standardization, joined the Danish national committee only three days before a vote.
"It seems that one of the main concerns that people have raised about the process is the broad-based participation in the standards body deliberation," he said. "I think it's ironic IBM is complaining about new members in national standards bodies when they have been working around the clock to get people to join."
Clarification: Robertson said that a Google employee joined the Finnish committee but a Microsoft spokesperson said that it was in Denmark. The quote was changed.
Now that Office Open XML (OOXML) has been certified as an ISO standard, there is a possibility that the vote leading to that result will be challenged. It seems Microsoft is already counting on it.
The ISO on Wednesday officially announced that Open XML received enough votes to become a standard, reversing a previous attempt in September that failed.
Those complaints could lead to a formal appeal of the votes which, under ISO rules, need to be lodged by those countries in the next two months.
Microsoft's director of corporate standards, Jason Matusow, wrote in a blog on Tuesday that challenges to the overall process will likely come. The source? IBM.
"We now see IBM/et al driving an orchestrated process attack in the hopes of overturning the ratification of Open XML, or at least to discredit what has come out of this long, global process," Matusow wrote.
IBM representatives heavily lobbied national standards bodies against approval in the run-up to the vote. But the company itself cannot lodge a formal complaint, a company spokesman said Wednesday.
"As always, the sentiment has to be organic. It will be up to people and organizations in individual countries to decide whether they want to try to appeal this," he said.
Instead, IBM is calling for reform of the standards process and "harmonization" between Open XML and ODF, the OpenDocument Format (ODF), another standardized format.
"While fully cognizant of these current results, I'm energized to take the bigger fight for openness to the next level with the thousands of individuals who are now convinced that the standards system needs fixing, and soon. I hope you'll take part," wrote Bob Sutor, IBM's vice president of open source and standards, who has been a vocal opponent of Open XML standardization.
The case of Norway
On Tuesday, the head of the committee in charge of forming Norway's position on Open XML, Steve Pepper, sent a letter to the ISO complaining that Norway's Yes position did not reflect the views of most participants in that committee.
Standards Norway responded to those criticisms, explaining why Norway changed its vote to Yes even though not all of its issues with the specification were addressed.
The letter from Standards Norway appears to indicate that the country's position will not be changed or suspended during an appeal. An ISO representative on Tuesday said that the ISO had not received a formal protest from its Norwegian delegation, indicating that Pepper's complaint did not represent the country's overall position.
In his blog, Microsoft's Matusow reproduced an English translation of Standards Norway's response to Pepper's complaint as well as letter from Germany's national standards organization.
In Germany, too, there were accusations of improprieties in what was a close vote, but Germany's standards organization said it will not change its Yes position to No or Abstain.
"The steering committee has accepted the process as compliant with the rules with a majority of 7 to 6 and therefore it has seen no reason to lift the decision of the working group. If the majority of the working group would have been convinced that the process of dealing with and voting were noncompliant to the rules then the German vote would have been changed to abstain," according to a translated statement.
The voting record shows that 75 percent of countries voted to approve and 14 percent voted against. To pass, it needed a two-thirds majority and not more than a quarter opposed.
As such, a change in one country's vote would not affect the overall result.
Meanwhile, one of the participants in Standards Norway's Open XML vote, Geir Isene, speculated that the European Union will investigate what happened in that country.
In February, the Wall Street Journal reported that European antitrust regulators were investigating earlier steps in the ISO voting process.
As expected, the ISO on Wednesday announced that Office Open XML (OOXML) has been approved as a standard, marking the end a long and sometimes contentious path.
The Geneva-based International Organization for Standardization (ISO) issued a press release with the details of a vote that showed Open XML receiving 75 percent approval and 14 percent disapproval. It needed two-thirds approval and not more than 25 percent disapproval to pass.
The effort to make Open XML an ISO standard did not pass a ballot in September, which precipitated a follow-on Ballot Resolution Meeting in February where, after redundancies were eliminated, over 1,000 issues were considered, according to the ISO.
Following the BRM, delegates from participants had the option of changing their position from No or Abstain to Yes--something that enough did to have the effort pass.
Microsoft originally submitted the Open XML file formats to standards organization Ecma International in 2005. Ecma then proposed the specification to ISO in its Fast Track process, which many considered to be too fast for a complicated specification.
ISO standards status means that software that uses Open XML, notably Microsoft's Office products, will be more attractive to governments and large corporate customers that prefer to purchase ISO-certified goods.
It also means that development of the specification will be done through the ISO, which counts members from over 100 countries. There were 87 countries which participated in the Open XML vote.
Microsoft and other software companies that support Open XML in their products, such as Apple and Novell, are expected to conform to the standard as it changes over time.
The entire endeavor was opposed by many, although certainly not all, open-source advocates who feared that standards status would give Microsoft more market power.
The run-up to the vote was marked by intense lobbying from Microsoft, IBM, and their business partners.
There were reports of what have been called irregularities in the vote which ended Saturday. The head of the committee which formed Standards Norway's position lodged a complaint saying that the yes vote did not reflect the opposition of the majority of the committee.
However, ISO spokesperson Roger Frost on Tuesday said that the organization has not received a complaint from its Norwegian ISO member, referring the matter to Standards Norway.
It's all over except the press release. But in other ways, it's just the beginning.
After a document appeared on Tuesday showing that Office Open XML (OOXML) gained enough votes to be ratified as an ISO standard, Microsoft on Tuesday confirmed the result.
Once final, ISO/IEC certification means that development of the specification, officially called Draft International Standard (DIS) 29500, will be done by members of the ISO, an international organization with representatives from over 100 countries.
On a technical level, changes proposed during the standardization process will need to be incorporated into Open XML, which is now the default document format in Microsoft Office 2007.
In the near term, that means Microsoft, Novell, and other companies that have software that works with the file formats will need to update their products. As the specification evolves in the future, these companies are expected to conform to the changes.
There remains distrust of Microsoft's efforts to promote interoperability between its products and others, including open-source software. But ISO ratification is a significant step towards Microsoft's pledge to support standards, said Peter O'Kelly, an analyst at the Burton Group.
Open XML will now be subject to more scrutiny on a technical level and people can feel less apprehensive about any possible legal entanglements from writing software based on the specification, he said.
"A lot of people continue to believe that Microsoft hasn't changed its modus operandi and the onus is on Microsoft to very clearly demonstrate a track record," O'Kelly said. "It's not like 1998. They really are enlightened about the importance of standards."
Asked what impact the ISO status would give Open XML, standards expert Jan van den Beld who now works for the pro-Microsoft industry group CompTIA, said "not much at the very moment."
Government customers and large corporations that favor certified standards now have a choice, he said. "It certainly is one less barrier to doing business in an environment that otherwise would be difficult," van den Beld said.
Not buying it
But many people committed to standards, including open-source advocates, are certain to be dismayed by the ISO ratification.
A number of national standards bodies voted "no" in the Open XML vote or abstained even after a Ballot Resolution Meeting (BRM) in February, which was meant to resolve technical issues. The tally shows that 75 percent voted to approve with 14 percent voting against.
A representative from Standards Norge, Norway's standards body has lodged a complaint over how the voting was conducted but a reversal to a "no" vote does not look like it will affect the overall result.
IBM executives lobbied heavily against the standards bid, arguing that Open XML was redundant with the OpenDocument Format (ODF) standard, technically flawed, and not sufficiently "open." A spokesperson declined to comment before the official ISO announcement.
A few days before national standards bodies were to submit their votes, the Free Software Foundation issued a legal analysis saying that the legal protections on Open XML were not to be trusted.
Some issues raised in the long-running debate seem intractable.
Microsoft executives and others have said that different "standards" suit different purposes, while others claim that multiple standards for the same purpose is wrong.
The episode has also stirred up intense anti-Microsoft sentiment.
The editor of the ODF specification, Patrick Durusau, lobbied for Open XML ISO approval to improve interoperability with ODF. At one point, he said that businesses opposed to Open XML were operating with "spite as a business strategy."
One of the most common complaints was that Microsoft and Ecma--the standards body that controls the specification--sought ISO status through its accelerated Fast-Track process, which made thorough examination of the 6,000 specification challenging.
Standards expert and ODF advocate Andrew Updegrove predicted on Tuesday that there will be a thorough review of the entire process.
"Clearly some changes need to be made in how the process works, so that the next time such an important and commercially strategic standard is processed, the process works better than this," he said.
Update: Microsoft has confirmed that Office Open XML has been ratified as an ISO standard. See updated information below.
A document that appears to be the voting record over a contentious ISO vote shows Office Open XML (OOXML) having enough support to pass as a standard.
The PDF document appeared on at least two Web sites Tuesday morning.
The results in the document show that Open XML received 75 percent approval from the "P-members" of the International Organization for Standardization (ISO) committee and 10 negative votes, or 14 percent.
To become certified, Open XML needed at least two-thirds of the votes to be "yes" and not more than one-quarter to be "no."
ISO spokesperson Roger Frost on Tuesday said he could not confirm the results.
"We will be issuing our press release on Wednesday," Frost said.
The document correlates with an unofficial tally done by Open Malaysia and standards expert Andrew Updegrove on Sunday, a day after the voting on Open XML closed.
A Microsoft representative declined to comment on the appearance of the unofficial tally, saying that it will wait until an official statement from the ISO.
An international document standards advocacy group called the OpenDoc Society posted what appear to be the official voting record on a mailing list.
In a letter to OpenDoc Society members, board member Michiel Leenaars said the impact of OOXML standardization will have a minimal impact on the adoption of OpenDocument Format (ODF), another document standard favored by most open-source advocates.
OOXML which was submitted by Microsoft to ECMA, and by ECMA to ISO, has literally crawled through the needles eye. After a year of discussion and repairs it still receives the very minimum of support. The BRM (Ballot Resolution Meeting held in February) convinced some yet unconvinced others, and counter votes from large countries like China, India, Brazil, Canada, South Africa and Iran speak volumes. This must be one of the worst results ever for a standard to pass within ISO/JTC1 in years.
Norway on Monday lodged a formal protest over its Yes vote with an eye to reversing it. Standards Norway responded to the complaint to the ISO, saying that it voted for approval to influence future development.
It does not appear as if a change to Norway's vote would affect the overall result.
Update 8:15 a.m. Pacific: ZDNet has posted the document here, which is a Zip file of PDF.
Update 8:44 a.m. Pacific: Microsoft on Tuesday issued a press release confirming the information found in document posted by the OpenDoc Society. Microsoft said that after the end of voting, 86 percent of the 87 countries participating voted to approve Open XML as a standard."The input from technical experts, customers and governments around the world has greatly improved the Open XML specification and will make it even more useful to developers and customers. Once it is formally approved, we are committed to supporting this specification in our products, and we will continue to work with standards bodies, governments and the industry to promote greater interoperability and innovation," said Tom Robertson, Microsoft's general manager of standards and interoperability, in a statement.
Norway's national standards body has lodged a formal protest over the country's vote on Office Open XML (OOXML) at the ISO.
The chairman of the Norway's standards committee said there were "serious irregularities" with the voting process and that the vote should be changed to oppose ISO certification of Open XML.
In the letter, sent on Monday to the ISO, Steve Pepper asked that Norway's vote be suspended pending the results of an investigation.
Here is the text of what Pepper wrote in the letter (PDF):
I am writing to you in my capacity as Chairman (of 13 years standing) of the Norwegian mirror committee to ISO/IEC JTC 1/SC 34. I wish to inform you of serious irregularities in connection with the Norwegian vote on ISO/IEC DIS 29500 (Office Open XML) and to lodge a formal protest.
You will have been notified that Norway voted to approve OOXML in this ballot. This decision does not reflect the view of the vast majority of the Norwegian committee, 80% of which was against changing Norway's vote from No with comments to Yes.
Because of this irregularity, a call has been made for an investigation by the Norwegian Ministry of Trade and Industry with a view to changing the vote.
I hereby request that the Norwegian decision be suspended pending the results of this investigation.
Reports of the voting process surfaced on Friday at Computerworld Norge. In a translation of the article at Groklaw, participants said that representatives from Microsoft and Statoilhydro on the Standards Norge committee voted for approval of Open XML. But the other members of the committee were opposed because their comments on the specification were not addressed. Yet the overall vote changed from changed from No to Yes.
The text of Pepper's letter to the ISO was posted by Gere Isene, an IT professional in Norway.
An official announcement from the ISO on whether Open XML received enough votes to pass is expected Wednesday. It's not clear how an appeal to the process from Norway, or potentially other countries, will delay that communication and whether potential changes in votes will affect the decision.
Based on a tally from Open Malaysia and standards expert and advocate Andrew Updegrove, it appeared on Sunday that Open XML had the required voting to pass.
Update 6:57 a.m. Pacific. Microsoft employee Stephen McGibbon posted an English translation of a statement from Standards Norge responding to the criticism of the process. The statement calls for "critically evaluate the Fast-Track procedure at the ISO," the accelerated process for standards submission and consideration.
Regarding the actual vote, the statement reads:
"In the committee there is a clear majority that is opposed to making OOXML into a ISO/IEC standard. Standard Norge must however in its overall assessment also consider the result of the formal hearing, and thus there is a majority of yes votes. There was greater number of end-users of document standard formats among those in favor, than those who were opposed to the standard.
ISO spokesperson Roger Frost on Tuesday said the ISO will have a response to the issue later.
The International Organization for Standardization is expected to announce the results of an Open XML vote on Wednesday.
Representatives from 87 countries had until Saturday night to submit their votes on whether the Office Open XML (OOXML) file formats should be certified as ISO standards.
The final tally was expected to be communicated on Monday. But a spokesperson for the ISO told Reuters that it will issue a press release on Wednesday to notify member countries first.
Tallies of official and unofficial sources by two Web sites--one run by standards expert and ODF advocate Andrew Updegrove and the other by open-source advocates at Open Malaysia--indicated that Open XML is expected to be approved.
Microsoft on Monday issued a statement saying that it will not comment on the matter until Wednesday "out of respect for the standards process."
