Editors' note: Correction, March 3, 12:46 p.m. PST: This post, which originally carried the headline "White House ditches YouTube after privacy complaints," significantly misconstrued the White House's policy on and use of YouTube. In the interests of disclosure and transparency, we are leaving the contents as originally posted, with two subsequent update notes and with the exception of the headline change. See also our follow-up story, "No, the White House hasn't ditched YouTube."
* * * * * * * * * * * * * Original story follows * * * * * * * * * * * * *
Updated at 5:50 p.m. PST March 2: The New York Times is reporting that the White House has denied any change in online video policy. While the White House spokesperson admitted to using an in-house flash based solution for the latest of the president's weekly video messages, he said the White House is just "experimenting" with different solutions.
Updated at 2:59 a.m. PST March 3: Late Monday, Google posted on its Public Policy Blog a rebuttal to this report: "White House videos on YouTube."
Responding to complaints by privacy activists, the White House has quietly abandoned YouTube as the provider of the embedded videos on the president's official home page.
With the release of the latest weekly video address, the White House has shifted to a Flash-based video solution using Akamai's content delivery network.
The White House's decision to move away from the Google-owned video-sharing site will likely be met with praise by privacy activists and could mark the beginning of a real backlash in response to Google's insatiable thirst for detailed data on the browsing habits of Web surfers.
Ironically, the decision by the White House comes days after YouTube began to roll out new policies to better protect the privacy of visitors who view videos embedded into federal government Web sites. The move by YouTube may prove to be too little, too late.
This is the new embedded video tool used by the White House.
(Credit: Whitehouse.gov)The White House's decision to embed YouTube videos in the president's official home page drew instant criticism from privacy activists. In addition to several critical posts on my blog, by the Electronic Frontier Foundation (here and here), the Center for Democracy and Technology and the Center for Digital Democracy blasted the choice of video providers.
The focus of the criticism was on the use of long-term tracking cookies by the Google-owned video-sharing site. When the new White House site first went live in January, every visitor to the president's blog would be issued a tracking cookie, even those who did not click the "play" button to watch the video.
The White House acted quickly, and soon deployed a technical fix to the cookie issue, which protected Web surfers who did not click the play button. However, the tens of thousand of people who clicked play were still issued a cookie, and thus tracked by YouTube.
In an unannounced change over the weekend, the White House appears to have solved the remaining cookie privacy issue for those Web site visitors who wish to watch the president's weekly video message.
Out with YouTube, in with Akamai
As of Saturday, the White House seems to have ditched YouTube as its video provider. Visitors to the White House blog can now click play to view a Flash-based video that loads directly from the White House's own Web servers. This solution, which appears to use Akamai's content delivery network, does not make use of tracking cookies.
The president's tech team seems to have finally hit on an optimal solution--one which protects the privacy of the visitors to the White House site, while still permitting the president to spread his message.
The White House is still posting copies of the videos to its official YouTube channel. However, the president no longer provides free advertising to YouTube by embedding those videos on a taxpayer-funded site.
Furthermore, the White House has copied one of the coolest of YouTube's social features: the ability for users to easily share and embed videos on their own sites. Each of the White House-hosted videos includes an "embed" link under it that can be copied and pasted onto any other Web site or blog.
It is unclear whether this switch away from YouTube marks a permanent shift in policy for the White House, or whether the Oval Office geek squad is merely testing an alternate video provider. While the latest video is served using Akamai's servers, the older videos remain as embedded YouTube files.
YouTube's new cookie rules
The timing of the White House's decision to switch to Akamai is rather strange, given the recent moves by YouTube to offer a more privacy-preserving solution for videos used on federal government sites.
Within the last couple weeks, YouTube has silently rolled out its own updates in response to the cookie-related criticism. People wishing to embed a YouTube video can now select a delayed cookies option when copying the embed URL.
This is the new delayed cookies option for YouTube embeds.
(Credit: Screenshot of YouTube)That choice will cause the embedded videos to be served from an alternate domain, www.youtube-nocookie.com, which registrar records reveal was first registered on January 23 2009, just one day after this blog first mentioned the White House/YouTube cookie issue.
New documentation on the YouTube site reveals:
Enabling delayed cookies means that the YouTube video player will not set any non-session cookies on the computer of a visitor (viewing the page on which the YouTube video is embedded). The YouTube video player may set non-session cookies on the visitor's computer once the visitor clicks on the YouTube video player.
This option is rather similar (yet still inferior) to the technical fix that was previously used (and since disabled) by the White House, as well as the open source MyTube tool developed by the Electronic Frontier Foundation.
A prominent privacy policy
In another new move by YouTube, the site now appears to be directly embedding a link to its privacy policy in all videos that are played from government sites.
This is the new privacy policy link in .gov-hosted YouTube videos.
(Credit: Whitehouse.gov)When those same videos are viewed at YouTube.com, or when embedded in a blog or other non-.gov site, the clickable link to the privacy policy is gone.
Webmasters for various state agencies seemed to notice the new policy last week and initially complained to YouTube, thinking that the new youtube-nocookie.com was a phishing site.
A representative from YouTube told the Webmasters:
The privacy policy link you see on your embed player is in response to federal regulations regarding privacy on embed players. We're working to remove it from state and local .gov sites as soon as possible.
Still not perfect
While the decision by the White House to ditch YouTube is a good one, unresolved issues remain.
First, as previously noted by the Electronic Frontier Foundation, the White House Web site makes use of an "invisible pixel" style Web bug/tracker on every page on the site, hosted by WebTrends.com.
Ideally, the White House should take its Web analytics technology in-house and abandon the use of this third party tracking technology. Otherwise, at the very least, the White House privacy policy should be updated to note the tracking cookies used by WebTrends.
Second, the White House still has not published the waivers it issued to YouTube (and potentially other third parties), which permitted the sites to use long-term tracking cookies. The Electronic Frontier Foundation has repeatedly asked for these documents-- requests that the White House has ignored.
Given the president's much-publicized commitment to transparency, it is time that the White House publishes these documents.
Third, in its recent move to include privacy policy links in videos embedded at .gov Web sites, YouTube has clearly demonstrated that it has the ability to modify the services it provides depending on the referrer information associated with incoming requests. YouTube should build on this and adopt a policy of not logging any data associated with .gov-referred requests.
That is, the site would be free to keep logs on the videos viewed by visitors to its own site as well as those embedded on blogs, but it would opt to immediately forget all identifying information associated with requests from government sites.
While the White House seems to understand the cookie privacy issue, it is unlikely that members of the House and Senate are equally as tech savvy. After all, some of them can barely figure out Twitter.
YouTube videos are heavily used on the Web sites of those in the House and Senate. YouTube should adopt sane logging policies for visitors who view these videos, so that we don't have to wait for the House and Senate to fix the problem themselves.
YouTube did not return a request for comment, while a representative for the White House Web team declined to speak on the record.
(Credit:
Recovery.gov)
Update: As of 8 a.m. PST, within three hours of this story first going live, it appears that President Obama's Web team has (silently) pulled the robots.txt file from the Recovery.gov Web site. The site is now open to Web crawlers of all kinds.
The Obama administration has apparently opted to forbid Google and other search engines from indexing any content on the newly launched Recovery.gov.
Is this even more evidence that the administration's much-publicized commitment to transparency is simply hype?
Recovery.gov, which went live Tuesday, is set to act as a central clearinghouse for information related to the newly signed American Recovery and Reinvestment Act. The legislation is designed to stimulate the flagging U.S. economy.
In a video message, available on YouTube and embedded into the new site, President Obama states that the "size and scale of (the stimulus) plan demands unprecedented efforts to root out waste, inefficiency, and unnecessary spending. Recovery.gov will be the online portal for these efforts." He adds that the new site will be used to publish information on how the stimulus funds will be spent in a "timely, targeted, and transparent manner."
Although the site is advertised as proof of the president's commitment to transparency, its technical design seems to betray that spirit. Most importantly, the site currently blocks all requests by search engines, which would ordinarily download and index each page to make the information more accessible to the Web-searching public.
The site's robots.txt file has just a few lines of text:
# Deny all search bots, web spiders
User-agent: *
Disallow: /
Although the White House Web team did not immediately respond to a request for comment, the single-line comment at the top of the file indicates that the blocking of search engines is no accident but rather a statement of policy.
Many sites use a robots.txt file to communicate, in machine-readable terms, the Web pages that they do and don't wish to be indexed by search engines. While the files don't carry much, if any, legal weight, most search engines act as good Internet citizens and honor the requests.
Luckily for the millions of Americans who might wish to find out how their money is going to be spent, it seems that Google has opted to ignore the administration's restrictive robots.txt on the stimulus-related site. It is unclear if this is due to an error or a manual override by someone at Google, but a quick search turns up more than 60 Web pages on Recovery.gov that have been indexed by the search engine's Web crawlers in just the past three days.
Also, the stimulus bill requires that the site be run by the new Recovery Accountability and Transparency Board, but it seems to currently be under the control of the White House Web team--the same folks who revamped Whitehouse.gov and whose use of the robots.txt search engine-blocking code was expanded after the site initially was praised by bloggers for its openness.
It is this blogger's hope that with a bit of gentle prodding by members of the pro-transparency community, Recovery.gov's administrators will correct the "unintentional oversight" that was made in launching the site with such an restrictive robots.txt file.
The White House has silently tripled the number of Web pages that it forbids Google and other search engines from accessing. Is this a bad omen or much ado about nothing?
Within hours of Barack Obama being sworn in as president, bloggers and tech journalists began to closely examine the new White House Web site for hidden indicators as to how he would shape future tech policy.
While I focused my efforts on the White House privacy policy, others looked to the new administration's robots.txt file, which lays out boundaries that search engines like Google should follow when scraping the site.
When the new Obama geek team posted its sparse robots.txt to the Web, tech pundits soon hailed it as a sign of the President's commitment to openness, transparency, and proof that someone tech-savvy was finally running the show.
Blogger Jason Kottke hailed the move, writing that it was "a small and nerdy measure of the huge change in the executive branch of the U.S. government today." Another blogger, Ben Orenstein, compared the new Obama robots.txt file to the 2,400-line file used by the Bush White House, "I think you've got a lovely little microcosm; one that points to a hopeful and open future."
The big fuss?
These digerati were excited by the fact that the new White House robots.txt file contained just two lines:
User-agent: *
Disallow: /includes/
Fast-forward one week, and the White House has silently started to expand its use of the robots.txt search engine-blocking mechanism. As of Friday morning, the file now contains the following text:
User-agent: *
Disallow: /includes/
Disallow: /search/
Disallow: /omb/search/
While it would be accurate to state that the White House has in one day tripled the number of sites it excludes from Google crawling, it is also important to note that this is not a big deal--in fact, it doesn't matter at all.
For the most part, the Bush White House's use of robots.txt was totally legitimate, something that Kevin Fox, an engineer at Friendfeed told the folks at Google Blogoscoped:
This is a bit silly. The old robots.txt excludes internal search result pages and redundant text versions of HTML pages. This is exactly what robots.txt is for. Google's Webmaster Guidelines state "Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don't add much value for users coming from search engines."
It's understandable that the robots.txt of an 8-year-old site is longer than that of a 1-day-old site, and it's not as if '/secrets/top' or '/katrina/response/' were put in the robots file.
Fun as it may be, this is a nonstory.
Those bloggers drunk on hope who desperately wanted to see proof of Obama's commitment to his campaign promises of transparency and Google Government now find themselves with a difficult choice: they can either accept and acknowledge that robots.txt files are not a set of digital tea leaves through which you can read the new administration, or, if robots.txt does carry weight, they can try to come up with a way of explaining a 200 percent increase in the number of directories blocked by Obama's Web team as anything but Cheney-esque secrecy.
Simply put, the robots.txt file was created and managed by engineers, not lawyers or policy makers. It is not the place to judge the president on tech policy issues.
The president's tech policy should instead be judged on real issues: how many former RIAA and MPAA lawyers will be given positions of power in the administration, who ends up working at the FTC and FCC, and who will be named the new cybersecurity czar.
As for the president's commitment to transparency, he has already violated his pledge to post all nonemergency bills on the Whitehouse.gov Web site for five days before signing them. The text of the Lilly Ledbetter Fair Pay Act of 2009, which was signed into law yesterday, was certainly not posted to Whitehouse.gov for anywhere near five days.
Obama's broken commitment to transparency remains advertised on the White House blog:
One significant addition to WhiteHouse.gov reflects a campaign promise from the president: we will publish all nonemergency legislation to the Web site for five days, and allow the public to review and comment before the president signs it.
It is by looking to these kinds of concrete issues by which we can judge the president, not robots.txt
Someone at the White House appears to be listening to those of us in the privacy community.
For the third time in just six days, the Obama administration has modified the White House Web site privacy policy in response to criticism from the blogosphere.
When the site launched on January 20, it exempted YouTube from federal anticookie tracking rules that would have otherwise cast a legal shadow over the use of embedded videos on the White House blog.
Reacting to criticism from the blogosphere, the White House first modified its Web site on Friday to limit the cookie exposure to only those users who clicked on videos. Then, on Sunday, the White House again tinkered with its privacy policy to scrub YouTube's name from the cookie exemption.
The original YouTube-specific exemption stated:
For videos that are visible on WhiteHouse.gov, a "persistent cookie" is set by third-party providers when you click to play the video.
This persistent cookie is used by YouTube to help maintain the integrity of video statistics. A waiver has been issued by the White House Counsel's office to allow for the use of this persistent cookie.
However, by Sunday evening, the exemption had been edited to remove all mention of YouTube:
For videos that are visible on WhiteHouse.gov, a "persistent cookie" is set by third-party providers when you click to play the video.
This persistent cookie is used by some third-party providers to help maintain the integrity of video statistics. A waiver has been issued by the White House Counsel's office to allow for the use of this persistent cookie.
The decision by the White House to revisit the cookie exemption does not come as a complete shock. The YouTube rule had in just a few short days generated both bad press and direct criticism from several public-interest groups.
It should be noted that this change is, for the most part, cosmetic. YouTube continues to be the only company whose video content is embedded within the White House Web site. Furthermore, the Google-owned video-sharing site is the only one that has received both official legal clearance from the White House Counsel and direct assistance by the White House tech staff (who embed the YouTube content) in planting tracking cookies within the Web browsers of millions of Americans.
Google CEO Eric Schmidt, who has advised President Obama and who personally donated $25,000 to the president's inauguration celebration (out of a total of $150,000 by six Google executives) must be rather pleased.
Still no transparency
In spite of Obama's much-publicized commitment to transparency, the White House has yet to actually provide a copy of the waiver (something this blogger has requested from White House officials informally, as well as via the Freedom of Information Act).
The text of the original privacy policy implied that a specific waiver had been issued for the cookies forced upon end users who intentionally viewed YouTube videos embedded within the White House Web site. The text now implies a far broader waiver for multiple video-sharing Web sites. However, it remains unclear if a new waiver has been issued, or if the old waiver was broad enough to cover multiple sites.
When I first wrote about the privacy policy text last week, I criticized the White House for providing YouTube with a specific exemption. At the time, I noted that no other company had received such special treatment.
The motivation of my criticism was to try to shame the White House staff into doing away with the exemption--as cookies are in no way required in order to serve online video. Instead of recognizing the need to protect consumer privacy, White House officials reacted by expanding the exemption to other companies.
In many ways, the current policy is actually worse than before: non-tech-savvy consumers now have no idea how many companies might be forcing their Web browser to accept tracking cookies. At least up until last week, visitors could take some comfort in the knowledge that only one company might be invading their privacy when they visited the White House Web site (and then only by a firm that had pledged to "do no evil"). Now, at least according to the White House's wide exemption, there could be many.
Last week, I said we should be reasonable and give the White House Web team a bit of time--after all, it is in a brand-new office, managing a new computer network, and scrambling to meet the demands of a very busy boss. However, if the team has had enough time to tinker with the privacy policy at least three times in the past six days, then it has more than enough time to post a copy of the waiver.
- prev
- 1
- next
