The Pervasive Data Center

Delicious, the social-bookmarking service owned by Yahoo, has unveiled home page changes that are intended to do a better job of showcasing links that are currently popular. Although Delicious isn't sharing the exact details of its algorithm, it apparently includes using the number of Twitter messages related to a given item.

Writing on the Delicious blog, Vik Singh, an architect at Yahoo, writes that "For this new Fresh homepage, our system displays recently bookmarked links and tweeted messages focused mostly on technology, web, politics, and media. Underneath the hood, Fresh factors several features into the ranking like related bookmark and tweet counts, "eats our own dogfood"  by leveraging BOSS to filter for high quality results, as well as stitches tweets to related articles even if the tweets do not provide matching URLs."

The issue that Delicious is trying to address here is that the existing "Popular Bookmarks" tab (which will continue to be available) tends to point to what Singh describes as "authoritative resources rather than fresh news." This is because, although Delicious is often described as a social-bookmarking service, in fact, many use it primarily as a way to store bookmarks online solely for their own purposes. And, in fact, Delicious even introduced private tags in 2007 that made it possible to save bookmarks without sharing.

Not everyone is happy about the change. Delicious founder Joshua Schacter, who left Yahoo last year (and is now at Google) writes on Twitter that "I can't BELIEVE delicious did integration with other social networks before finishing with its own." He adds that "i had always wanted delicious to show notes from your social network on the links that you bookmark."

Social bookmarking has often seemed like rather the red-haired stepchild of social media. Its evolution in general has been slow and there's long been a tension between bookmarking to share and bookmarking to store.

In general, social bookmarking services have also failed to surface the data that they have stored in ways that allow for useful and serendipitous exploration. This latest announcement tries to do something about that by making use of data from Twitter, a service that's all about the now.

LAS VEGAS--The consumerization of the Web will be as disruptive to distributed computing as distributed computing was to the mainframe. That was the central theme of Citrix Systems CEO Mark Templeton's keynote speech at this week's Synergy 2009 conference.

Mark Templeton, CEO, Citrix Systems

(Credit: Citrix)

This is an oversimplification, of course. Over the years, companies have run their business software in many different ways--not all of which are easily categorized as either mainframe-like or PC-like. One whole era of computing architectures during roughly the 1980s commonly went by the term "client-server." However, if we think of how distributed computing in the enterprise has evolved, this broad-brush statement makes a lot of sense.

That's because the enterprise PC isn't really a personal computer any longer. The administrative and security requirements around desktop and notebook devices running an increasingly complex stew of locally installed software have seen to that. In many enterprises, they're stringently locked down as a way to protect their often fragile software payloads from corruption.

This is a drum that virtualization and cloud-computing specialist Citrix has been pounding for quite a while. Writing after Citrix iForum (Synergy's predecessor) in November 2007, I noted:

We've seen and heard a lot of praise for the democratic impulse associated with this particular phase of computing that often goes by the Web 2.0 moniker. Anyone can post. Anyone can publish. Anyone can photograph. Your vote matters in social media.

And alternative ways of accessing and running applications have indeed made it easier to do things outside of a strict IT framework. In his closing iForum keynote, Citrix CEO Mark Templeton used the phrase "making the personal computer personal again" for this idea.

It's perhaps not too surprising that the proffered solution to this problem is a variety of technologies that Citrix collectively describes as application delivery. The framework to think about it is something like a satellite TV system. A controller, a delivery network, and a receiver transmit and receive the bits; they do so independently of the actual end-point device (i.e. the TV) and the content, so long as those adhere to certain interface standards.

One could use such an architecture to deliver enterprise applications to a truly personal notebook, an employee's personal system rather than an IT asset. Although still relatively uncommon in an enterprise context when it comes to PCs, it's a fairly common model with smartphones, though we're starting to see the beginnings of such an approach in the PC space too.

What this means specifically in a Citrix environment is that Citrix Delivery Center "head-end controllers" such as XenApp and XenDesktop advertise services--that is, applications that are available for users to run. New services or service updates are then loaded or streamed to a client.

One of Tuesday's major announcements was Citrix Receiver, which the company describes as "the first universal client for IT service delivery":

Under the hood, Citrix Receiver is a lightweight universal software client with an extensible browser-like "plug-in" architecture. Receiver comes standard with a variety of optional plug-ins that communicate with head-end infrastructure in the Citrix Delivery Center product family such as XenApp, XenDesktop, Citrix Access Gateway, and Branch Repeater.

These plug-ins support functionality such as online and offline app usage, virtual-desktop delivery, secure access control, password management, app acceleration, multimedia acceleration, service-level monitoring, and voice communications. This model enables IT to effectively operate as a service provider to their own employees, proactively and transparently monitoring end-user experience from a central location.

Receiver is available for Windows, Macs, and iPhones. Citrix also plans to support Windows Mobile and Symbian operating systems. It's also working with Open Kernel Labs to support Android. In all cases, Receiver is free.

In general, as with XenServer, Citrix' strategy is to make its money from the management and delivery software infrastructure rather than all of the base-level components.

The final announcement of the day was Dazzle. It's built on top of Receiver and accesses the same head-end services. It is, in a sense, Citrix application delivery meets Web 2.0.

I mean that in a somewhat metaphorical sense. But Dazzle is a self-service application store for employees that very deliberately and consciously mimics the conventions and approach of something like the iTunes Store. Web 2.0 and cloud-computing attributes, like self-service, device independence, and remote access are what help so many consumer applications make traditional enterprise apps look a bit shopworn by comparison.

And that's what Mark Templeton was talking about when he said the enterprise application delivery model is being disrupted by the consumer Web.

This is a big week for Intel processor-based server announcements. Intel is rolling out its new "Nehalem" Xeon 5500 processor for dual-socket servers, far and away the biggest chunk of the server market by volume.

As Brooke Crothers notes on this CNET Blog Network post, "Nehalem offers some important firsts for Intel, including an integrated memory controller for better performance, hyper-threading for up to 16 virtual cores (which improves multitasking), and Turbo Boost Technology, which dynamically increases the processor's frequency (speed), as needed."

Just about any new Intel Xeon processor is paired with a spate of server announcements--after all, it's the servers that most end users buy not the chips. However, because Nehalem gives the heart of the server market such a nice performance boost, this launch is bigger than most. All the big system suppliers are making significant product introductions.

Take HP, for example. Paul Gottsegan, who leads marketing for HP's Industry Standard Servers (ISS) group, described the launch of their new ProLiant (x86) servers to me as "the biggest announcement in 20 years for ISS."

What really struck me about the HP launch though wasn't its scope--broad though it was.

Rather, it was that HP didn't take it as an opportunity to pile on an endless litany of speeds and feeds. Sure, it provided me with specifications but in the vein of supporting data rather than the core of the announcement. HP similarly focused primarily on higher-level operational and business value messages at its Technology Solutions Group (TSG) industry analyst event in Boston last week. (TSG is the business unit within HP that includes servers, software, and service.)

Unsurprisingly, in the current climate, a lot of that value message is around doing more without spending more.

The HP ProLiant G6 line's advances in energy efficiency, virtualization and automation, make it ideal for all customers. These innovations are combined with comprehensive financing programs and service offerings to redefine server economics. The new HP ProLiant G6 servers are available in 11 standards-based tower, rack and blade platforms. This represents the largest HP ProLiant rollout in company history.

"Now more than ever, customers want the best possible return on their server investments," said Christine Reischl, senior vice president and general manager, Industry Standard Servers, HP. "Building on HP's long history of hardware and software development, G6 brings together the best HP innovations in energy efficiency, virtualization and services to enable our customers to do more with less."

Now, if you haven't been a longtime HP follower as I have, the fact that technology isn't front and center may seem unremarkable. Sure, IT vendors have a proclivity to getting lost down in the weeds. But the largest and most sophisticated of those vendors--of which HP is certainly one--do understand that customers buy outcomes rather than individual products.

However, HP as a whole has perhaps struggled more than most to build on, rather than lead with, a technology message. The company is, after all, in no small part an amalgam of very engineering-centric cultures--the old HP and Digital Equipment perhaps most of all. But also Compaq and Tandem in their own ways.

This is a solid server rollout. But it's also a clear indication of HP's evolution.

Last week, CNET News' Stephen Shankland related the story of how the Scion public-relations team added me as a contact in Flickr. As he noted, "I thought it might be a marketing move, given that the Detroit auto show was under way, and indeed, a little digging showed that to be the case."

I likewise contacted the Flickr user, and here's the reply that I received (via the Yahoo.com address in their profile):

We saw you were on Flickr and, as a technology/online aficionado, we thought you might be interested in the new car. The new release series vehicle is loaded with tech gear and, yes, is debuting at the Detroit auto show this week.  Since your blog covers mostly tech but does the occasional stray (nice Lobster recipe!) we thought you'd be interested.

I'm not entirely clear through what avenue I came to their attention. The referenced blog in her reply would seem to be my personal blog and there is, indeed, a link to my Flickr account there.

This seems a good opportunity to ponder the appropriate uses of social media for commercial purposes.

This case certainly doesn't strike me as any great suborning of the system, a la the recent attempt by a Belkin employee to insert flattering reviews into Amazon.com via Amazon's own Mechanical Turk crowdsourcing service. Two very different things.

This is, if anything, pushing the boundaries of an ill-defined boundary between public and private, between commercial and noncommercial. And the reality is, that boundary never will get better defined, unless people push it from time to time.

In this case, I certainly had the option not to accept the contact request. It was an opportunity to see first-hand an attempt to innovate in public relations using social media. And I do have a reasonably high-profile online presence that, at the least, blurs the line between my public and private personas.

That said, I'm inclined to put this in the "OK this time, but don't make a practice of it" bucket. In part, it's a matter of numbers; I didn't mind this one contact, but I'd hate to be deluged with Flickr contact requests from all manner of people with nonobvious commercial purposes.

The PR agency wasn't really hiding its identity, but it wasn't advertising it, either. For example, they used a Yahoo.com e-mail address. We're also not talking about a highly focused PR contact by way of Flickr; I may write about technology, but I'm not sure that I've ever written about cars specifically.

Using Flickr in this way also seems to go a step too far into the personal space. My public and personal personas may well be blurred together, but my Flickr account is pretty obviously not related to the technology writing I do. It feels a little bit like someone calling my home number on a business matter.

Again, I don't fault Scion's PR team for trying something new and different. In fact, I applaud it. But as companies innovate with social media, they also have to develop an exquisitely fine ear for that which is ringing true to their audience and that which strikes a false chord.

What do you think?

In his post, "What Tim O'Reilly gets wrong about the cloud," Nick Carr takes Tim to task for describing Google as an example of a business that has grown to dominance because of network effects. The basic idea behind network effects is that something gets more valuable as more people use it. The canonical example is the telephone system. With one telephone it would be pretty uninteresting. With limited penetration, only mildly interesting. With near-universal connectivity, extremely powerful. There's a lot of debate about the details, but few dispute the basic concept.

But Nick argues that network effects don't underpin Google's success. Google determines the best search results using algorithms, not the "wisdom of the crowd." If I were the only person using Google, it might be hard for Google to continue making hardware and software investments. But, money aside, it doesn't inherently depend on having lots of users to deliver quality results. By contrast, social networks or a many-to-many selling site like eBay (especially in its earlier days) are network effect businesses. (In fairness, the myriad creators of Web content make Google's PageRank possible but I see this as a weak form of network effect compared to the other examples.)

However, network effects aren't the only reason why a given industry may end up with just a few--or even one--large players. Nick lists a few that may well be relevant to Google and other "cloud computing" suppliers:

1. Capital intensity. Building a large utility computing system requires lots of capital, which itself presents a big barrier to entry.

2. Scale advantages. As O'Reilly himself notes, big players reap important scale economies in equipment, labor, real estate, electricity, and other inputs.

3. Diversity factor. One of the big advantages that accrue to utilities is their ability to make demand flatter and more predictable (by serving a diverse group of customers with varying demand patterns), which in turn allows them to use their capital more efficiently. As your customer base expands, so does your diversity factor and hence your efficiency advantage and your ability to undercut your less-efficient competitors' prices.

4. Expertise advantages. Brilliant computer scientists and engineers are scarce.

5. Brand and marketing advantages. They still matter - a lot - and they probably matter most of all when it comes to the purchasing decisions of large, conservative companies.

6. Proprietary systems that create some form of lock-in. Don't assume that "open" systems are attractive to mainstream buyers simply because of their openness. In fact, proprietary systems often better fulfill buyer requirements, particularly in the early stages of a market's development. As IT analyst James Governor writes in a comment on Macleod's post, "customers always vote with their feet, and they tend vote for something somewhat proprietary - see Salesforce APEX and iPhone apps for example. Experience always comes before open. Even supposed open standards dorks these days are rushing headlong into the walled garden of gorgeousness we like to call Apple Computers."

The reason this question is of more than academic "nature of the firm"-type interest is that the question of scale in a world where more and more computing happens out in the network has implications that go beyond the cloud computing suppliers themselves. (For our purposes here, cloud computing refers to Internet-based computing generally, whether Software as a Service,SaaS, or some more direct use of computing resources over the network.)

A world with many different cloud computing suppliers, operating at different levels of abstraction--SaaS, Amazon Web Services-style virtual machines, Google Apps-style developer platform, and so forth--looks very different from the world hyperbolically described by Sun CTO Greg Papadopolous as having five "computers." (Computers in the sense of cloud computing providers with distributed worldwide datacenters.)

The computer industry in the first world would look much like today's. Independent Software Vendors (ISV) might deliver applications in the form of Web services rather than programs to be locally installed.  But they'd still be ISVs. And systems vendors would be selling more to those ISVs and other cloud computing suppliers than they would be the ultimate end users. But the sales model wouldn't be all that different from current enterprise sales.

In the second case, things would be much different, however. As I discussed in "The New Systems Companies," if computing were to become something largely consumed by a relative handful of mega-scale service providers, that would fundamentally alter the dynamic between system supplier and system customer that exists today. In the extreme case, the biggest service providers could become the new systems companies, as Google has already done to at least a partial degree.

Understanding to what degree size matters in a cloud computing world is therefore a very important question.

A bunch of us were debating over Twitter yesterday whether it's desirable to have separate personal and professional identities on the service. The consensus seemed to be: "it depends." It depends on your professional situation. It depends on how personal and workplace-safe you want your posts. And so forth.

I find this whole question of what I call "identity 2.0" fascinating. Increasingly, there's a blurring line between personal and professional identities--and even between multiple compartments within those buckets.

As Wendell comments in a post: "It's kinda like living in a small town again." There are a lot of analogs. Just as locality and small size break down barriers between public and private in a small town or village, so, too, do the Internet and the search engine.

This is a trend that we're all going to be wrestling with for years to come. Although things I've written back in my college days are readily available online, if you know where to look, it was mostly stuff written for newspapers or Usenet posts.

There are doubtless matters on which I've changed my thinking, but there is probably nothing that I'd find especially embarrassing. What I don't have online--because it didn't exist back then--is "off the record" commentary written purely for a circle of friends. (In Here Comes Everybody: The Power of Organizing Without Organizations, Clay Shirky describes how many blogs are clearly written for a close circle of friends, even though they can potentially be viewed by anyone.)

Wall Street may not be Main Street. Neither are Silicon Valley and its relatives (Research Triangle Park of North Carolina; Cambridge, Mass.; Austin, Texas, etc.). The general sort of "live and let live" attitude toward activity outside of the workplace that may predominate there--as well as among employees who are highly visible bloggers, pundits, and so forth--isn't really the norm.

Suggestions that we do something about the "ephemerality of the Web" would also, to a certain degree, exacerbate any issues. Old Web sites, comment threads, discussion boards, and so forth do tend to evaporate over time, providing a loose statute of limitations. The better we get at preserving the Web for the sake of history, the less likely that youthful indiscretions will vanish into the mists of time.

Of course, much of the Web's most vacuous inanity--think comments on Digg--is cloaked in effective anonymity. (By "effective," I mean that it can often be pierced by legal action, but is anonymous from the perspective of ordinary searches.) Transient anonymity has its own problems. However, a blogging pseudonym--perhaps known to friends--is doubtless a reasonable response in many circumstances.

I touched on some of these issues, as well as others related to data portability, previously here.

What do you think? Do you keep your personal and professional identities separate?

I spent the past couple of days attending Technology Review's EmTech08 conference at MIT. Lots of interesting speakers and ideas, some in areas of tech that I follow day-to-day (such as cloud computing) and others that I follow more in the vein of an interested observer (alternative fuels, open voting systems). In many respects, it's a refreshing change of pace from the events I commonly attend that tend to be more focused on today's immediate IT concerns.

EmTech08 gave me lots to mull--and I'll roll that mulling into more in-depth pieces down the road. For today, though, I'm just going to expand a bit on a few statements and thoughts I ran across in the course of the two days that particularly caught my attention.

The state of the market for tools in parallel computing is abysmal. (Marc Snir, University of Illinois)

There seems to be a default assumption around the IT industry today that, as processors evolve to more cores and more heterogeneous processing (and as computing architectures get more distributed), the software will evolve apace. Changes will be required, of course, but nothing to really worry about. I'm not so sure. Even if one discounts the most dramatic doomsayers, lots of researchers and IT executives see serious gaps in both tools and training to deal with highly threaded processing. Consider that several of the panelists in the Parallel Programming session spoke warmly of the potential for functional languages. Yet it's very early days for the likes of Haskell and Fortress and, in general, language development and adoption is a very long process.

People choose killer apps. (Craig Mundie, Microsoft)

Craig Mundie spoke at length about many of the characteristics that he saw such an application as having. He used terms like context- and location-aware, immersive, and personal. The general theme was "client + cloud," the idea being that this level of realism and immersion requires huge processing power and function at the client even if the data and orchestration takes place in the network someplace. He didn't really specify a specific next-generation application though. I'm not sure killer "app" is really the right term though. The original PC had a true killer app--the spreadsheet. But subsequent generations have really had interaction models--first the GUI and then the browser. The next generation will probably be something similar, exposing an even more varied set of applications but in richer ways. (CNET News' Dan Farber has an in-depth post on Mundie's keynote.)

Ephemerality of the Web is something that needs to be addressed. (Web 2.0/3.0 panel)

One of the ironies of the digitized world is that it's a potential enabler for an unprecedented level of preservation but, in practice, it often ends up opening the door to huge amounts of content vanishing in an instant. At the consumer level, photos offer an illustrative example. The combination of external hard drives and online services can far better protect digital photos from mishaps than is possible with negatives, slides, and prints. But, in practice, most consumers don't have good backup systems and can easily lose everything with the crash of a hard disk. Long-term preservation is an even bigger problem, both online and off. What happens as companies are purchased or go out of business to the content that they create or host?

Web technologies are the best platform for mobile development. (Kevin Lynch, Adobe Systems)

The iPhone hype (or, indeed, the babel that surrounds Apple in general) can be wearing. However, one thing that the iPhone has really accomplished is to crystallize the notion that the browser is a viable interface for mobile phones--at least high-end mobile phones. Kevin's contention that Adobe's AIR runtime is also necessarily part of the mix is more debatable, but the general concept that mobile applications will tend to center around the same mobile technologies that are used on "PCs" seems sound. (I tend to think that the mobile device will be a smartphone rather than a separate "Mobile Internet Device" (MID) but that's a separate debate that centers more around form factors and networks than programming models.)

Many energy solutions are not relevant at the scale that matters. (Vinod Khosla)

Finally, a sizable chunk of the conference was devoted to "green" and energy. Unsurprising given how it's a hot (if also overhyped) topic in IT and elsewhere. It's also an area that lends itself to transformative innovation--which fits well with the general focus of EmTech. Which is what venture capitalist Vinod Khosla, a co-founder of Sun Microsystems, is really talking to with that statement. It's not that incremental changes aren't desirable. They are. Indeed, a lot of power efficiency work in technologies such as microprocessors is a sort of whack-a-mole game of accumulating small wins. However, from a macro and policy perspective, big wins don't come from the niches. They come from making substantial impacts on substantial use cases.

Over five years ago, I wrote a research note titled "Latency Matters!" The impetus was the following observation:

What's the best way to estimate travel time? Would you rely on an estimate based solely on the number of lanes in the road and the sound of the engine? Nope. You need to know, at minimum, how far you have to travel, the condition of the road, and how fast you'll likely be able to go. Obvious, right?

You'd think so. But system and networking specs rate computer performance according to bandwidth and clock speed, the IT equivalents of just measuring the width of the road and the engine's revolutions per minute...

Latency is the time that elapses between a request for data and its delivery. It is the sum of the delays each component adds in processing a request. Since it applies to every byte or packet that travels through a system, latency is at least as important as bandwidth, a much-quoted spec whose importance is overrated. High bandwidth just means having a wide, smooth road instead of a bumpy country lane. Latency is the difference between driving it in an old pickup or a Formula One racer.

Some of the particulars discussed in that research note are less central to everyday IT concerns than they were at the time. For example, although as much engineering attention goes into designing high-end systems as ever, the details of memory architectures, internal processor interconnects, and the like have increasingly receded into the background as far as most IT generalists are concerned.

But, as Todd Hoff notes in "Latency is Everywhere and it Costs You Sales - How to Crush it," latency concerns are still very much with us. In fact, the nine sources of latency that he lists suggest that latency is actually a much thornier problem in a world where applications are broken into pieces and often distributed around the world.

This strikes me as a particularly important observation amid the cloud computing hoopla which tends to reduce relationships between software components and their associated data as taking place within some idealized network abstraction. That's not to say that such an abstraction isn't a useful concept. But it does tend to de-emphasize how parts interact in the real, physical world.

Consider, for example, the case of storage. Even a lot of staunch cloud computing advocates who liken using processor cycles out of some network grid to a computing version of the electric utility generally concede that storage is a trickier problem. Whereas computing is something you just consume, data has state. And if you lose that state, it's gone. That's a fundamentally more serious problem than losing access to a compute utility for a few minutes or even an hour. For this reason and others (regulatory compliance, etc.), my Illuminata colleague John Webster has written that "Internal storage clouds will become way more popular than external storage clouds."

OK, you say, so cloud computing (in the sense of external clouds out in the network somewhere) will be more popular for processing things than storing them. So what?

The so what (or one of them anyway) is latency. We tend to run applications close to the data they operate on for a reason. That's because application performance is often largely a function of how quickly it can read and write the data that it's working on. And data stored on a local hard disk can almost always be accessed faster than that same data sitting at the other end of a network pipe hundreds or even thousands of miles away.

Thus, if storage stays inside organizations, that implies that a lot of the processing of that data will as well. And the general trend towards more data-intensive modeling and mining only strengthens this relationship. Because latency matter more than ever in a world where the pipes are distributed networks.

Over at Plagiarism Today, Jonathan Bailey notes:

A recent post by photographer J.M. Goldstein raised a very interesting question about Flickr and its API, namely whether or not Flickr was policing its API well enough and doing an adequate job protecting the rights of photographers and artists that post to the service.

I would have thought the answer was obvious. No.

Or, perhaps more accurately, Flickr has apparently decided either deliberately or as a matter of generalized neglect that providing its users with more sophisticated and granular tools to protect their content isn't a priority.

While there is much that I like about Flickr, it's simply not the best service if you want to carefully control who accesses your photos and exactly how they can access them. SmugMug and PhotoShelter are two services that have put far more thought and effort into this aspect of their respective sites.

Speaking personally, I still use Flickr anyway. The price is right--$25 per year for a Pro account which gives me unlimited storage and uploads. While I would prefer to secure my photos a bit better, doing so isn't really all that important to me given that I don't sell them.

So, while the criticism seems valid enough, it's also part and parcel of Flickr's emphasis on sharing and community over tight user control of their creative product. When picking Flickr or any other photo site, it's important to understand not just its pricing scheme, reliability, and how well their user interface works but, as importantly, the underlying priorities that drive all sorts of design choices.