August 7, 2008 3:43 AM PDT

Scaling Amazon EC2 to 512 active nodes

Among all the cloud-computing hype, one thing that hasn't been evident is just how far you can scale across a provider. I haven't seen any other vendor come near Amazon.com's ability to reach the massive scale that the cloud itself connotes.

Max Gorbunov from Grid Dynamics ran a 512-node Monte Carlo simulation to find out how well Amazon EC2--short for Elastic Compute Cloud--would perform. He used GridGain, a Java-based open-source grid computing infrastructure for the test.

All in all, this test clearly shows that you can utilize Amazon's massive infrastructure for high-end processing with an acceptable performance hit. And while I am sure I am oversimplifying the difficulty associated with getting this all set up, based on the development notes it seems like it was fairly easy (at least for Max.)

GridGain deployment on EC2

GridGain deployment on EC2

(Credit: Grid Dynamics)

The test consisted of a custom setup based on open-source components including GridGain and Open MQ running on the default EC2 Fedora Core 8 distribution and using a custom test harness developed for this project.

The performance degradation of 3 seconds (about 20 percent) should be considered minor given roughly 250-fold increase in scale. The curve rises two times: in the ranges 2-8 and 256-512, while 8-256 remains almost flat.

While I don't have the math available to me about what this cost to run (I would guess well less than $5,000), the way you might have done something like this in the past would have involved expensive software like DataSynapse or very technical open-source tools like the Globus Toolkit.

It's a new world out there.

Thanks to Eugene at TSS for the pointer.

Recent posts from Negative Approach
The best-selling video games of 2008 (so far)
Ubuntu on the Thinkpad X300: Impressions and installation tips
Build your own Cloud with the Eucalyptus open source project
Free-to-play, ad-supported games the winners?
Why is Spotlight using 98% of my MacBook Air CPU?
Add a Comment (Log in or register) 2 comments (Page 1 of 1)
by alexisbellido August 7, 2008 7:11 AM PDT
Interesting news. It seems cloud computing will be taking the world by storm in the next years.

I'm just waiting for persistent storage to be available in EC2 to move a few sites for production in the cloud.
Reply to this comment
by benjaminstraight August 8, 2008 3:19 AM PDT
Good article.
Reply to this comment
Powered by Jive Software
advertisement
  • About Negative Approach

  • Dave Rosenberg is CEO and Co-founder of MuleSource, a venture-backed company that develops open source integration and infrastructure software. On the Negative Approach Blog, Dave discusses the dynamics of growing a startup company and how the software market is evolving against monolithic software corporations whose corporate hegemony stifle innovation and annoy developers worldwide. With experience at both large corporations and several startups, technology has long been his best friend and mortal enemy. He is a member of the CNET Blog Network and is not an employee of CNET. Disclosure.

Add this feed to your online news reader
Google
Yahoo
MSN
advertisement
On BNET: 21 ways to make yourself more likeable
Advanced
search
Advanced
search
Visit other CBS Interactive sites