• On TV.com: Dollhouse CANCELED, What Went Wrong?
August 7, 2008 3:43 AM PDT

Scaling Amazon EC2 to 512 active nodes

by Dave Rosenberg
  • Font size
  • Print
  • 2 comments

Among all the cloud-computing hype, one thing that hasn't been evident is just how far you can scale across a provider. I haven't seen any other vendor come near Amazon.com's ability to reach the massive scale that the cloud itself connotes.

Max Gorbunov from Grid Dynamics ran a 512-node Monte Carlo simulation to find out how well Amazon EC2--short for Elastic Compute Cloud--would perform. He used GridGain, a Java-based open-source grid computing infrastructure for the test.

All in all, this test clearly shows that you can utilize Amazon's massive infrastructure for high-end processing with an acceptable performance hit. And while I am sure I am oversimplifying the difficulty associated with getting this all set up, based on the development notes it seems like it was fairly easy (at least for Max.)

GridGain deployment on EC2

GridGain deployment on EC2

(Credit: Grid Dynamics)

The test consisted of a custom setup based on open-source components including GridGain and Open MQ running on the default EC2 Fedora Core 8 distribution and using a custom test harness developed for this project.

The performance degradation of 3 seconds (about 20 percent) should be considered minor given roughly 250-fold increase in scale. The curve rises two times: in the ranges 2-8 and 256-512, while 8-256 remains almost flat.

While I don't have the math available to me about what this cost to run (I would guess well less than $5,000), the way you might have done something like this in the past would have involved expensive software like DataSynapse or very technical open-source tools like the Globus Toolkit.

It's a new world out there.

Thanks to Eugene at TSS for the pointer.

Dave Rosenberg dishes up "Software, Interrupted" with nearly 15 years of technology and marketing experience that spans from Bell Labs to multiple start-up IPOs to open-source enterprise software companies. He is co-founder of MuleSource and currently serves as the general manager of Hardy Way. He is a member of the CNET Blog Network and is not an employee of CNET. Disclosure. You can contact Dave via e-mail at softwareinterrupted@gmail.com or follow him on Twitter @daveofdoom.
Recent posts from Software, Interrupted
Trend watch 2010: Mobile movies
Survey: IT spending to recover in 2010
Nintendo launches paid video content for Wii
Analyst: Money transfer soon to be No. 1 phone app
Apple's App Store review irking developers
Moving to the virtual layer (and taking advantage of the cloud)
Why Windows Mobile and Palm will continue to fail
Is Ohai the next big thing in social games?
Add a Comment (Log in or register)
by alexisbellido August 7, 2008 7:11 AM PDT
Interesting news. It seems cloud computing will be taking the world by storm in the next years.

I'm just waiting for persistent storage to be available in EC2 to move a few sites for production in the cloud.
Reply to this comment
by benjaminstraight August 8, 2008 3:19 AM PDT
Good article.
Reply to this comment
advertisement

S.F. hacker space: Heaven for the DIY set?

The Noisebridge hacker space offers sewing and Mandarin classes, soldering workshops, Internet-controlled front door access, and a server room with no door.
• Photos: Circuits, code, community

The browser battles go on and on

roundup From Firefox to IE and from Chrome to Opera and Safari, there's no sitting still for browser makers looking to keep their products fresh and competitive.

advertisement

About Software, Interrupted

In "Software, Interrupted," Dave Rosenberg discusses disruption in the software market, as well as the products and services that keep business technology norms in perpetual flux.

With nearly 15 years of technology and marketing experience spanning from Bell Labs to multiple start-up IPOs, Dave co-founded open-source software company MuleSource and now serves as general manager of Hardy Way. He also happens to be a U.S. patent holder and a workaholic. Technology is his best friend and mortal enemy.

Add this feed to your online news reader

Software, Interrupted topics

advertisement
advertisement

Inside CNET News

Scroll Left Scroll Right