open source

Trawling for ideas in Open Cirrus — HP’s stab at a free cloud

HP plans to release its own open source cloud computing platform, according to the Director of HP’s Service Automation and Integration Labs, Chris Whitney. The Open Cirrus project, which HP Labs sponsors along with Yahoo! and Intel, was designed to put together “the ultimate stack of software people can use to build a cloud,” he said.

The project is a collaboration between the IT giants to pool computing resources at different sites into a “testbed” cloud and open it to researchers. Whitney said that Open Cirrus has about 300 researchers onboard since announcing partnerships with far-flung computer science labs in Russia, Malaysia and South Korea. He said HP has contributed about 10,000 computing nodes(virtual servers), Yahoo! about 3-4,000, and the other partners are kicking in at least 1,000 nodes each.

“We’re definitely envisioning an open-source software stack under the GPL” he said, similar to the eponymous LAMP software stack. He said they are experimenting with existing free cloud technologies (like EUCALYPTUS) and also gathering data from HP Labs’ own hardware installation, which runs on commodity HP data center servers.

Whitney said his team is also experimenting on a low level with using optical fiber communications on server backplanes and heating and cooling techniques in HP’s installation.

At a higher level, researchers are focusing on different applications for Hadoop, like data mining applications and “wide-area Hadoop” — data processing over distant geographical locations. There is a short list of current projects, but more are expected from the new partner sites.

Several open source cloud projects exist already, like Spanish Abiquo and UC-Berkeley’s EUCALYPTUS (released on Ubuntu 9.10), Canadian Enomaly, the Globus Nimbus project and others. Cloud leaders like Amazon and Rackspace run their clouds on open source technology but do not release their technology publicly. IBM is facilitating an EU-funded project called RESERVOIR, but it’s goals appear to be stretgic rather than practical.

HP’s entry, when and if it arrives, will mark the first open source cloud platform released by a major commercial vendor; certainly something to watch.

Rich Wolski on the difference between data centers and clouds

I’m working up an article about EUCALYPTUS and Eucalyptus Systems, cf. my earlier post on’t.

Leaving aside the giggles over nomenclature, I had quite a nice talk with Dr. Rich Wolski, the lead scientist on the original open source project (also with CEO Woody Rollins and the VP of marketing).

Anyway, Wolski had an interesting and quite succinct definition of the differences between a data center employing virtualization in its currently accepted form and a cloud infrastructure, because, on paper, the two share enough common elements that lots of people (and marketers) are happy to fuzz the two together.

I didn’t think that was quite right, and neither does the Good Doctor, (ha! like I’m the expert over here). Otherwise, why get excited over cloud? If that were true, then it’s just re-packaged old news, and nobody needs to do anything but change the badges and maybe dice up the trim package, if I may borrow from the big book of automobile industry metaphors. But cloud is fundamentally something different, and new, and it’s worth knowing why.

He says it’s down to access and the control structure. The major difference between a data center and a cloud is access- a cloud is set up so anyone can drop a penny in the slot and start up a server or six- in a data center, you ask, and someone does it for you, then hands over the steering wheel.

He said something like, pardon the paraphrasing, “in a data center, virtualization is the grease that lubricates resource management” for the admins; it allows the guy in charge to move his resources around- “it’s a reconfiguration mechanism,” but “in a cloud, [virtualization] is a fence.” it separates and protects resources and lets everyone have their own private playground without knocking over the other kids’ toys.

A subtle difference? Wolski says it’s down to a bedrock set of premises and assumptions that drove the development of the cloud model.

“We tried to look at the cloud paradigm from an analytical perspective,” he explains, and “cloud is an ecommerce model-it’s a transactional model [in a] distributed system”.

Did that sink in? At its most basic, cloud is not about computers. It’s about sales. Start with the premise that you have a product (server instances/CPU time/bit buckets), you want to sell them to any and all comers over the internet (ecommerce) and you want to do a lot of it. What you get is “cloud computing”, and logically, it’s no surprise that Amazon pioneered it commercially. They didn’t assume they needed a resource farm and a way to sell access to it- they assumed they had servers and needed to sell those instead.

So that’s interesting to me. Cloud is not a utopian access opium dream- it’s a logical outgrowth of commodity commerce.

UPDATE:Story’s done, look for it soon. One more little nugget from Wolski on defining cloud computing: When he and his crew started thinking about the project in 2007, they found it was “utterly impossible to get a consensus” on what cloud was, so they “decided to sidestep the debate by picking the thing that was demonstrably was a cloud — the one thing no one could say was NOT a cloud.” Their answer? Amazon Web Services.

So there you have it - not sure what an elephant is? Look around; you can’t miss one if it’s in the room with you.

EUCALYPTUS sprouts business shoots

In what is surely a contender for the most-complicated backronym of the ages, the Elastic Utility Computing Architecture for Linking Your Programs to Useful Systems, first developed at Middleware and Applications Yielding Heterogenous Environments for Metacomputing, UC- Santa Barbara (MAYHEM in case you weren’t seeing the silly nomenclature trend) has jumped the public lagoon for the open waters of commerce:

Eucalyptus Systems, formed out of the team that developed EUCALYPTUS has gone commercial to sell help with their open-source software to cloud-minded types.

EUCALYPTUS (fingers…getting…so… tired) is a set of open-source software tools that allow users to interact with and deploy AWS-type clouds. You can use the software to administer your EC2 images or create your own private data cloud along the same model. Pretty neat, and free, but what happens when AWS decides to re-tool their APIs? Questions soon to be answered.

read the press release here.

watch this page for updates.