How To Use Cloud Computing To Do Astronomy (and other sciences)

[Here is an excellent balanced presentation on the advantages (and disadvantages) of using commercial clouds for astronomy and other research programs.
 I have long argued that commercial clouds are not yet well suited for HPC applications, but they can play a vital role in helping medium and small science (“boutique”) science teams in addressing many of the mundane tasks  in handling their data deluge. See my recent CIFAR presentation http://www.slideshare.net/bstarn/cifar.

 Simple things like file transfer can be a major hurdle for researchers who are not computational scientists. Tools like  Globus on Line or SURFconext using commercial clouds are ideal for these boutique science teams in  simplifying or eliminating many of these mundane tasks. Use of these tools does not necessarily mean that the actual research computation is done in the cloud.  HPC facilities may still be needed to do the raw number crunching.

At a recent cyber-infrastructure event hosted by the Canadian Institute for Advanced Research (CIFAR) Dr Larry Smarr, chair of the National Science Foundation Cyber-infrastructure Committee,  pointed out that cyber-infrastructure is not a recent phenomenon.  A commitment to cyber-infrastructure reflects a long history in the US and other jurisdiction of the recognition of the importance of computation and networks to advanced science and commercial spin offs.  As Dr Smarr stated “ Cyber-infrastructure is not an option for advanced societies”.

Commercial clouds are going to play an increasingly critical component of cyber-infrastructure.  Although the authors of this study point out that the cost of commercial clouds, in some cases, can be more expensive than deploying your own cluster or HPC facility, even if you take into account depreciation and energy costs,  the big advantage of commercial clouds is “time to market”. While, in some cases a fully loaded HPC facility is cheaper, the time to get funding approval and then peer review to actually use the facility can take years.  With a commercial cloud a researcher can start immediately focusing on their science and scale up their application once they have sorted out the initial bugs and code. More importantly many graduate students and  researchers are moving their tools sets to the cloud for easy access by other members of their community, as well as for the potential to make money from various “click compute” initiatives.

While Canada may be far behind other nations in terms of developing a national cyber-infrastructure strategy with or without commercial clouds, being late has one advantage in that we can learn from other’s mistakes.  As well, rather than reinventing the wheel and trying to develop our own common science platforms or middleware, we can beg, borrow or steal from others. Most cyber-infrastructure middleware is open source and have many excellent examples we can use in Canada such as Globus On Line, COmanage, HubZero, SURFconext, NECTAR, etc – BSA]

How To Use Cloud Computing To Do Astronomy
http://astrocompute.files.wordpress.com/2012/05/cloud-ipac-may-9-gbb-ewa-final-copy.pdf



------
R&E Network and Green Internet Consultant.

email:     Bill.St.Arnaud@gmail.com
twitter:  BillStArnaud
blog:       http://billstarnaud.blogspot.com/
skype:    Pocketpro