We just returned from an extremely successful foray into Bio-IT World’s Cloud Computing Conference in La Jolla, Calif., where we were honored to present some of Opscode’s contributions to the world of cloud computing.
The most exciting news was Cycle Computing’s announcement of the successful deployment of a 30,000-node cluster in the cloud, the first of its kind in the industry. It performed significant scientific work in molecular modeling for a top-5 pharmaceutical company, employing multi-AWS-region support, massive spot-instance support, and enormous CycleServer monitoring, all for the amazingly low operating cost of $1,279/hour. The full story of this deployment is detailed on Cycle Computing’s blog.
This effort triples Cycle’s earlier scaling of its Tanuki 10,000 core cluster last April for biomedical giant Genentech. A full case study on this earlier Cycle Computing deployment can be found here.
As with that 10,000-core cluster effort, Opscode played a major role in the 30,000 cluster Nekomata spin-up. Cycle Computing used its flagship cluster and performance analytics software product, called CycleServer, to track utilization, diagnose performance and manage the progress of the scientific workflow. In addition to the CycleServer software, Cycle engineers leveraged open source projects including Condor, Linux, and Opscode’s Chef.
At the conference this week, Cycle Computing also announced a new version of CycleServer with a plug-in for Opsode’s Chef monitoring and analytics, called Grill, providing visualization into what’s cooking infrastructure-wise for the 30,000-core Chef environment. Keeping track of all the moving parts on a cluster of this scale is nearly impossible, and Cycle needed to take IT automation to the next step to manage and monitor every instance. Opcode’s Chef software enabled Cycle to consistently configure over 3,800 servers for this cluster, shaving off days of preparation and operational overhead. Grill enables CycleServer’s visualization and analytics-based alert technology to now support data about Chef installations.
For Nekomata, Cycle built a Grill view that shows a timeline of all converges and a window showing drill-down to individual hosts and their converge history. Important information such as converge duration, number of resources updated and an exception stack trace on failure are all just a click away.
With Cycle helping push the envelope of cloud-based, scale-out HPC environments for a number of industries, including life sciences, more and more companies will certainly begin adopting these new approaches to advance their businesses. That will require new methods of configuration management and infrastructure automation. Opscode is proud to help play a dynamic role in this profound leap of science.
Here is to 30,000 cores in the cloud. Science!