How cloud bursting could accelerate discovery
Implemented properly, technology innovations like cloud bursting can fundamentally accelerate the rate of important research.
Article by: Daniel Chow, COO/CTO Silicon Mechanics


At the recent Bio-IT World Conference in Boston, I had the privilege of speaking to an audience made up primarily of life sciences and medical researchers. My main message concerned the cloud, specifically the trend of cloudbursting. This audience is extremely important to me personally, as my daughter was diagnosed with autism at the age of two.

She is now seven years old, stands 3’9” and is in the first grade -- the sweetest kid you could ever meet -- and though she is reading at grade level, her life is not without serious challenges. As a parent, when you see your child hooked up to machines and sensor probes that monitor her brainwaves due to a recent string of epileptic seizures, it is heartbreaking to watch.

Looking through my lens as a technologist, however, I learned that there really is a strong marriage between technology and medicine. And that as technologists we need to continue to hone that marriage.

Many of those researchers in attendance at Bio-IT World currently conduct research that may someday hugely benefit millions of kids with special needs, and in turn their parents. Technology, implemented properly can fundamentally accelerate the rate of this research. The question is, how can we help those researchers and the people that support them accelerate their adoption of new technology, but do so in a meaningful and cost-effective manner without making a litany of mistakes?

In my opinion, the biggest boost currently available from a technology standpoint is the use of the cloud, and specifically the act of cloudbursting.

Cloudbursting, by definition, is when an application runs in an on-premise private cloud or an off-premise private cloud (data center), and then "bursts" into a public cloud when the demand for computing and/or storage capacity spikes. The advantage of such a deployment is that a business or corporation only pays for the extra computing resources when they are needed. The disadvantage to cloudbursting is in how it handles sensitive information.

Compliance and security is admittedly an afterthought, versus integrating a cyber security strategy into a technical strategy. This makes implementing those policies and security controls in the cloud much more difficult after the fact.

For example, if you burst to the cloud to get an application or platform to market faster, but suddenly found out that the network ports that were opened to facilitate network communication were closed. There is a common need for providing an end point to upload and download files. Some technologists or developers may open "clear text" ports like FTP versus SFTP as a basic example.

Cloudbursting implementations provide a wealth of benefits -- high-performance cluster extensions, or high-performance cluster on demand, or temporary storage offload, but also brings some challenges associated with deployment, such as:
  • Is there a cohesive strategy? Alignment of the research initiatives and the available technology may seem like common sense, but there are instances where this connection point is often overlooked. The IT decision to offer or move to cloud-based architectures should not be handled in a silo, but through coordinated communications between IT staff and scientists with laser-like focus on: gathering front-end requirements; understanding workflows and how IT decisions can impact hardware, compute, or storage requirements; and what compute set up is ideally suited for bursting to the cloud.
  • Networking bottlenecks and performance tuning. If working in an environment which requires transferring large amounts of data, either pre or post-analysis can present communication latency challenges as well as a risk of extra costs. If seeking unique performance tuning options, keep in mind that the options may be limited. Most cloud service providers (CSPs) charge for network traffic utilization. It's no cost to move data to the cloud, but it definitely costs money to move that data back on-premise. In addition, Amazon and other CSPs don't allow you to create large virtual machines that utilize a lot of memory, or a dedicated high bandwidth pipe.
  • Turn it off! You pay for what you use, which is nice, but like light bulbs, sometimes even the most brilliant people forget to turn stuff off. For example, a researcher may burst to Amazon and spin up various virtual machines, but once the research is completed or workloads start to decrease there are times that they forget to delete or "terminate" their instances, while the service provider continues to charge for the usage.
Fortunately, there are tools available to help you succeed with your cloud deployment -- management dashboards for example -- and there are lots of them out there, but it is imperative that you choose the right one for your team’s specific implementation. For example, if your team was taking on an Openstack cloud implementation, I might suggest Bright Computing, or Domo. Simply do your homework to find the best fit, but don’t do all of the heavy lifting yourself. If you have a relationship with a consultant or system integrator, reach out to them for help, and ask for the documentation mentioned above.

Secondly, how does your organization evaluate disruptive technology? In the case of life sciences companies and organizations, that research typically spans a long lifecycle and technology is constantly iterating on an almost quarterly cadence, so these evaluations are key to successful implementations. Does it have a standard methodology in place to help make informed and accurate evaluations?

At the end of the day the key takeaway for those looking to implement a cloudburst system is simply to understand that no single solution fits everyone’s needs. Researchers at a life sciences organization will have very different needs from an IT team building out data centers for an enterprise company. Knowing what your organization’s needs are before you start collecting build bids is the smartest first step you can take. Do your due diligence on the associated costs. Create a methodology for evaluating disruptive technology. And make sure everyone involved in the upgrade has some skin in the game regarding the implementation.

Researchers, especially pediatric researchers are very important to my family, and to the world at large. As the medical research world collides with emerging and disruptive technology it is upon the technology community to provide those researchers with the strategies and tools to most effectively help them get their research to market in the fastest possible manner. We will all be better off as a result.