HPC in the Cloud: HPC, the Cloud, and Core Competency

Dedicated to covering high-end cloud computing
in science, industry and the datacenter

Translation Disclaimer

Subscribe | Sign In

Blog: Behind the Cloud

April 11, 2010

HPC, the Cloud, and Core Competency

What does HPC have to do with cloud computing? Well, given that HPC environments are constantly growing, consume large quantities of fairly generic compute resources, and have both peaks and valleys in workload profiles, it would seem that HPC would be the perfect candidate for cloud computing, if only we could get past the barriers to adoption.

What I would like to do is present a series of blogs, intended to be a philosophical framing, not a technical roadmap, that will show why HPC is the perfect consumer of cloud computing. These blogs will be broken up into distinct topics in an attempt to create a logical progression aimed at having a common frame of reference. The initial set of blogs will address the barriers to adoption as follows:

1. Ego – IT as a core competency

2. Cost – getting more value for the same money

3. Trust – a historical lesson

4. Control – changes to organizational structure

5. Security – perspectives on internal security

6. Performance – realities of simultaneous optimization theory

Once we frame the barriers, we can then discuss incremental steps to get to value:

1. Cloud enablement – transforming your environment, internal private cloud

2. Private Clouds – external private clouds

3. Hybrid Configurations – leveraging public clouds for appropriate workloads

4. Public clouds – where and when they may make sense

This is the intended general direction, but I reserve the right to deviate based on input from the forum, any needed clarification, or recalibration necessary to stay true to intent of the site.

Having said that, let’s move into the first topic of discussion, IT as a core competency.

Companies need IT to be executed competently, and to control IT direction, but IT is not the primary product of the company (IT companies aside), and therefore should not be considered a core competency. We can debate tying into the primary function (core business) of the company as a criteria to determine core competency, but I believe it goes to the investment decision process of the company leadership. The primary drivers for the business revolve around delivering product to customers, development of new markets, and customer relationship management. When given the option of where to invest critical resources and assets in the business, executive management will be driven to primarily invest in the direction of the core business, and minimize expenses around all other aspects of the running of the business. Core competency would imply sufficient investment to differentiate the business from the rest of the world.

Further reinforcement of these concepts can be seen by looking at where IT is accounted for within the business. Quite commonly, IT is accounted as an SG&A function. This places it into the “overhead” bucket and it gets to compete with facilities, HR, accounting, purchasing, and all other groups that make up the SG&A bucket for the company in order to get resources. I only say this to frame the mindset of financial decisions. Given that companies are measured by how well they control expenses in SG&A (SG&A as a function of revenue), and that many of the components of the SG&A bucket are fixed or based on headcount, you then start to see that budgets for IT are scrutinized with a control oriented mindset, optimized on the cost variable. The R&D side is usually the “spend money to make money” side of the house, where SG&A is driven to control or even cut costs. Having said that, I have also not met anyone who can flip between these mindsets.

In order to control costs as much as possible and to get as much value out of what is spent in the area of IT, most companies take the approach to limit change and hire resources with a breadth of skills as compared to a depth of skills in a specific area. They will attempt to limit change in order to get maximum value out of existing assets, maximize the ability to automate, and to minimize the quantity of personnel required to manage. By limiting change like this though, it defeats the ability of technology to ultimately deliver maximized value. Also, by limiting change, the organization is really promoting a philosophy of maintenance instead of development, and in doing that, many times symptoms will be addressed (just patch it up) instead of the root cause.

Additionally, by hiring generalists, the business accomplishes many things, like having the ability to solve any problem in the environment while minimizing overhead staff in addition to having the ability to have fault tolerance in personnel resources (people can take vacations, get sick, or leave for another position). The downside is that many times these generalist resources are attempting to mange the infrastructure, but lack the experience on new technologies that are brought in to properly manage them(they have not have the opportunity to gain experience). Solutions that they develop or integrate are more prone to configuration or design mistakes (doing it for the first time), are many times less efficient solutions than what is possible (not optimized), and are not designed to scale into the future with technologies that are not yet available to solve problems that have yet to surface. And finally, the complexity of the environment is growing faster than the capacity of the organization.

This is not to say that internal IT organizations are not excellent, that the personnel are not very talented, or that these organization don’t bring great value to the companies they work for. The only point is that there is more value that could be achieved, and that the company does not (and should not) invest in this function like they do the core product(s) of the company. How many times have we all sat in on meetings listening vendors explain to us what the “perfect solution” is, and knowing that they are right because we thought of it a long time ago, but just have not had the time, funding, resources, and priority to go execute that “perfectly”. Cloud computing has the promise to grant us access to that optimized, “perfect” solution, and next time, we will talk about getting that solution for the same price we are paying for IT today…

Posted by Scott Clark - April 11, 2010 @ 7:48 AM, Pacific Daylight Time

Subscribe to HPC in the Cloud

Discussion

There are 5 discussion items posted.

Impact on the ISV community?
Submitted by peterdenyer on Apr 21, 2010 @ 5:52 PM EDT

Scott

A great start into a very interesting topic!

Another topic area, if not already anticipated in your proposed Barriers to Adoption blog sequence, could explore the impact of Cloud Computing on the application vendors (ISVs) in this market segment.

Application vendors want to run a profitable business. Changes in business model can impact that in interesting ways. We saw what happened when the business model (at least in the Electronic Design Automation segment) morphed from a perpetual license model to the current time-based licensing scenario. A somewhat chaotic time revenue-wise for that few years until time-based revenues became the norm in the industry.

Are we liable to see another big inflection point as companies move to a Cloud Computing business model, or add Cloud Computing as an offering in their product line-up?

I see some signs of early Cloud Computing experimentation by some significant EDA vendors. We certainly see Cloud Computing as being a major part of the business strategy for some of the latest entrants in the EDA space. Are other HPC-focused vendors doing the same? Is Cloud Computing in this commercial HPC market a viable , long term direction?

Without presupposing the details of the rest of your blog topics, I'd guess that it will be business (revenue) issues on the part of the application vendors that will turn out to be one of the bigger barriers to adoption for HPC in the Cloud. I'd like to hear your thoughts on this in a future blog.

Post #1

ISV part 1
Submitted by ScottClark on Apr 22, 2010 @ 8:16 PM EDT

Pete,
Great to hear from you, and glad you enjoyed what you read. Let me break this into two responses to get the entire thought out there.

I had not intended to cover the ISV situation (and there is definitely an issues there as you indicated) in the barrier section. I was going to map a potential solution to that issue in the value section, since that is where I think it is going, and it will take a bit of time as you noted about the last change that happened in this space.

If I had to layout a roadmap for this, what I would propose is that semiconductor companies should look to leverage private clouds from an external facility. This would give all the security they have today (it is a private network, just housed somewhere else) and gets the efficiency of best of breed datacenter design. I would also expect that those datacenters will begin to specialize, and so there will emerge EDA Cloud datacenters, where many semiconductor companies will all have their private clouds (hosted datacenters), and at those same facilities, the major EDA companies will also have their hosted services (IAAS from each vendor). The ISVs are already offering services this way, so proposing another facility instead of on premise at the ISV should not present a huge barrier. What this solves is the bandwidth and latency issues of moving data from the customer infrastructure to the ISV infrastructure, so it allows for multiple clouds to seamlessly participate in the customers workload.

Post #2

ISV part 2
Submitted by ScottClark on Apr 22, 2010 @ 8:20 PM EDT

What this does is allow the ISVs to get used to the idea of leveraging cloud resources, but it will just be their cloud to start. I believe that over time, they will realize that IT is not their core competency (no self promotion intended), and they would then partner with a cloud vendor to provide the infrastructure, and the ISV would supply the tools (licenses). They would by that time have some history with how to price, usage models, etc., and would feel comfortable just providing the tools, and be able to do so in a manner consistent with the interests of the shareholders of their company…

I would really enjoy hearing your thoughts on this direction, let me know what you think…

Post #3

I agree with Scott
Submitted by Anonymous on Jul 23, 2010 @ 8:53 PM EDT

The ISV market is a huge Cloud HPC confidence builder for large design houses with mission critical HPC needs that are growing at a rate outside the companies internal financial and technical deployment capabilities.

Were not talking you tube startups were talking public high tech fortune 500 companies that must achieve you tube like capabilities growth with "proven" deployment strategies.

Something only a handful of people like Scott know how to do!

Post #4

Submitted by Scott on Aug 4, 2010 @ 12:23 AM EDT

Well, not sure where the compliment came from, but thank you... and I think this is correctly stated, growth is outpacing internal capabilities, and it really has to be done right...

-swc

Post #5

Join the Discussion

Become a Registered User Today!

Registered Users Log in join the Discussion

RSS Feeds

Feeds by Topic

Feeds by Content Type

Subscribe to All Content

All

Scott Clark

Scott Clark has been an infrastructure solution provider in the EDA/Semiconductor industry for almost 20 years.

More Scott Clark

Feature Articles

Cloud Services Satisfy a Higher Calling

Higher education involves many collaborative projects that lend themselves to cloud services, however often those services are not tailored to the uniqueness of an academic environment. That's where the Internet2 NET+ project comes in. By partnering with 16 major cloud providers, the networking consortium is seeking to expedite the delivery of cloud services and by doing so advance research and innovation in the United States.
Read more...

Read more HPC in the Cloud features...

Around the Web

NVIDIA Raises Its Game to the Cloud

May 17, 2012 | NVIDIA GeForce GRID, a cloud gaming platform announced at the 2012 GPU Technology Conference (GTC), seeks to reduce the the latency associated with cloud gaming.
Read more...

Breaking the Cloud Barrier

May 15, 2012 | New Microsoft report shows that beyond the expected financial benefits, cloud services may offer more comprehensive security features compared to in-house IT operations.
Read more...

Vendors Demo Next-Gen Sequencing Platforms for Pharma

May 14, 2012 | During the second annual Pistoia Alliance conference, three teams demonstrated their newly-implemented cloud-based next-generation sequencing platforms.
Read more...

Zunicore Offers Bare Metal by the Hour

May 10, 2012 | PEER1's cloud division, Zunicore, will soon be offering GPU-equipped servers on-demand.
Read more...

US Cloud Providers Struggle With Data Privacy Laws

May 08, 2012 | The Patriot Act leads foreign governments to question the security of US cloud services.
Read more...

Read more headlines...

Appro White Paper: Enabling Performance-per-Watt Gains in HPC

04/05/2012 | Appro | Designed to meet the growing global demand for HPC solutions, Appro's Xtreme-X™ Supercomputer delivers superior performance-per-watt and reduced I/O latency while bringing significant flexibility to HPC workload configurations including capacity, hybrid, data intensive and capability computing.

Exploring the Potential of Heterogeneous Computing

04/02/2012 | AMD | Developers today are just beginning to explore the potential of heterogeneous computing, but the potential for this new paradigm is huge. This brief article reviews how the technology might impact a range of application development areas, including client experiences and cloud-based data management. As platforms like OpenCL continue to evolve, the benefits of heterogeneous computing will become even more accessible. Use this quick article to jump-start your own thinking on heterogeneous computing.