HPC in the Cloud


Dedicated to covering high-end cloud computing
in science, industry and the datacenter

Language Flags

Database Automation for the DBA Shortage


As the amount of data continues to grow, the need for experienced database administrators (DBAs) is exploding. The U.S. Department of Labor and Statistics reports that employment of DBAs is expected to grow significantly and outpace the growth of all other occupations through 2016. The DBA has a very specific, technical set of skills that makes the position one of the most difficult to fill in the datacenter. With only a handful of skilled database administrators, many companies will struggle to meet this demand.

Database automation is going to play a much greater role for future success. By implementing database automation technologies, DBAs can focus on strategic issues and allow junior staff to perform tasks traditionally executed by more skilled workers. This is a win-win situation for business decision makers; the highly skilled DBAs can focus on revenue-generating strategy, while the less skilled employees can maintain compliance metrics and best practices.

According to the leading analysts, DBAs spend the vast majority of their time performing mundane tasks that focus on keeping the ship afloat. Senior DBAs have been reduced to robot operators, following run books, escalating to support, and constantly reacting to the latest wave of fires. When you consider that these are your data people and the importance of understanding your data can mean a company's success or failure, it is surprising that so little has been done to change this situation, especially considering the following:

  • The number of databases and amount of data tracked is growing, and growing fast.
  • Database environments are getting more complex as the number of databases and amount of data within those databases continue to increase. Combined with hundreds of different patch levels, the introduction of new versions and security updates, it's all DBAs can do to keep up with their job.
  • The DBA is a high-turnover position. Nearly half of all DBAs will leave their job within 24 months. Additionally, the DBA is one of the most difficult positions for an organization to replace.

With these concerns and the U.S. Department of Labor and Statistics reporting that the "database administrators occupation is expected to grow 37 percent," how will you prepare for the demand? As the amount of data under management and the standards by which that data is managed are exponentially increasing, companies have no choice but to alter the way they manage information. Automation is the answer.

While automation has spread widely across the datacenter, DBAs have yet to apply these principles to the database. Companies need to implement a centralized, automated mechanism to manage their databases. With automation tools, senior DBAs are free to stop "pushing buttons." They can give the repetitive tasks to less skilled workers and focus more on strategy and revenue-generating practices. Meanwhile, mid-level workers can take on roles that were traditionally reserved for the IT industry experts. Because automation eliminates human error, business leaders can be assured that jobs are performed to their satisfaction.

Once the power of automation is realized, the implementation begins. The first step toward implementing a complete automation solution is creating an extensible and maintainable framework. This framework must be able to handle several system requirements, such as deploying an entity to a server, executing a script in the context of a database object, and removing the database object. However, it is quite challenging to create a framework that is adaptable enough to manage complex databases and is easy to use. Interacting with a complex application, such as Oracle DBMS or Microsoft SQL Server, requires running several commands, each with unique arguments and each in a unique way to determine if the task was successful or not. Frequently, running these specific commands requires the execution of additional commands just to judge the results of a prior act. The framework must be able to deal with these interactions independent of the architecture (e.g., clustered, non-clustered) and environment (e.g., different operation systems, different database versions, etc.).

Each organization also has customizations beyond the scope of managing the actual database application that must be integrated into the automation framework. These requirements can exceed the complexity of the core database automation. In many cases, organizations can leverage preexisting work to ensure this logic is incorporated into the activities being automated.

It's not a good use of time or money to design or implement your own automation framework and scripts to manage the database. It's also unlikely you are the only candidate for automation, so if you are considering automation, it's best to buy rather than build this aspect of your infrastructure. There are solutions that have this framework and come with pre-built automation that understands and adapts to database versions, operating systems and architecture. There is a constant stream of work required to update an automation framework based on new requirements. Beyond that, the actual code used to call out to the managed applications is fluid, due to the managed software updates.

Implementing an automation process for your DBAs to manage is a win-win solution for business decision makers. It allows the highly skilled DBAs to focus on revenue-generating strategy, while the less skilled employees maintain compliance metrics and best practices. If companies are able to adopt an automation strategy into the mix, they will be able to leverage the skills of their existing staff, without hiring additional staff in the coming years. Given that the U.S. Department of Labor and Statistics has predicted that employment of DBAs is expected to grow "much faster than average," demand for this type of skilled worker is very high and qualified candidates will demand highly competitive salary and benefit packages. Why not make them as efficient as they can be? Now is the time for organizations to consider automation as an essential part of running an effective database environment.

Most Read Blogs

Aspen

Short Takes

NASA Builds 'Climate in a Box'

May 23, 2013 | he study of climate change is one of those scientific problems where it is almost essential to model the entire Earth to attain accurate results and make worthwhile predictions. In an attempt to make climate science more accessible to smaller research facilities, NASA introduced what they call ‘Climate in a Box,’ a system they note acts as a desktop supercomputer.
Read more...

Running Computational Fluid Dynamics in the Cloud

May 16, 2013 | When it comes to cloud, long distances mean unacceptably high latencies. Researchers from the University of Bonn in Germany examined those latency issues of doing CFD modeling in the cloud by utilizing a common CFD and its utilization in HPC instance types including both CPU and GPU cores of Amazon EC2.
Read more...

In Support of Cloud-based Rendering

May 10, 2013 | Australian visual effects company, Animal Logic, is considering a move to the public cloud.
Read more...

Internet2 Awards Program Seeks Innovative Applications

May 10, 2013 | Program provides cash awards up to $10,000 for the best open-source end-user applications deployed on 100G network.
Read more...

Sponsored Whitepapers

Best Practices in Big Data Storage

05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.

Exploring the Potential of Heterogeneous Computing

04/02/2012 | AMD | Developers today are just beginning to explore the potential of heterogeneous computing, but the potential for this new paradigm is huge. This brief article reviews how the technology might impact a range of application development areas, including client experiences and cloud-based data management. As platforms like OpenCL continue to evolve, the benefits of heterogeneous computing will become even more accessible. Use this quick article to jump-start your own thinking on heterogeneous computing.

Sponsored Multimedias

Newsletters

Stay informed! Subscribe to HPC in the Cloud email Newsletters.

HPC in the Cloud Update
HPCwire Weekly Update
Digital Manufacturing Report
Datanami
HPCwire Conferences & Events
Job Bank
HPCwire Product Showcases


ISC

HPC Job Bank


Featured Events



  • June 16, 2013 - June 20, 2013
    ISC'13
    Leipzig,
    Germany

  • June 17, 2013 - June 18, 2013
    Forecast 2013
    San Francisco, CA
    United States




HPC in the Cloud Conferences & Events