August 28, 2012
Amazon has just announced a new service called Glacier. The cloud giant describes its new offering as a secure, extremely low cost storage technology for data archiving and backup. Like the name implies, the service is slow, cold and remote. As with Amazon Simple Storage Service (S3), Glacier allows users to host their data on Amazon's infrastructure, but overall functionality and use cases are expected to vary quite a bit.
Amazon Evangelist Jeff Barr explains in a blog that Glacier's main advantage is low-cost storage, which is advertised at $.01/GB/month. Users only pay for the amount of storage they actually utilize, as opposed to subscribing for a certain data limit. The service is also set up to handle petascale volumes "and beyond."
Given these characteristics, Amazon believes Glacier will provide a decent home for digital media archives, financial records, genomic data, healthcare records, long-term backups and data held for regulatory purposes. These uses are different from S3 implementations, which focus on Web applications and content distribution. Both services claim similar reliability rates with four nines availability and eleven nines durability. this means that if a company uses it to hold 100 billion objects, it can expect to lose one each year.
Amazon explains that Glacier's lower price point is enabled through slower processing times. When a user wants to retrieve an archive, their request gets queued. In roughly 3-5 hours, the data is made available for download and remains available over a 24-hour period.
Keeping in mind that S3 was designed for rapid retrieval of data: it costs $0.125/GB/month to store up to 1 TB using the simple storage service. As data levels increase, the price can drop as low as $0.055/GB/month but this requires the user to store over 5 petabytes on Amazon's infrastructure. On the other hand, Glacier runs $0.01/GB/month regardless of how little or much a user decides to store.
To incentivize customers to choose the appropriate storage service, Amazon charges for retrieval of data on Glacier. Users can download up to 5 percent of their average monthly storage for free. Beyond that, Amazon charges retrieval fees to the tune of $0.01/GB.
Setting up a Glacier account appears to be a relatively simple process. As Barr explains:
To store data in Glacier, you start by creating a named vault. You can have up to 1000 vaults per region in your AWS account. Once you have created the vault, you simply upload your data (an archive in Glacier terminology). Each archive can contain up to 40 Terabytes of data and you can use multipart uploading or AWS Import/Export to optimize the upload process. Glacier will encrypt your data using AES-256 and will store it durably in an immutable form. Glacier will acknowledge your storage request as soon as your data has been stored in multiple facilities.
Initiating the process will require client software or familiarity with Amazon's SDKs and APIs. FastGlacier is a free client for Windows users, although as of today, no Mac clients have been developed. This is unfortunate, as the writer of this article considers the cost and functionality of Glacier as good reasons to store old media archives on Amazon. The only roadblock to migration is the lack of a Mac client.
The tutorial video below explains how to setup Glacier for the first time.
While the new service offers a low-cost solution for archiving and backing up data, Amazon is not the only player in this field. Yesterday, Venture Beat covered another storage provider named Quantum. Their Q-Cloud service aims to handle similar workloads as Glacier, but combines on-premise and cloud storage to enable faster access to saved datasets. Like S3, the service has a variable pricing structure based on how much data is stored by the user. While initial costs offer little advantage to Glacier, when data stored consumes more than 72 TB, the price drops to $.15/GB/mo. This assumes Glacier users chose a 15:1 deduplication ratio.
Enterprise IT typically dictates archiving with off-site tape backups. With this development, Amazon is looking to change the norm with their Glacier service. Whether the project will pick up steam is yet to be seen, but it appears to be a solid use of cloud technology without overplaying its hand.
Amazon Glacier is available now across the US, and in Europe and Japan. The Amazon Glacier detail page offers additional details and pricing information.
Jun 19, 2013 |
Ruan Pethiyagoda, Cameron Boehmer, John S. Dvorak, and Tim Sze, trained at San Francisco’s Hack Reactor, an institute designed for intense fast paced learning of programming, put together a program based on the N-Queens algorithm designed by the University of Cambridge’s Martin Richards, and modified it to run in parallel across multiple machines.
Jun 17, 2013 |
With that in mind, Datapipe hopes to establish themselves as a green-savvy HPC cloud provider with their recently announced Stratosphere platform. Datapipe markets Stratosphere as a green HPC cloud service and in doing so partnering with Verne Global and their Icelandic datacenter, which is known for its propensity in green computing.
Jun 12, 2013 |
Cloud computing is gaining ground in utilization by mid-sized institutions who are looking to expand their experimental high performance computing resources. As such, IBM released what they call Redbooks, in part to assist institutions’ movement of high performance computing applications to the cloud.
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/02/2012 | AMD | Developers today are just beginning to explore the potential of heterogeneous computing, but the potential for this new paradigm is huge. This brief article reviews how the technology might impact a range of application development areas, including client experiences and cloud-based data management. As platforms like OpenCL continue to evolve, the benefits of heterogeneous computing will become even more accessible. Use this quick article to jump-start your own thinking on heterogeneous computing.