You think you have high storage capacity needs? Well, this week the National Center for Super Computing Applications (NCSA) added 20 PB – that’s right, petabytes – of tape capacity for online data for its Blue Waters supercomputer. And that should last about a year.
The NCSA uses four SpectraLogic 17-frame T-Finity tape libraries with IBM TS1140 tape drives for all of its active archiving for the Blue Waters super computer that went into operation a year ago. That setup can store 380 PB, which should be enough for Blue Waters’ expected five-year lifespan. Michelle Butler, NCSA senior technical program manager, said Blue Waters’ archiving data is expected to grow by about 60% to 70% per year.
The SpectraLogic libraries are connected to a Cray-branded Seagate high performance computing disk system that holds 35 PB of raw data and 25 PB usable.
“We need to be able to stay ahead of our users,” Butler said. “We are continuously growing, but we have stored a little less than 20 PB in data in our first year.”
Blue Waters, based at the University of Illinois at Urbana-Champaign, is used for a variety of research data. Applications include weather prediction and analyzing how the cosmos developed after the Big Bang. Butler said there are 36 teams of from 10 to 20 researchers per team that use the system, with between 100 and 200 users online at any time. The supercomputer includes 28 systems dedicated to online data movement and 50 for nearline data movement using one or two 40-Gigabit Ethernet cards.
Butler said NCSA chose tape for archiving because hundreds of petabytes of disk would be too costly. It is also easy to non-disruptively add capacity to the tape libraries.
Writing that much data to tape efficiently and concurrently did require the Blue Waters team to write a RAIT (Redundant Array of Inexpensive Tapes) utility for its IBM HPSS (high performance storage system) hierarchical storage management system.
RAIT enables nine wide data stripes and allows for the loss of two tapes without losing access to data.
“With RAIT, we can stripe data and still protect it,” she said. “We needed to strip data and write data extremely fast to the tape drive. We couldn’t single-stream files, that’s too slow. But we didn’t want to lose users’ data if a tape or drive fails. With a seven-wide stripe of data, that would be 28 TB of data if we drop a tape. Now we can do seven wide stripes of data and two wide stripes of parity. We can lose two tapes and still continue to retrieve users’ data.”
Now in its twelfth year, the Storage magazine/SearchStorage.com Products of the Year competition is open for entries. If you’re a vendor, a PR firm representing a vendor or just a very satisfied user, fill out the entry form today to make sure your product is considered for one of these prestigious awards.
As you may recall, our Products of the Year program is a bit different from some other “best products” or “user choice” awards—we focus on what you have done lately, so we consider new products that debuted during the last year or existing products that were significantly revved during that time.
This year, our panel of judges will consider products in five categories:
- Storage Systems – SAN/NAS/multi-protocol systems, converged/hyper-converged infrastructure products, HDDS and SDDs, disk controllers, caching appliances, storage virtualization appliances, cloud-integrated storage
- Backup Hardware – tape libraries/drives, backup media, disk backup targets, VTL, dedupe devices, cloud backup gateways
- Backup and DR Software and Services – backup/recovery software, cloud backup/recovery services, disaster recovery, snapshot and replication, electronic vaulting, archivers
- Storage Management Tools – SRM/SAN management software, performance monitoring, configuration management, provisioning, data reduction
- Storage System Software – file systems, volume managers, storage virtualization software, security software, storage optimization, solid-state caching, cloud storage software, software-defined storage
Our panel of judges includes some of the most knowledgeable and best known storage industry experts—analysts, consultants, users and editors.
Don’t let this opportunity to gain some recognition for your product slip by—the deadline for entries is Friday, October 3 at 5:00pm PST. You can enter as many products as you like, but please don’t enter the same product in multiple categories (that’s a no-no!). Included on the entry form page are links to additional information: You can get more information about the judging criteria for the product evaluations, as well as some entry submission tips from the judges.
Don’t wait—enter now. And good luck!
Terms of the agreement were not disclosed but Axcient already has integrated DirectRestore’s technology for Microsoft Exchange servers. The company intends to integrate the capabilities into its cloud platform for SQL Server and SharePoint databases over the next two quarters, with other applications to follow..
“DirectRestore specializes in granular application recovery and they support more than 100 application formats today,” Axcient CEO Justin Moore said. “This is very synergistic with our existing platform. We now have the team that has deep experience in granular recovery so we can expand Axcient to other applications.”
Axcien claims more than 3,000 customers on its platform. It takes a full image copy of servers that customers can access and it can restore data from any device when systems go down.
With the DirectRestore capabilities, Moore said the Axcient platform will be able to do granular recovery down to the object level along with files and folders. That means customers won’t have to buy individual products for specific applications.
“There are only two or three companies that have the type of technology that DirectRestore has. Anyone that has granular technology, almost all OEM it,” said Moore. “(This acquisition) not only gives us control of the technology but by integrating it into our technology, we control the entire stack.”
Moore said granular recovery is available now for Microsoft Exchange servers, and has about 500 customers using it.
DirectRestore’s 15-member engineering team will join Axcient, and Moore said he intends to double the group over the next 12 months.
“Our DNA has been in the small to medium-sized businesses but with the (new) technology we want to take on larger customers that can get access to all of our functionality,” said Moore. “The granular recovery technology will expand beyond the products we have today.”
The industry’s two hard drive vendors had a busy week with product rollouts and future-gazing.
Western Digital’s HGST division launched a flurry of products Tuesday, including 8 TB and 10 TB helium drives, nonvolatile memory express (NVMe) PCIe flash drives, and flash software. It also revealed its air-filled UltraStar hard drives will be replaced by the helium drives after the current generation.
Rival Seagate launched a raft of products Wednesday, and talked about its business strategy today at an analyst day.
Seagate’s new products include the ClusterStor 9000 Lustre-based high-performance computing system that it gained from its Xyratex acquisition; an EVault Enterprise Backup and Recovery Appliance that handles up to 100 TB of usable capacity with data reduction; Nytro XP6302 (1.75 TB usable) and XP209 (1.86 TB usable) PCIe flash cards; and 15,000 rpm and 10,000 rpm hard drives, including a 2 TB 2.5-inch drive.
While HGST is pushing hard to expand into solid-state storage, Seagate seems more interested in refining its hard drive technology for the cloud. It did buy LSI’s flash controller technology to “control NAND better than anybody else,” as Seagate president of operations and technology Dave Mosley said, but its executives seemed most excited about the cloud during analyst day.
“I think people accept that the cloud architecture will be the architecture of the future,” Seagate CEO Steve Luczo said.
“Cloud storage is the thing that’s really exploding,” Mosley added.
Seagate set up a cloud systems and solutions division this week, headed by Cisco veteran Jamie Lerner. Its Kinetic open storage architecture revealed last year is also built largely for the cloud.
CFO Patrick O’Malley said cloud and flash products can bring Seagate $2 billion in revenue over the next two years, before adding the hard disk drive business “is [still] a story of growth.”
Luczo played down the emergence of flash in enterprise storage, saying nearly all of the flash in use is connected to computers (clients and servers) rather than the storage bus.
Mosley said Seagate is on track to deliver a 20 TB hard drive by 2020. That’s twice the capacity of the largest HGST rolled out this week, and 2.5-times the 8 TB drive Seagate announced last month.
Companies are offering a number of deals to lure customers to the cloud, the most popular is bargain-basement price cuts on per-gigabyte storage. Riverbed Technology has joined in with a slightly different take. Riverbed is offering a virtual version of its SteelStore appliance, formerly known as Whitewater, with up to 8 TB of storage for six months but you have to sign up over the next month.
The deal started on Sept. 9 and runs through Oct. 9. The SteelStore appliance is bundled with the 8 TB of storage and six months of Amazon Simple Storage Service (S3) for backup and archiving.
When customers reach the 8 TB limit, they will have to upgrade and purchase and license one of Riverbed’s eight other physical or virtual appliances. Riverbed supports up to 95% of the cloud providers in the market but this deal is strictly tied to Amazon’s cloud.
“The idea is to give customers a kick start in cloud storage,” said Rachel Dines, Riverbed’s senior product marketing manager. “We wanted to give customers a way to dip their toes in cloud storage.”
This deal is only being offering in the United States and Canada. Customers can download a link from Riverbed’s website for the SteelStore virtual appliance and get the promotional code to connect to Amazon S3 storage. Dines said they are targeting small and medium-sized enterprises and only one appliance is being offered per customer.
SteelStore uses compression, de-duplication, network acceleration and encryption to increase performance and secure data moving to and from the cloud.
Dell’s newest PowerEdge servers are going to use the SanDisk DAS Cache software to automate high-performance data on SSDs.
The next-generation R730, R73oXD, R630, T630 and R920 uses direct-attached disk storage with SSDs to boost performance for OLTP, OLAP, analytics, business intelligence, HPC and other enterprise applications. The software supports up to 16 TB in each cache, whether it is SATA-based or PCI-e SSDs.
“The software observes the data reads and writes,” said Rich Petersen, SanDisk’s director of software marketing. “It sees data blocks that are accessed more frequently and put it in cache. It’s transparent inside the operating system. It has a very light footprint in the server. The applications don’t see it. They just see it runs much faster.”
The software supports any type of SSD, with customers choosing write-back and/or write-through caching modes. It supports multiple host operating systems, including Windows) and Linux, and Microsoft Hyper-V and KVM hypervisors. VMware and Ubuntu support on the roadmap. The SanDisk DAS Cache will work in any server rack, tower or blade server configuration.
“From the application’s perspective, the write is executed faster,” Petersen said. “It provides redundancy with either software or hardware RAID. Redundancy generally is not needed for reads.”
Scality’s Ring software came into the market focused on object storage and added support for unified file storage last year. Now Scality has extended Ring’s reach with VMware support.
Philippe Nicolas, Scality’s senior director industry strategy, said the goal is to consolidate multiple workloads in a single platform and have the software behave like Amazon EBS for an on-premise cloud.
“(Virtual machines are) so well adopted by the enterprise that, for us, it is a request from users,” Nicolas said. “Scality can behave like Amazon for on-premise cloud. The focus target is the VMware environment, especially vSphere. VM can talk to it parallel to the Scality Ring. ”
The company next year intends to support vStorage API for Array Integration (VAAI), an API framework from VMware that enables certain storage tasks, such as thin provisioning, to be offloaded from the VMware server virtualization hardware to the storage array. It will also support VMware Storage API for Storage Awareness (VASA).
Scality claims Ring 5.0’s performance matches Amazon EBS at 200 IOPs per virtual machine, with bursting capabilities of 3,000 IOPs per virtual machine. Scality Ring supports replication and erasure coding in virtual machines for data resilency.
Nicolas said the software supports OpenStack Cinder to provision Scality VM storage. Ring 5.0 also supports virtual machine file system (VMFS) and NFS.
“We let the customer chose if they want VMFS or NFS,” he said. “We don’t prefer one or the other. Some use enterprise backup solutions for VMs so they can use those for backing up.”
Revenues for external disk storage among all vendors fell 1.4% from last year during the second quarter, according International Data Corp.’s worldwide disk systems tracker.
It marked the fourth consecutive quarter that high-end storage sales declined, and IDC noted that midrange system revenue also dropped.
Startups and smaller companies made up a bigger piece of the external storage pie, which came to about $5.9 billion in the quarter. The “others” category – all vendors except for EMC, NetApp, IBM, Hewlett-Packard, Dell and Hitachi Data Systems — increased revenue 11.1 percent in the quarter and market share was at 21 percent compared to 18.7% during the same time last year.
IDC reported that leader EMC experienced a 5.2 percent decline in year-over-year revenue for the second quarter, with NetApp registering a 3 percent decline and IBM a 4.7 percent drop in external storage sales. Hitachi was hit the hardenst with a 12% year-over-year decline and Dell with the lowest at one percent. HP experienced a nominal gain with a 0.4 percent increase. EMC grabbed 30.1% of the external disk storage revenue during the second quarter but that was down 31.2% from the prior year.
NetApp and IBM followed EMC with shares of 13% and 12.1% respectively. In comparison, NetApp had 31.2% in revenue sales in the second quarter last year and IBM had 12.5% in the same time period. HP gained share during the second quarter with 10.1% compared to 10% in second quarter last year.
This picture is in stark comparison to smaller companies, like hyper-converged vendor Nutanix and flash vendor Nimble. Nutanix reported that it had exceeded $50 million in revenue for the second quarter that finished in July this year,more than tripling its revenue for the same quarter a year ago. The company picked up larger customers with 29 companies buying more that $1 million of Nutanix products and services. That number has more than doubled since January when Nutanix had 13 million-dollar customers.
Nimble reported it had hit record numbers with $53.8 million in revenue and 663 new customers during its fiscal second quarter this year and closed 444 deals in excess of $100,000 for the 12-month period ending on July 31. Nimble’s revenue increased 89 percent over last year.
“Large companies are starting to feel the impact,” said Nutanix senior vice president of product management Howard Ting. “The disruption created by young companies like Nutanix is eating into their revenue.”
It seems clear that all solid-state technology will become common in primary storage systems soon. In our IT client engagements, we see many companies including the transition to all solid state in their storage strategies.
As with many new technologies, there are both compelling reasons to adopt solid-state storage and valid reasons for waiting.
The reasons for using flash as primary storage include:
• Performance – As a memory-based technology, flash eliminates latencies seen with electro-mechanical devices (hard disk drives). The performance bump is especially noticeable for random access to data and the time to first byte of transfer, and changes the value from storage by accelerating critical applications.
• Reliability – Improving the reliability of the technology has been a hallmark of storage vendors and this continues with flash-based storage systems. Without the mechanical wear and the heat generation, solid state inherently is more reliable.
• Longevity – With greater reliability and intelligence added to manage page erases, the flash storage systems have been given longer warranty periods by many vendors. Customers are beginning to budget for a longer lifespan for flash-based storage systems than mechanical-based storage systems.
• Power, space, cooling – Flash storage systems require less power and cooling for a given capacity than a system with disk drives and are more efficient in space usage.
• Data reduction – Most of the flash-based storage systems include compression and/or deduplication. The data reduction exploits the performance of memory-based technology and provides the benefits of reducing the cost of data stored while contributing to the page erase management for flash.
Another advantage comes from the fact that many of the flash storage systems are scale-out architectures. Scaling out not only allows the capacity and performance to scale in parallel, it changes the acquisition model for storage. Additional capacity (without reducing effective performance) is added to the storage system when needed. With the continuing decline of the price of flash storage, the additional storage is less expensive than the initial purchase.
So when will flash systems become predominate as primary storage rather than used for particular applications such as virtual desktops? The triggering events for change are usually technology updates, new applications or major deployments such as a new site. However, several factors make people hesitate from making a transition:
• Operational changes are difficult to make because they require additional effort and introduce risk. These changes could include script changes and training. Consequently, flash storage systems that have evolved from existing architectures have less of an impact on the requirements for operational changes than new architectures built for flash.
• Not all flash systems can handle advanced capabilities such as stretched clusters and multi-site replication that are part of many IT operations. This removes them from consideration for primary storage, and relegates them to specialty usage.
• Vendor product support and system maturity are always a consideration for enterprise environments when considering primary storage.
The use of solid-state arrays as primary storage will bring an increase in the average size of the systems deployed. Significant capacity increases beyond the early use cases of application acceleration (primarily databases), server virtualization data stores, and VDI will be the bell-weathers of the transition. Actual primary storage capacity size changes may also be affected by another IT trend – moving data for non-tier one applications to less expensive storage platforms with different performance and data protection requirements.
(Randy Kerns is Senior Strategist at Evaluator Group, an IT analyst firm).
At a time when most of the activity in the storage market seems to be focused on taking away as much intelligence as possible from physical storage devices, there’s one vendor bucking that trend. DataGravity, which recently emerged from stealth, rolled out a couple of new arrays that up the ante for data intelligence and management in a storage system.
DataGravity’s Discover Series comprises two unified arrays at this time: the DG2200 with 48 TB of combined flash and hard disk raw capacity and the DG2400 with twice that capacity. These are dual controller systems that support both block—iSCSI—and file (CIFS, SMB and NFS) storage. Both models are configured with what DataGravity refers to as 2U “computing” and 4U “storage” enclosures. DataGravity was founded by Paula Long, CEO and John Joseph, president; the two previously teamed up to create EqualLogic, the iSCSI storage pioneer that was acquired by Dell in 2007.
So far, these two boxes from DataGravity might sound like just about any other midsized array that mixes in a little solid state with hard disk storage—right? As unremarkable the hardware configurations might be, it’s the systems’ software that’s the real story here.
A while back in an editorial in Storage magazine (Data protection methods, define thyself), I suggested that stored data needed to carry more intelligence about itself. In the context of the editorial, that intelligence would be in the form of metadata that instructed data protection systems on how to handle that particular piece of data and what to do with.
DataGravity’s Discovery Series takes a different approach, but the results are pretty similar to the system I imagined. The differences are that DataGravity isn’t imagining it—it’s here now—and they pool the metadata the system gleans in a central repository as opposed to packing it in with the data itself.
The clever engineers at DataGravity realized that a key component of a high availability storage system—the secondary controller—spent much of its time sitting idle. They use the horsepower of the controller to index stored data and to parse that data into useful information about the file. DataGravity also does the usual stuff and keeps track of creation and modification events and who was responsible for those activities.
But the collected metadata reveals even more about the contents of the data, allowing searches for specific items such as “personally identifiable information“—or PII—that could include social security numbers, credit card numbers or email addresses. Information governance, whether internal corporate governance or compliance with legal regulations, is aided by the ability to do complex, sophisticated searches that can identify some of the interrelations among disparate files.
This deep dive into data can also help add another level of protection to the data itself. The DataGravity system can create DiscoveryPoints which keep track of all changes and activities related to a piece of data. DiscoveryPoints work like snapshots and allow recovery of previous versions of data if the primary copies become damaged or corrupted. One of the neat things about DiscoveryPoint is that data can be recovered at the file (even from a VM’s VMDK), VM, file system or LUN levels.
The Discovery Series is brand new, but it isn’t tough to envision it serving as a platform for archiving, data protection and disaster recovery systems. As DataGravity opens their APIs to other companies, those vendors will be able to hook into the boxes and integrate their capabilities on top of the data intelligence that DataGravity provides.
[DataGravity recently won a Best of VMworld 2014 Award for New Technology; the award was presented by TechTarget’s SearchVirtualServer.com site.]