Storage Soup

Nov 19 2010   8:15PM GMT

IBM shares peek at shared-nothing cluster technology

Dave Raffo Dave Raffo Profile: Dave Raffo

IBM won the Supercomputing 2010 HPC Storage Challenge this week with a technology designed for signifcantly improvement performance of running analytics and queries for large data sets as well as cloud applications.

Developed at IBM Research Almaden, the General Parallel File System-Shared Nothing Cluster (GPFS-SNC) uses the Hadoop Distributed File System (HDFS) for what IBM calls “high availability through advanced clustering, dynamic file system management and data replication, and can even continue to provide data access when the cluster experiences storage or node malfunctions.”

Prasenjit Sarkar, master inventor for storage analytics and resiliency at IBM Research said the technology uses a distributed architecture where each node is independent and tasks are divided between computers. No node has to wait for another to perform a task. This removes bottlenecks associated with SANs because there is no single point of failure.

“The goal is to store large amounts of data as efficiently as possible,” he said. “This is an architecture for petabytes and even exabytes.”

He said the architecture includes enterprise features such as client-side caching, disk caching, wide area replication, and archiving.

Sarkar said he couldn’t talk about any product plans or roadmap for GPFS-SNC, but he said possible use cases include analytical queries, largescale data warehousing products and cloud computing where storage is accessed in parallel. GPFS is used in IBM’s SONAS scale-out NAS product and its Smart Business Compute Cloud, so the new architecture is likely to show up there. It’s also a candidate for IBM’s recently acquired Netezza data warehousing platform.

 Comment on this Post

There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

Share this item with your network: