We’ve coined the term big data and talk about it because it represents a problem. Basically, the sizes of data sets being analyzed and the speed with which users need results from that analysis have exceeded the capabilities of traditional IT infrastructures, especially storage.

There’s also a cultural aspect to big data. It seems that people (especially those outside of IT) assume that business value, market insight or even predictions of the future are available to all, if you can just analyze enough data. The movie Moneyball and several television shows, like “Person of Interest” and “Numbers,” are tapping into this sentiment. There’s even one, “Touch,” that puts the analysis engine inside a person’s head. Of course, all this serves to stoke the fires of big data expectations and enlarge the problem.

The term big data was originally applied (for the most part) to analytics applications. At Storage Switzerland, we call this (not surprisingly) big data analytics, referring to data mining, online transaction analysis, historical trending – basically, any problem that can be solved by comparing and cross-referencing structured information. These applications involve big databases and are pushing the popularity of the open source MapReduce solution, Hadoop.

Big data archive is the other main application, one that involves large (often enormous) files and typically large numbers of those files. The signature use case is in the media and entertainment vertical industry, where users process these large files, often in sequential workflows, to produce motion pictures and special effects — and then save everything. Again, storage is often the bottleneck as companies need these large files as quickly as possible to support our 24-hour information cycle. Or users need to perform their processing step when the previous user is finished — a user who may be located in another city or country — taxing networks as well as storage systems.

For VARs, big data is yet another buzzword that vendors are attaching their products to in an attempt to create a perceived need in the mind of IT users. As usual, VARs must be ready to clarify the concept for those users, who by this time have been told that everything is big data and every vendor has a solution for it. When you do run into a real big data application, it can be a big engagement with a big PO. There are several technologies that I’ve covered in this blog over the past several months that have viable big data solutions. One is the subject of this white paper on building a storage infrastructure for big data archives. Stay tuned to future posts for more big data-related solutions and more ideas about how to leverage all the attention it’s getting.

Follow me on Twitter: EricSSwiss

A friend of mine, the CIO for a Colorado state agency, recounted a near crisis the agency had with its Exchange environment a while back. Basically, the infrastructure he inherited was running on an old iSCSI array that failed — and that wasn’t implemented with enough redundancy in the first place or properly backed up. The team had some sleepless nights and weekends but eventually got the system stabilized and then moved onto another storage infrastructure. The thing that got my attention when he was telling this story was how the agency brought in a team of consultants to straighten out the mess and handle the redesign effort. The agency didn’t have the in-house people to do it — or weren’t confident in its current skill sets. I know that engagement wasn’t cheap, especially given the fix it was in — like trying to negotiate with a tow truck driver when your car is broken down in the middle of nowhere. Maybe it’s time for VARs and MSPs to talk more about professional services again?

Ten years ago we routinely got 25 points of margin on integration deals, and often 25% of that total was professional services (PS). Today, I know both those numbers are typically less than half of that and that customers routinely gut PS on the statement of work in an effort to bring the sticker price down. Then they end up trying to get the VAR to come back after the sale or call tech support to “fix” the things they were unwilling to pay for in the original implementation. So where’s the rationale for trying to sell PS in this day and age?

What if one of the VARs my CIO friend deals with had made him aware that the company also does Exchange engagements? It doesn’t mean the VAR would have gotten this deal, but it would certainly have been in a position to offer a competitive rate on PS hours, since it would have been in line to get the ensuing infrastructure upgrade. The VAR may even have been able to structure the contract with incentives to buy gear related to this project.

The point is that VARs and MSPs have talent that customers need in this environment of flat budgets and thin IT staffs. Ten years of IT devaluation have left your calling base with an expertise deficit, one that you can fill. Unfortunately, most VARs these days can barely keep up with their own engineering workload, from pre-sales work to installation and post-sales support. So it will probably take some investment in training or head count, but you may find that your vendors are willing to chip in. They too are getting squeezed by their customers for technical help and are hungry for VARs who are willing to step up and provide more than order fulfillment.

Follow me on Twitter: EricSSwiss

Big data initiatives are complex, expensive and often misdirected. Just being able to provide a clear, cogent definition of what big data is may be all that’s needed to satisfy a customer’s need to know and allow you to score some points with the customer in the process. Helping IT contacts understand more about big data technologies may enable them to make an intelligent case to their management about how to approach a big data project — or whether to take it on at all.

Surveys frequently mention the challenges with big data, things even more fundamental than how to store and process large amounts of data effectively. One of the main challenges is the lack of relevant expertise. Data scientists and IT professionals who understand complex analytics have new skill sets that everyone’s looking for, as companies try to figure out how to pull business value out of the mountains of data that are available.

Similarly, setting expectations with management, something that’s always been an IT practitioner’s survival skill, is even more important in a big data discussion, given the potential scope these projects can take. At some point soon, your customer may get called into the CEO’s office to explain how he would do a big data project and capture those competitive advantages through complex analytics that the CEO has been reading about. That’s when the information he got from you will be greatly appreciated, especially if it keeps him from getting one of those “time sink” assignments researching a project the company has no business pursuing. After all, enabling your IT clients to look good with their bosses and better manage their expectations can really help your standing as a trusted advisor.

So learn about big data and prepare yourself for a big data discussion. It may get you into a meeting with management contacts you don’t currently have and help you build your value with the ones you do. And of course, if a project does come down the road, you’ll most likely get a chance to participate.

Follow me on Twitter: EricSSwiss

NAS cache

For file storage environments that need better performance than is available from existing NAS systems, a network-based caching appliance like CacheIQ can be an appealing solution. These appliances sit in-line, between existing NAS systems and application servers, caching frequently accessed files. Most contain a combination of SSD and DRAM memory and provide a nondisruptive implementation of solid-state performance in an existing storage environment. Also, since they can support multiple NAS devices, the cost of the SSD purchase can be spread out across more storage systems.

SAN cache

For high-performance block storage environments, like online databases and SAN-attached virtualization implementations, block-based caching appliances like those from GridIron Systems are also available. These solutions contain DRAM and SSD flash capacity and connect to a Fibre Channel SAN, servicing multiple storage arrays. Block-based caching solutions install on the network and start improving performance, transparently. For databases, this means no changes to application code and no ongoing administration tasks.

Network-based caching appliances are designed for the performance characteristics of solid-state storage and aren’t limited by a legacy architecture that was built for HDDs. They can provide a transparent performance boost to existing SAN or NAS environments that doesn’t require the management of an SSD tier. Since they can front-end storage systems from different manufacturers, they allow SSD costs to be spread across multiple storage systems and give customers a way to keep using existing disk arrays.

For VARs, network-based caching appliances can be an excellent option for an account that needs more performance for critical applications but is tired of paying “minibar prices” for storage upgrades to their existing Tier 1 arrays, upgrades that may not meet their expectations anyway. These solutions can boost performance without complex storage tiering and enable users to keep running existing storage systems.

Follow me on Twitter: EricSSwiss

Flash is getting more affordable but is still several times the cost per gigabyte of hard disk drives (HDDs). Because of the cost disparity, it’s often used to augment HDD performance: The more performance-critical data sets are placed on flash, sometimes temporarily, to take advantage of its orders-of-magnitude better performance, especially IOPS.

Read caching involves placing a copy of the most frequently used data objects into SSD to speed access times by applications or users. When data’s changing, it’s more complicated (write caching) since the primary data set must be kept updated and writes must be protected against system failures until they’re committed to nonvolatile storage. But there are algorithms that can do this too, as part of the OS or application or as a part of a PCIe card that houses the flash storage itself.

Caching can be the simplest to implement, since it leaves the primary copy of data intact on existing disk storage, often operating transparently to the application. It can also be implemented with a relatively small amount of SSD capacity, since data can be moved into and out of the cache area rapidly.

Tiering is like caching except it involves moving an entire application or data set into cache and then copying back to primary storage when the period of high activity is over. For this reason, tiering typically requires more SSD capacity than caching and may involve configuration changes to the application.

Server-side flash implementations can be done with PCIe flash devices, which can have up to a terabyte or more of flash capacity and may include caching software as well. Flash can also be in SAS or SATA drive form-factor packages, which plug into 3.5-inch, 2.5-inch or 1.8-inch drive slots. There are also SSDs that plug into an empty DDR3 memory slot on the motherboard and connect via a SATA cable.

Server-side flash is dedicated to the server it’s installed in, meaning less flash capacity is required than with array- or network-based flash devices and implementation is simpler than in those shared storage scenarios. Although flash capacity is significantly more expensive than HDDs, SSDs caching or tiering can actually provide a lower-cost alternative, with better performance.

For VARs, server-side SSD implementation can be an ideal way to break into a new account or capture new business in an existing account that’s currently going to an array vendor. Whether implemented as a cache or tier or just a high-performance storage area, server-side flash can provide an immediate solution for a slow application.  For other use cases, an all-flash array or flash appliance may be a better alternative. We’ll look at those in another post.

Follow me on Twitter: EricSSwiss

With flash, writes are made to an entire cell of memory; flash devices can’t overwrite a few hundred bytes like a hard disk drive can. When a flash device is new (referred to as the “fresh out of the box,” or “FOB,” state), this fact is transparent since each new write goes to an already-empty flash cell. But after it’s been filled to capacity, in which case all the cells have been written to once, it’s a different story. This condition (known as “steady state”) requires that before each new write occurs, the controller must erase a cell to make room for it. In this process, called “garbage collection,” the controller copies segments of data that need to be saved and then erases the entire cell.

The result of this process is that steady-state performance is much slower than FOB, although it’s still orders of magnitude better than hard drives. But when two flash devices are being tested, it’s imperative that the steady-state condition be established, since this is where the device will operate for most of its life. Aside from impacting performance, this extra copying process also increases the number of write cycles or “program/erase” (P/E) cycles the device experiences, which impacts endurance.

Originally designed for consumer use in memory sticks and cameras, etc. (by Toshiba, we learned on a briefing call recently), there was no need for flash to be written and rewritten to a large number of times. But when it started being used in the enterprise space, flash endurance became an issue. Enterprise applications, especially the ones that are performance-critical enough to justify the expense of flash storage, generate a lot of read/write activity. Database transaction logs and caching, for example, generate a lot of IOPS, the performance metric most often associated with flash applications.

Flash designers have addressed this problem in a number of ways, but it’s imperative that users (and their VARs) understand how much write traffic they expect to generate on a daily basis. Then they can compare that with a standard endurance metric called “total bytes written” (TBW) to calculate how long that flash device will last in their environment.

Ask your flash vendors about their products’ performance and endurance. See how much they know about these numbers and what they mean and find out if the products you’re selling or looking to sell are going to make your customers happy in the long term. You can also set customers’ expectations around flash as a technology and demonstrate your value as a trusted storage advisor in the process.

We’ll talk more about flash in an upcoming blog. In the meantime, here are some video links on Storage Switzerland about flash performance and endurance.

Follow me on Twitter: EricSSwiss

This is one of those “the devil’s in the details” situations. In theory, much of the infrastructure that companies rely on VARs to provide could be moved to the cloud, especially in the midmarket. But the cloud won’t instantly replace customers’ data centers or make them models of efficiency overnight.

For now, most cloud storage and compute implementations in the midmarket are augmenting existing infrastructure, not replacing it wholesale. IT organizations tend to be conservative, so the rip-and-replace approach is usually the least favorite option; incrementalism is the preferred method. This often takes the form of adding capacity or off-site protection on the back end or supporting a new application altogether from the cloud. In keeping with this augmentation strategy, Zetta has a very simple way to implement a cloud backup solution or improve the one already in place.

ZettaMirror is a lightweight software agent that loads onto file servers or computers and simply replicates the files in the designated “virtual volume” to Zetta’s cloud. This provides an off-site DR capability for users who are happy with their data protection systems, or a simple backup solution where none currently exists. When users need to restore a file, they can browse the available snapshots made for each file (configurable) and return just the needed versions from the cloud. Zetta also has Tier 1 data centers on each coast to provide redundant copies of files if required. Storage Switzerland is currently doing an extended review of ZettaMirror.

Follow me on Twitter: EricSSwiss

It just occurred to me that we never grabbed that term and used it on other proposals. Looking back, we should have put a line item on every quote for “nonrecurring engineering” — instead of “integration” or “installation” or the least descriptive of all, “professional services.” It’s all about providing value, which includes making sure your customer understands what that value is and how often you’re providing it.

It’s essential to remind your customers, early and often, about the things you’re doing for them. Activities like on-site assessments, setting up demos or having your sales engineers stop by the data center and tweak a configuration for free are all things VARs do that most customers appreciate. But there are other things VARs do that they should probably make a bigger deal about, like the effort involved with vetting new technologies, learning them and then coming by with an SE to whiteboard them for the customer, when there’s no RFP involved. These are things that save busy IT folks time and thereby provide them value.

Another idea is to ask your customers what you do that they value the most. They may say “provide a non-biased source of information” or “put together solutions from multiple vendors,” both of which are certainly valid. But there are probably things you do that they really like that you don’t realize. Maybe they appreciate the fact that your salespeople seem to know their stuff and don’t have to call an SE to answer every question. There may even be a few things another VAR does to provide value that you could do as well.

Follow me on Twitter: EricSSwiss

Users download the Symform client and specify which folder they want to upload to the cloud (their “sync” folder). Then they designate a “contribution” folder with the storage capacity they want to provide to the Symform cloud. The Symform client parses the data in its sync folder, encrypts it and disperses it out to the rest of the Symform network. The contribution folder is managed by the Symform cloud and contains pieces of data from the other users throughout the network.

VARs and MSPs that are reselling the Symform service can set up the contribution folder from their own data centers and apply this capacity to their customers’ accounts. This enables them to resell the Symform cloud service without involving their customers in setting up contribution space, etc. They can sell the benefits of data encryption, geo-dispersal and an enterprise cloud service while setting their own prices for the capacity their customers put into the cloud.

Symform’s cost structure is significantly lower than other cloud services, by virtue of its ability to use lower-cost disk capacity that’s already available locally in its clients’ environments — or that’s provided by its partners. Even when procured specifically for the purpose of contributing to the Symform cloud, this capacity is still less expensive on a per-gigabyte basis than the enterprise-caliber disk arrays that regular cloud storage services must acquire, implement and maintain. And, there’s another feature Symform recently announced that resellers can leverage to address a fundamental problem that cloud backup services have.

In its original process, data that’s placed into the sync folder is encrypted, parsed into multiple pieces and then dispersed throughout the Symform cloud. The nodes in this cloud are the contribution folders that all users provide, including, theoretically, the same user that owns that data. In this original scenario, data’s not protected until this entire encryption and dispersal process is complete, which can take quite awhile for smaller users with modest bandwidth.

Symform’s Turboseeding feature separates these two processes and does an export process on the data in the sync folder; that process parses, encrypts and allows the data to be put directly into the contribution folder, which is logically part of the Symform cloud. Or, in the scenario described above, it can be copied to a portable disk drive for transport back to the VAR’s or MSP’s data center.

Users skip the entire initial upload process that makes traditional cloud storage a problem for many businesses. As soon as the Symform client completes the export process, the data is backed up. From that point on, the user is protected and can even start doing incremental backups, without waiting for the initial “cloud seeding” upload process to complete. When the VAR/MSP copies its customer’s data into the contribution folder and completes the upload to the Symform cloud, the customer’s data is available for restores.

Symform is available for users directly, without going through a reseller. But a VAR or MSP shouldn’t have any problem showing its value add by handling the contribution side of the process and providing an instant upload with the Turboseeding feature. Then it can leverage the economics of Symform’s shared cloud architecture to provide a cost-effective cloud backup/storage service with some good margin potential.

Follow me on Twitter: EricSSwiss

The CIO I spoke with said his biggest concern wasn’t really the BYOD movement; it was the security risk from the use of laptops outside the company — hacking or snooping at places like Starbucks and the potential for lost laptops. I had a big “Oh yeah” moment and was brought back into the real world, one I’d stopped thinking about a year or more ago. Shame on me for not focusing on the issues users have today and talking about solutions they can use. While making this mistake is a “my bad” situation for me, it’s much more serious for a VAR.

Being a year or two removed from what users are implementing is a mistake VARs can’t afford to make. They do need to bring in fresh ideas to show their clients, but they also have to keep their line cards stocked with a comprehensive set of solutions to the problems users have today. Taking the example of my discussion with the CIO above, data loss prevention (DLP) is a topic that probably got more ink from folks like me a year or two ago. It may never have scored particularly high in gee-whiz value, but it’s what he needs — and it’s what VARs should be showing him.

Other examples could be addressing performance issues on a SAN or a WAN or improving SAN troubleshooting. SSD caching and tiering are certainly collecting more than their share of attention, considering how much of those technologies users are actually buying as a percentage of their total IT spend. But there are some basic server-side SSD implementations that users should know about, like simply replacing a boot drive with an SSD.

The point is that while VARs do need to know about what’s coming up from a technology perspective, they must keep rooted in the “hear and now.” Customers want to hear about what they need now.

Follow me on Twitter: EricSSwiss