For many storage VARs, e-discovery was one of those promising technologies that generated a lot of interest, drove quite a few appointments but didn’t result in many new customers. A lot of products in the compliance arena seem to be like this, but log management may be different.
Historically, log monitoring has been a security effort, part of the intrusion detection process. Events from firewalls, for example, generate information about who’s accessing the system, which is captured in logs and compared with policies to determine if a threat condition exists. Security information and event management (SIEM) technology was developed to improve security effectiveness and meet some regulations around this external threat detection and alerting.
Regulations like HIPAA, Sarbanes Oxley and PCI, for example, involve the data captured in these logs as well. But, they’re also interested in data from the myriad other logs that are generated in the environment, from elements like switches, routers, applications, databases, operating systems, Web proxies, etc. These data contain information with a potential compliance impact and organizations are mandated to collect, analyze, report on and archive all log monitoring activities across their infrastructures. Similar to files that contain personal information, logs represent another type of sensitive data that organizations must manage.
Log management as a technology category in IT has grown out of the need to control this new group of sensitive information. For example, transactions involving credit cards generate a log that contains personal data that must be stored securely and reported on. Similar to the case of e-discovery, these data must be kept for years and made available for searching and retrieval, etc. Log management solutions provide the system to collect, store, maintain and report on information that’s collected in all these different logs.
Unlike e-discovery, log data isn’t just a concern if legal action arises, it’s a day-to-day requirement. Also, since logs are transactional, log data can consume significant amounts of storage. Even a small organization can generate tens of gigabytes per day, or terabytes per month. Some companies write their own solutions to store logs, others use multiple, point solutions, but all companies have to deal with log data. Compared with other compliance-related products, the need for comprehensive log management represents a potentially better opportunity for storage VARs.
Follow me on Twitter: EricSSwiss.