Compliance requirements have long since pushed organizations to adopt better log management. Like IT staff at public companies, where the Sarbanes-Oxley Act and log management go together like peanut butter and jelly, IT execs working in government have to retain, manage and store logs to comply with FISMA, the Federal Information Security Management Act.
Last month I moderated a panel on enterprise log management at the CA IT Government Expo in Washington, D.C. The two panelists were Jon Kim, security architect of GTSI Corp., and Joe Ford, CTO and vice president of professional services at Patriot Technologies.
Both men offered useful best practices and practical insight into the challenge of managing logs within the enterprise. When it came to distinguishing between log management and security event management (SEM), both men observed that the utility of SEM lay in monitoring for specific access issues. Identifying changes in access or usage patterns that could indicate an issue is a key component and capability of security information and event management systems. In this context, the importance of of keeping raw log data around for forensic analysis was clear.
Department of Defense (DoD) directives on log management are directly related to achieving better “situational awareness,” a much-used concept in the cybersecurity world. The DoD Information Technology Security Certification and Accreditation Process, or DITSCAP, includes groups of activities and tasks that must be performed over the lifecycle of any existing or upgraded DoD system that collects, stores, transmits or processes either unclassified or classified information.
Better automation and monitoring of logs has the potential to reveal issues across critical infrastructure, which is critical as threats emerge. Recently released Consensus Audit Guidelines specifically refer to the importance of tying identity to activity. Both panelists saw considerable maturation in this area of log management. The next challenge will be building better authentication into enterprise systems that more effectively interoperate with log management software. Security vendors are actively pursuing this goal.
The final question that the panel considered was the lack of clear guidance from the government on how long logs should be retained. Both men acknowledged the issue, referring to the National Institute for Standards and Technology (NIST) SP 800-53 document, which describes several controls related to log management, including the generation, review, protection and retention of audit records.
Both panelists suggested that best practices should be based on the risk profile for each organization and the potential relevancy of the information to audits. In the back and forth between the panel and the audience, it was also clear that considerable differences exist within the defense community on how much and how far back to retain.
Few network, security or compliance managers would dispute that log management formats and standards are in a state of flux. In fact, enterprise log management may be the least standardized area of IT. There are efforts under way to agree on common standards for logs, however. Earlier this year, the Open Group Security Forum updated its log standard, Distributed Audit Services, or XDAS. The Security Forum has also announced work on a new compliance standard, automated compliance expert markup language, or ACEML.
Standards aren’t the only challenge for enterprise log management. The sheer volume of log files generated across networks is a huge issue. As NIST offered in its SP 800-92 guidance on security and log management:
A fundamental problem with log management that occurs in many organizations is effectively balancing a limited quantity of log management resources with a continuous supply of log data. Log generation and storage can be complicated by several factors, including a high number of log sources; inconsistent log content, formats, and timestamps among sources; and increasingly large volumes of log data.
Best practices — and regulatory mandates — mean logs should be generated, archived and monitored regularly for insight into employee activity and to detect and prevent system outages and security breaches. Without effective enterprise log management and analysis, more organizations will be found noncompliant and remain at greater risk.