Last month’s Amazon Web Services cloud outage sparked a lot of online discussion and debate over the viability of cloud services. According to published reports, an online dating company ditched AWS after massive storms caused power outages and knocked out service in one of Amazon’s U.S. East-1 Availability Zones June 29.
But Netflix – one of Amazon’s biggest cloud customers – said it remains “bullish on the cloud” despite the AWS outage. In a blog post Friday, Greg Orzell, software architect at Netflix and Ariel Tseitlin, director of cloud solutions at the company, wrote a post mortem of the outage, which they said was one of the most significant Netflix had experienced in over a year. The outage showed up things that both AWS and Netflix could do better, they wrote.
“Our own root-cause analysis uncovered some interesting findings, including an edge-case in our internal mid-tier load-balancing service,” they wrote. “This caused unhealthy instances to fail to deregister from the load balancer which black-holed a large amount of traffic into the unavailable zone. In addition, the network calls to the instances in the unavailable zone were hanging, rather than returning no route to host.”
Netflix is working to improve its resiliency and is working closely with Amazon on ways to improve the cloud provider’s systems, “focusing our efforts on eliminating single points of failure that can cause region-wide outage and isolating the failures of individual zones,” Orzell and Tseitlin wrote.
“While it’s easy and common to blame the cloud for outages because it’s outside of our control, we found that our overall availability over the past several years has steadily improved,” they wrote. “When we dig into the root causes of our biggest outages, we find that we can typically put in resiliency patterns to mitigate service disruption.”
Last summer, I attended a session at the Gartner Catalyst Conference 2011, on planning for resiliency in the cloud. Richard Jones, a managing vice president at Gartner, said the public cloud is a utility and utilities fail, making it critical that customers prepare for downtime. Enterprises often assume cloud services are reliable but they need to take responsibility for uptime, he said.
Seemed like sound advice to me. Other companies may want to look to Netflix for cues on planning for cloud resiliency.
DNSChanger infections have declined precipitously, but remaining systems could have Internet access turned off today.
It appears the Internet will not be thrown into turmoil as a result of the FBI shutting down the servers feeding systems containing DNSChanger malware.
The DNS Working Group, made up of a number of experts from security firms, DNS providers and the government, has been tracking infections. As of June 11, there were only about 69,000 DNSChanger infections in the United States and far fewer in other countries. The working group also estimated that globally there were approximately 303,000 systems containing the malware.
When the FBI arrested six Estonian nationals in November, charging them with running a sophisticated Internet fraud ring, investigators seized servers in data centers in Estonia, New York, and Chicago that were pointing victims to spoofed websites. The FBI estimated at the time that there were 500,000 infections in the U.S. and up to 4 million abroad.
With the news coverage aimed at consumers with little knowledge of the malware, it is very likely that the number of infections has drastically declined, although the working group hasn’t released updated figures. When the replacement DNS servers designed to avoid disruption are turned off today there won’t likely be any serious problems. It has still generated a number of hyped headlines including “Internet doomsday virus,” and “Internet blackout looms.” Let’s put this in context: There are still 2.5 million machines infected with Conficker.
The DNSChanger malware is a good example of the need for increased security vigilance on the part of average computer users. It can go a long way to reducing the number of serious incidents by disrupting the spread of malware. The working group has a great security protection Web page that leads computer users to additional information about phishing, antimalware and Windows 7 security features. The links lead to solid information from the U.S. Computer Emergency Readiness Team, the Carnegie Mellon Cylab Usable Privacy and Security Laboratory and the FBI. The advice is good, and is without the marketing spin designed to sell security software.
Another great resource that puts the DNSChanger problem into context is Canada’s Public Safety office, which published a document in November. The Canadian DNS Changer TDSS/Alureon/TidServ/TDL4 Malware Web page has been updated to help people determine if their systems have been infected and contains tools to help victims remove the infection.
Checking a system can be done by simply visiting a websiteor manually depending on your operating system.
Mobile device security threats are taking center stage as IT managers strive to protect and control these nimble creatures that contain company information and access the company network. But looking at the big picture of all IT security concerns, just how significant are specific types of mobile device threats? According to one expert, mobile botnets, at least, should not keep you awake at night.
Mobile botnets are created when an attacker infects a number of mobile devices with malicious software. The infected devices communicate with other mobile devices, thus spreading the infection and growing the botnet. The attacker’s goal, in theory, is to gain root control of the mobile devices in order to use their combined bandwidth and computing power for nefarious means.
In an interview with SearchSecurity.com News Director Rob Westervelt, Joe Stewart, director of malware research at Dell SecureWorks, provided his perspective on the relative importance of the mobile botnet threat. Because mobile networks don’t have as much bandwidth as broadband connections, Stewart said, mobile botnets are not likely to be very profitable for the botnet operator.
“I don’t think you can say at this time that someone will get a whole lot of value out of a mobile botnet,” Stewart said. “There are certain categories where it is useful, but as a DDoS botnet, it would probably be pretty abysmal.”
However, findings by Symantec Corp. suggest revenue for the mobile botnet “industry” may be on the rise. Writing in Symantec’s official blog in February, Symantec Security Response Engineer Cathal Mullaney noted the discovery of one particular mobile botnet that had the ability to use premium SMS scamming to generate millions of dollars a year.
Still, all indications suggest mobile botnets are a small niche in the overall threat landscape. Antimalware investments might be better spent in other areas right now, but be wary of a possible invasion of mobile botnets in the future as attackers prey on the relatively easy vulnerabilities of mobile platforms.
I’ve covered a lot on online bank fraud in the past – there seems to be no end to the increasingly sneaky techniques cybercriminals develop to siphon money out of victims’ bank accounts. This week, McAfee Inc. and Guardian Analytics Inc. released the findings of their investigation into a global fraud ring that takes the old techniques up a notch.
In their report, “Dissecting Operation High Roller” (.pdf), the companies report cybercriminals — building on older Zeus and SpyEye tactics – are targeting high-balance bank accounts belonging to businesses and individuals. Unlike past online bank fraud attacks using Zeus and SpyEye, though, these new attacks use server-side components and heavy automation. According to the report, the attacks have been mostly in Europe, but are now spreading to the U.S.
Criminals have tried to steal more than $78 million in fraudulent transfers from at least 60 financial institutions, including large global banks, credit unions and regional banks, the report said.
In a blog post, Dave Marcus, director of advanced research and threat intelligence at McAfee, noted that by shifting from traditional man-in-the-browser attacks on a victim’s PC to server-side automation attacks, criminals have moved from multipurpose botnet servers to cloud-based servers that are purpose-built and dedicated to processing fraudulent transactions. The strategy, he said, helps criminals move faster and avoid detection.
The report describes attacks in the U.S. and The Netherlands as using a server located with an ISP with “crime-friendly usage policies” and moved frequently to avoid discovery.
All pretty unsettling stuff, suffice to say. And, according to the report, financial institutions can expect even more automated and creative forms of fraud in the future.
As the opening day of the 2012 Olympic Games nears, IT teams in the U.K. are busy expanding their companies’ security policies and reviewing their security contingency plans. They are preparing for 17 days of Games, which will surely produce crowded transportation systems, overloaded Internet connections, and employees whose attention may be diverted by swimming relays and equestrian events.
The Olympics provide a good opportunity for companies in the U.S. and around the world to review their security policies and plans, too. Security pros can watch how their peers in the U.K. handle the pressures and disruptions caused by the Olympics, and consider how they would handle such an event if it occurred in their city.
Security contingency plans, which are similar to disaster recovery plans or business continuity plans, lay out the steps IT should take as soon as a disruptive event occurs. The idea is to make important decisions in advance, and have the necessary resources already in place, so the team can react quickly to maintain the security of their company’s data and other IT assets. Yet, according to our application security expert Michael Cobb, many companies’ security contingency plans are either unrealistic or woefully out-of-date.
Could your company continue operating securely in a chaotic environment — whether that chaos is caused by a scheduled event, such as the Olympics, or by an unplanned natural event? The 2012 Olympics serve as a reminder for all firms to review and revise their security contingency plans in light of current concerns and resources.
The relatively quiet summer months may be a good time to set up components of your security contingency plan. One of the most important components to handle in advance is widespread telecommuting. During a major event, more employees may have to work from home. You can prepare now by having all employees sign a remote working policy agreement, test the security of the Internet connection in their home, and receive training on topics, such as securely filing sensitive documents from their home office.
The Olympics also provide a hook for continued security awareness training. The IT department could send out an email educating users about Olympic ticket scams, providing a helpful lesson for any too-good-to-be-true email offer. Or the IT department could run a summertime security contest, posting a short information security quiz on the company’s Intranet and awarding gold, silver and bronze medals to the employees or departments who score highest on the quiz.
Even if the Olympics are not held in the U.S. until at least 2024, there is bound to be a significant event that will affect your company’s security posture in the near future. Prepare and practice now so your security team can execute flawlessly and take home the gold.
Security awareness training often teaches the importance of password length and password complexity, but these best practices, as it turns out, may be creating a false sense of security. Even worse, users who cooperate and create long, complex passwords may feel betrayed when the organizations they trusted prove fallible and their passwords are hacked.
The recent LinkedIn hacking incident, in which 6.4 million LinkedIn passwords were stolen (or possibly leaked), demonstrated the strength of a user’s password is no defense when an Internet application provider is attacked. Even if each LinkedIn password was as long and complex as possible, it wouldn’t have mattered; the Russian hackers still found the hashed LinkedIn passwords and posted them for all to see.
According to some analysts reviewing the LinkedIn breach, the social networking site had failed to protect users’ passwords with a strong hashing algorithm. That’s where the sense of betrayal comes in. If users are doing their part by using strong passwords, they should be able to trust the application provider to take strong precautions, too.
The situation spurred LinkedIn to take stronger precautions now. In a blog post, LinkedIn said it would use better hashing and salting to protect its account databases in the future.
Organizations can learn from LinkedIn’s public mea culpa. If your IT staff has been lecturing users on strong passwords, but your organization’s passwords are stolen, how will your users react? After years of building trust between IT and users, an incident like this could destroy the relationship in one day.
The LinkedIn incident is a reminder of the need to properly balance responsibility for secure access management among users and IT. Yes, user training is important, but IT security teams must go the extra mile to protect account credentials and prove themselves worthy of users’ trust.
Wednesday’s Cornerstones of Trust Conference featured an interesting CSO discussion of some of the hottest topics infosecurity pros are dealing with today, including the BYOD trend, cloud computing and big data security. The annual conference, held in Foster City, Calif., is sponsored by ISSA’s Silicon Valley and San Francisco chapters, and San Francisco Bay Area InfraGard.
Mobile, cloud and BYOD are all part of an overarching trend towards consumerization of IT that’s driving demand for convenient, easy access to corporate data, said Preston Wood, CSO at Zions Bancorporation, a Salt Lake City-based bank holding company. “We need to find a way to enable that and not be a roadblock,” he said.
At Cisco Systems, the mobile trend is far from new, said Steve Martino, a Cisco vice president in charge of information security for the networking giant. Thirty percent of the workforce has more than two mobile devices. “If we try to prevent it, they’ll find ways around it,” he said.
Instead, organizations should consider flexible mobile policies that permit network access based on the user, device and location, Martino said. For example, a user with a phone that doesn’t have mobile device management (MDM) software may get access to some services but not others.
With cloud computing, information security’s historic reliance on preventative controls won’t work so well, Wood said. The cloud trend presents the opportunity to focus more on detective controls of rapid response and risk mitigation. Each organization will have a different risk appetite and some aspects of the business will still require preventative controls. “There’s no one-size-fits-all,” Wood said. “You need to ask the business that risk question.”
On the topic of big data security – using big data techniques for security analytics — Wood suggested organizations can get started on that path by digging into data they already have on hand, such as firewall or IDS logs. Administrators often don’t look back to see if firewall policies are still working – that might be an area to explore, he said. The approach of mining data to obtain more security builds on itself.
“Start with what you already have,” Wood said. “And start by asking some innovative questions of that data.”
Earlier in the day, Wood presented a keynote on big data and security analytics, which unfortunately I missed, but I did cover his presentation at RSA Conference 2012, as did many other reporters. His RSA presentation was widely covered and justly so. He’s put into practice what others are only talking about at a conceptual level. At RSA, he and others from Zions detailed how the company harnessed information from its disparate security data sources by developing Hadoop-based security data warehouse. Using big data techniques enabled the company to speed forensics investigations, improve fraud detection and overall security, they said.
On Wednesday, Wood also offered some career advice to security pros: Don’t limit yourself to the “echo chamber of security.” Security pros should try to learn about other disciplines; big data security, for example, offers the opportunity to reach out to business units that have experience with analytics, he said.
At Cisco, employees are rotated, for example, from security to IT or from a business unit into security, Martino said. That practice helps the security organization understand the pain points throughout the business, he said. The company also has created security advocates in other parts of the business, which gets others involved in security.
Wood also urged attendees to spend more time on strategy. A lot of security organizations find themselves fighting fires all the time instead of looking at the big picture, he said. Security teams need people with the skills to deal with daily operations but who can also look ahead and strategize.
Unemployment is at 0% for information security professionals! This good news was reported this spring in CompTIA’s 9th annual Information Security Trends report. The report cited U.S. Bureau of Labor Statistics (BLS) research conducted in the spring of 2011, which also noted the unemployment rate at just under 4% for the IT industry overall. Clearly, skilled security professionals should have no trouble getting information technology security jobs right now.
But companies are having trouble filling those jobs. According to CompTIA’s survey of 500 IT and business executives in the U.S., conducted at the end of 2011, 40% of companies are having difficulty hiring IT security specialists.
During a recent conversation with Todd Thibodeaux, president and CEO of CompTIA, I asked him why companies are having hiring problems, and I expected his answer would relate to the need for more CompTIA certifications. Or perhaps he’d say companies can’t pay enough to hire the talent they need. But Thibodeaux’s response brought up another perspective on the hiring challenge. He believes organizations are having trouble hiring IT security pros in the U.S. partly because of depressed housing values.
“The challenge is recruiting within physical regions,” Thibodeaux said. “Organizations don’t want to outsource their security, and they certainly don’t want to off-shore their security. So they need to hire locally.”
Yet with many IT professionals’ homes underwater with their mortgages right now, would-be employees are not able to move to take new jobs. So even though hiring organizations are willing to pay good salaries, they are largely at the mercy of larger economic forces beyond their control.
This phenomenon is more noticeable in some parts of the country, Thibodeaux said. Areas with high concentrations of technology companies are fortunate enough to have a larger pool of IT professionals from which to hire. But for companies not located in high-tech regions, it appears hiring has stalled. Companies and employees alike are waiting for home values to rise so people can move to fill IT security job gaps.
Is the answer simply to wait out the housing market? Thibodeaux believes a better answer may lie in college education. “Many colleges want to teach, not train,” Thibodeaux said. “But companies need people coming out of college who have been trained in technical skills.”
Perhaps this unusual situation of low unemployment in IT security combined with low home values will motivate some U.S. colleges to beef up their IT security courses with more hands-on training. Sure, that will take time — at least four years if incoming freshmen start now. But with home values inching back up slowly, those four years may turn out to be the quicker fix.
Security experts have warned about the potential problems caused by military cyberstrikes. Experts say cyberwarfare is difficult to plan and worse, it puts innocent people at risk.
Stuxnet was part of a secret joint U.S.-Israeli cyberattack operation which began with approval by the Bush Administration and continued with the nod from the Obama White House, according to a detailed account of the attack written by David Sanger in a report published today in the New York Times.
To put the pieces of the Stuxnet puzzle together, Sanger conducted interviews with unnamed sources involved with the Stuxnet operation dubbed “Olympic Games.” While it confirms a lot of speculation about the nation-states behind the Stuxnet worm, it also raises a lot of questions about cyberwarfare and its use by a sitting president. Should members of Congress have been notified of the operation? Were any U.S. citizens put at risk?
Even well planned military cyberstrikes go wrong
A 2009 study by the nonprofit research firm RAND Corp. urged the United States not to invest in offensive cyberweapons. It is too difficult to predict the outcome of an attack, making strategic planning a guessing game, according to the report’s author, Martin C. Libicki. “Predicting what an attack can do requires knowing how the system and its operators will respond to signs of dysfunction and knowing the behavior of processes and systems associated with the system being attacked,” Libicki wrote. Indeed, according to the Times story, Stuxnet clearly caused some disruption, but it was anyone’s guess as to how far it set back Iran’s nuclear program.
Even worse, Sanger’s account of the operation detailed a major coding error that enabled the offensive malware to escape into the wild. This led to its detection and analysis by antimalware vendors. Indeed there were facilities in the United States using the Siemens systems that the worm could have sought out. While the threat was minimal – Stuxnet still would have to get through the buffer zone isolating a facility from the Internet – those quoted in Sanger’s story said it was easy to get through the Iranian facility’s buffer zone using a simple thumb drive. I’ve heard of penetration testers using this trick to great success: dropping thumb drives in areas throughout a targeted organization to see if any curious employees would insert the device into their computer. “It turns out there is always an idiot around who doesn’t think much about the thumb drive in their hand,” according to an unnamed official referring to how Stuxnet was planted at the underground uranium enrichment site in Natanz, Iran.
If that’s the case then the operation certainly could have put U.S. citizens at risk right here on our own soil. It also has the potential to fan the flames of retaliation or similar offensive cyberwarefare operations from our adversaries. We’ve already encountered reports that government agencies and even critical infrastructure facilities, such as power plants have been penetrated in some way.
Network security luminary Marcus Ranum, CSO of Tenable Network Security, told SearchSecurity about his concern over militarized cyberspace and even outlined the problem caused by the Stuxnet-like strikes.
Critical infrastructure protection
I wrote about a 2010 report by the Center for Strategic and International Studies (CSIS), which consisted of a global survey of more than 600 IT pros at critical infrastructure facilities. The main finding was that systems that run power plants, manage the distribution of hazardous chemicals and help monitor water treatment plants are in a dire need of stronger safeguards. The survey found that those facilities are under a constant barrage of attacks. A U.S.-China Economic Review Commission report last October cited a significant attack targeting U.S. Satellites. The examples go on and on.
But the problem goes beyond the potential threat to power plants and oil and chemical refineries. Earlier this year researchers demonstrated a theoretical attack targeting the systems that control the locking mechanisms at a prison. Imagine the chaos that would cause if cybercriminals were to target the prison system.
There is plenty of recognition of the seriousness of the problem, but very little transparency of where the nation stands on protecting critical assets, said Andy Purdy, chief cybersecurity strategist at CSC, and a member of the team that developed the U.S. National Strategy to Secure Cyberspace in 2003. In an interview I had with Purdy at the 2012 RSA Conference, Purdy cited some progress, but admitted that the lack of transparency leaves very little information for authorities to track the progress the nation is making in protecting critical systems. Purdy cited substantial federal funding being invested into SCADA system security, the progress of the Industrial Control Systems CERT and several plans and reports outlining the role of the public and private sector in protecting critical systems, digital identities for Internet users and the role ISPs should play in controlling customers with compromised systems.
Perhaps security luminary Dan Geer is thinking ahead to disaster recovery after a cyberstrike. He speaks incessantly at security conferences and summits about the need for system redundancy and manual processes to help lessen the disruption and chaos when Internet connected systems fail. Not only do we need redundant systems and manual processes, but we need skilled people who know how they function, Geer says.
Stuxnet details conclusion
The details about the planning operation behind Stuxnet should be a reminder that military action, whether physical or digital, needs to be thoroughly vetted or else innocent citizens could be inadvertently put at risk. It should be a call to action for stricter oversight of the security of critical infrastructure both publicly or privately owned. It’s amazing to me that despite all of the increased rhetoric about better protecting the nation’s critical infrastructure there has been very little evidence of progress. Just words.
After working hard to create sound security policies, it’s easy for enterprise information security managers to be dismayed when users ignore the rules and knowingly bypass security controls. When those rule-breakers are executives, it feels like salt on the wound. After all, who should understand the importance of protecting an organization’s assets better than its top executives? Yet, a survey at Infosecurity Europe revealed that, in 43% of organizations, senior managers and even the board of directors do not follow their organizations’ security policies and procedures.
The survey was conducted last month by security consulting firm Cryptzone Group. They asked 300 IT professionals who within their organizations is least likely to follow security policies and procedures. According to the Cryptzone report, Perceptions of security awareness (.pdf), 20% said senior managers are least likely to follow the rules, and 23% pointed their finger directly at the CEO or CTO.
The Cryptzone report didn’t dig into the reasons behind these perturbing findings, but I’d venture there are five primary reasons why executives disobey corporate security policies. (You’ll either laugh or cry about the last one.)
1. They are discreetly excused from taking security training programs;
2. They do not agree wholeheartedly with the security policy;
3. They believe the risks they are taking aren’t all that bad;
4. They are in a hurry;
5. They think IT will take care of things if something (like a data breach) occurs.
The antidote for all these reasons can, of course, be found in corporate security training. But because senior managers probably can’t or won’t take time out of their workdays to attend more training (see reason #4), security pros will have to keep finding creative ways to get the message out. Multimedia playing in the office kitchen, occasional text reminders sent to managers’ phones, and other friendly methods of interjecting bits of the security policy into managers’ minds must be a never-ending process in every organization.