Incident Response

Things You Can Do To Help An Investigation, Part II

In a previous column, I talked about the importance of locking up a computer and not continuing to use it after it has been compromised, or the fraudster was fired.

This works in a lot of situations, but there’s also situations where it’s NOT the best thing to do. If you know a computer has been compromised by an external entity, the best things to do are:
1. leave it on,
2. don’t let anybody use it, and
3. call your experts in.

Why leave it on? There are things running in memory that won’t be captured if you shut it down. Remember that you lose everything that’s in RAM, as well as network connections and processes running. It’s critical information if you want to find out who is doing it, and how they’re doing it.

Don’t log into it to “see what you can find out.” In some cases, servers get hacked, and admins tend to log in to “fix it.” As I noted earlier, Sometimes they reboot the box to “clear it out.” There goes all your information, and very probably the ability to at least find out how it was done so that you don’t restore the box to the same “hackable” condition.

Don’t have experts you can call on, that you know are good? That means you’re suffering from the ostrich syndrome. The time to build relationships that can help in a crisis is not during the crisis. Do yourself a favor and at least research the mostly likely people you’ll need to get the job done.

Things NOT to Do When You’ve Been Hacked, Part II

I finally asked that deadly question: “What do your Incident Response Procedures say?” Whoops, there goes all the buddy-buddy geekiness: I have morphed into The Auditor Who Asks Questions.

“Umm, well, they pretty much say to do what we just did.” I notice the vagueness of the reply, but decide to let it pass, for the moment. They don’t really know what their procedures say they should do. Probably the procedures are too generic.

“OK. But what if he has jumped to this box from another box he compromised first? How would you know?” More pained and irritated looks coming my way. “By now, you won’t really be able to tell what happened unless you go to a backup and start analyzing whatever you can find for connection information. But that won’t necessarily give you rootkit information. If you’re lucky, you might see a netcat connection, but only if he hasn’t erased the Event Logs.”

“Even so,” I continue, knowing I am now excluded from the Kool Kids Klub, “If he has gotten your SAM database off the server, wouldn’t he know the administrator password? Is that password the same on every server?”

Turns out the password IS the same, and the Event Logs overwrite according to defaults. Now they can’t trust the server OR the administrator password. But I’m leaving, and besides, this isn’t an audit anyway, just some consulting.

So they left the server alone, because “There are all those websites on it, the users would scream and we’ll watch it carefully.” And never mind about passwords because “It’s a really tough one they’ll never crack.”

I wonder what will happen next, don’t you?

Things NOT To Do When You’ve Been Hacked, Part I

The problem with being a “geek” is that we truly love to tinker, to fix, to improve, to test….etc. So when you announce to a bunch of us that a website on the network has been broken into, there’s lots of leaping into action.

Which is exactly what you don’t want to do. At all.

While visiting a client to talk about network architecture, an engineer rushed into our room to announce that one of their websites had been hacked. We all hopped up and went out with him. (My lecture was boring, anyway.) I wanted to see what they were going to do, and if they were going to follow their own Intrusion Detection Policy. Plus, I was, like them, vastly interested.

Turns out it was a fairly generic attack, with the break-in artist simply using the website for cross-site scripting and redirection.

By the time we got there, two engineers had already been working on the web server, analyzing the code in the html, and checking other settings on the server. They took the web server offline, removed the offending code, looked at the event logs and brought it back up. All good, they said.

“Not really,” I said. “You do know that you can never trust this box again?”

“Not to be a party-pooper, but there’s no way of really knowing if a rootkit has been installed, is there? He could come back tomorrow.”

The four geeks looked pained. “What should we do?”

“Well, we can start with reformating the disk and reinstalling the OS.” I knew the minute I said that I was not going to be the most popular girl in the room. That sort of thing is awfully tedious and boring; no fun for geeks.

“But there’s ten other websites on this server!” Oops, this was going to be a LOT of work.

We segued briefly into the advantages of virtual machine backups, and then returned to the discussion of what to do.

I finally asked that deadly question: “What do your Incident Response Procedures say?”

Remember the Lowest Common Denominator

I recently attended a seminar at a well known southwestern school on building an Incident Response Team. During the discussion about Team membership, management oversight of the Team and related responsibilities, I noticed that the membership of the Team and the Oversight Committee was lacking some critical input.

An area often overlooked, especially when being developed by those in the Information Technology field, is the aspect of physical security. The campus police and the maintenance department were the two members lacking in this particular seminar. When I brought up this issue, it was dismissed with the equivalent of: “Oh, them.”

(They may never be getting into their offices again, or have decent air conditioning. And keys? forget it.)

Considering an “IT event” to be the only worthy event included in the IRT criteria for action is truly shortsighted. Physical events such as a string of burglaries on campus, flooding or water damage can have just as much impact on communications as a network outage. Not to mention the idea that those events would be a great shield for someone intent on attacking the network. If the IRT is unaware of these events, they become ineffective.

Not only that. Bringing physical security to the common IRT table is important for those folks, as well. They may be unaware of events in the IT world that would impact on securing the overall physical environment. Having all parties educate each provides a unified response, and that’s a much better incident response overall.