Communication

Communication Runs Through the Whole Process

When it comes to incident response, what I found is a huge part of the success of incident response is the communication level. Communicating with our end users, communicating with people within the company, communicating between the troubleshooters is all really important.

Communication is one of those things that's important throughout this whole process. Everything from prep - because we've got to train, or anytime we make changes we need to notify people - to the detection, to the analysis, to the containment, eradication and learning afterwards. Really, communication is a key part to incident response success.

Because of this, communication should be a very big focus when it comes to the incident response plan: outlining who we're going to communicate to, what we're going to communicate, when, how we're going to communicate. We need to figure all of that stuff out ahead of time.

One thing we need to communicate is when we declare that there is an incident. There's a lot of people that need to be brought on board. Well, how are we going to contact them? We need to discuss that ahead of time, to know what people's phone numbers are. Are we going to text them? Are we going to call them? Are we going to send them an email? We need to know all of those details.

When people are troubleshooting an issue, the communication between the different troubleshooters is really going to help make sure that we troubleshoot in an efficient manner, that people are looking at the right systems, that we're figuring out what's going on. So communication is a huge part of getting to the heart of what the issue is and how we're going to resolve the issue.

Communication and Reputation

One thing that I found is that communication really helps out with reputation. What I mean by that is that when you have a group of users that are expecting services to be up, and then they go there and it's not, it's very frustrating and you're losing reputation with that. If they don't hear back from you and don't understand what's going on, that reputation goes down much faster.

But what people understand is that sometimes things happen. If it happens too often that's a problem, but every once in a while they understand that systems go down, that things are not perfect. There is a bit of understanding by most people that not everything operates perfectly all the time. If you're communicating well during those times, they understand better and really feel like you are giving the attention that it needs to fix the issue. But if you don't communicate, a lot of people's minds start thinking the worst case scenario - that you're not even paying attention to it, that you're not even trying to do anything about it.

So communication really helps reinforce good reputation. If you are communicating well during these times where there are problems, you are going to be better off from a reputation standpoint.

Who You Need to Communicate To

If you're a typical business, there's all sorts of people within the company that may need to be communicated to. Maybe there's shareholders of the company, or the board of directors, or there are certain committees, or there's some sort of legal representation, or maybe there are some sort of things that we have to communicate. So there are different people within this organization that might need to be communicated to at different times.

There's also people outside the company. We have clients and vendors and partners. We need to analyze all of who would be communicated to during these times.

What to Communicate

We also need to figure out what to communicate. The key is to find out the right balance, not oversharing situations. One thing that can be problematic is if we say we've lost all your information and then later find out we didn't lose all your information. That could be problematic, that could cause a lot of panic. So what we need to do is we really need to establish what it is that we need to communicate, and at what times we need to communicate, to make sure that we can reinforce our best reputation and make sure that the right people are involved with this whole process.

Here's just a quick example. Maybe we're running some sort of e-commerce site where we have a bunch of users on our site and we see some signs of malfunction. We want to maybe just give customer support a heads up: hey, we're investigating some concerns, nothing too critical yet, we're just looking into this, so they know. And then maybe the system goes online, then we're going to definitely reach out to customer support and say, hey, you could get lots of customers complaining about our services right now. Or you reach out to the whole company and say, hey, we're experiencing some issues and we're trying to figure out what's going on. Then customers start reporting in, and then we are going to declare this is a critical incident and notify everybody, and when the next communication is going to be, and we're going to take communication to the next level at that point.

So what is it that we're communicating? During this process and outages, what we're going to do is communicate any kind of updates, time estimates. Time estimates are hard to do in these scenarios, but if we can give time estimates, people definitely appreciate that. We're going to communicate things like resolution and impact. Afterwards, we're going to figure out what we need to do from a legal perspective and do any kind of regulatory reporting, or we may even need to report things to law enforcement, depending on what type of incident it is, what type of issue it was.

And speaking of that reputation, we also want to get public relations involved. If you've got some sort of marketing department, or a PR firm that you're working with, or maybe an in-house PR department, then you want to make sure that they're involved so they know and understand what's communicated to the customers, what they're communicating to media, so that way they are taking on the role that they should be taking on of that public relations, and making sure that things are going smoothly from that perspective.

Juggling Multiple Roles

One of the difficulties that I noticed is communication and trying to juggle multiple roles. For instance, if you're working for a small company, maybe your incident response manager is one of your troubleshooters. Maybe it's the system administrator. Or in fact, maybe you have an IT person who is the customer support, is the incident response manager, and also the system administrator, database administrator and developer. You have one role that does all of these.

This could be very difficult, because how much time do they spend communicating, how much time do they spend troubleshooting? If they're trying to fix the issue, they can't communicate as well. If they're trying to communicate, they can't fix the issue. So it can be problematic.

Figuring out ways to get over that - things like maybe automation, or maybe there's somebody from another department that can come in and help out during these times with the communication - all helps with this. So just figuring out what those roles and communications might be, and how to distribute some of that load to overcome some of these difficulties.