Data Masking and Obfuscation

Two Scenarios

There's a couple of terms, data masking and data obfuscation, where the idea is we take data and we make it really unusable or unreadable.

We're going to come up with two scenarios. For both scenarios we're going to say that this is production data. We have people's names, we have their date of birth, we've got their address. This is information that needs to be super secure — this is personally identifiable information, so we need to be secure with that. They also have some survey information, the date they took it, some notes on it. So we've got information that we have within the system.

For scenario number one, we want to incorporate this data into our testing and development environment. That is, we have a whole other environment that we do testing in, that we have quite a few people in that are testing, different developers, maybe we have some partners in there. We want to incorporate this data — we know this is known good data that our customers have, and we want to incorporate that into the environment — but we need to take out these sensitive pieces of data within here. So scenario number one is, we need to remove this sensitive data so we can incorporate it into our testing environment.

Scenario number two is, maybe these customers are no longer customers, and we have data that we want to keep over here, but we can't keep the more sensitive data.

Now we have several different options that we can do for removing the sensitive data for these two scenarios, but keeping the data that we want to keep.

The Terms

When it comes to this selective altering or deleting of sensitive data, we have data masking and data obfuscation, anonymization, and de-identification. A little later we're going to talk about masking and obfuscation and what those two terms mean by themselves, but when it comes to data masking and data obfuscation, we're going to say the term is really essentially the same, because many sources say that it's the same thing out there.

So what does it mean? Modifying sensitive data in a way that it is of no or little value to unauthorized intruders. What we're doing is we essentially have rendered this data useless for others; however, it's still useful for us. That's what data masking and data obfuscation is, and we use it for scenarios like when we're trying to make production data into something we can use in our testing environment, or we use it in the cases where we want to remove certain pieces of information because we need to utilize the information within the system in other ways.

De-identification

De-identification I'd say is a type of data masking where we're targeting certain information specifically — we're targeting personally identifiable information. Within our data set we're targeting that personally identifiable information and we're de-identifying it.

In one of our prior scenarios here, Alex is no longer a customer of ours and needs to be removed from this database. So we identify the name, date of birth and address, and we somehow mask this, or somehow change it, or we do something to it, so it's not tied to Alex anymore. We may do this because we still have some really valuable information that we want to store within our system, so we take away anything that identifies this person.

Anonymization

Anonymization is essentially the same thing, except for taking things a step further. We have this PII information that we would remove with de-identification; anonymization would take a look at the rest of this data and start questioning, is there any way that you could reverse engineer this information to get back to a person?

For instance, there could be the date that they logged in and did this particular action, which maybe is the survey, when they took this survey. Is there anything about the survey that they took that would identify them? Or are there certain notes on the way that they've spoken, or the terminology they used, that could reverse engineer to figure out the people that were involved? A lot of times it's some sort of combination of information that's in there that could somehow make it back to identifying a specific person.

So it's taking the extra steps beyond just a simple de-identification — taking the extra steps to make sure that there's no association of the data with the person.

Methods

Data masking or data obfuscation is how we would carry out de-identification or anonymization. That is, we can use some sort of method like substitution, we could use shuffling, number and date variance, we could do encryption or hashing, we could do a null or deletion, or we could do masking.

Substitution and Shuffling

For an example of substitution: maybe I had a database of names, of 10,000 names, and all I do is select one of the names and replace Alex for one of those names, and Bev for another one, and Zane for another one. So I just do a replacing, or a substitution.

Another way I could obfuscate these is by shuffling. Shuffling is just moving something around, so for instance I could move the names around. Maybe I move the names into different spots, and now this essentially makes most of this information null and void, because they're not lined up with their date of birth or address. I probably want to shuffle some of this information around as well.

Another thing I could do is number and date variance. In this example right here, maybe I shuffle the date of birth so that they're different — maybe it's plus or minus maybe 10 years or something. And maybe the address, I also shuffled the numbers in the address and it's plus or minus 10% within the address. So I shuffle all these numbers and these dates around so they're different, and I can just do some sort of random generator to generate what these dates and what these addresses would be.

Hashing

I could also use hashing. I could take some of these values, for instance name, and then put it into a hash and get a hash that I put in here instead. Now the data is still there.

I'd always be worried about if the hash algorithm got broke — that would be a concern. But the other biggest concern that I would have is, one of the reasons why I choose one of the substitution methods, or I do some sort of shuffling method, is because the data remains very intact for the most part. It's very valid information that's in there, except that it's just anonymized, it's de-identified, it doesn't tie back to individuals. So we have really true data that's true to our users and what kind of data we have in the systems.

When you do something like hashing, and some of the other ones I'm going to talk about as well, now we get a little bit further away from the true data and what the true data looks like, and that can cause a few problems when we're trying to work with these different systems. Or it could look a little funny in our reports, or it's not a great representation always. So I would probably shy away from hashing in some of these cases.

But we do use hashing for things like passwords. We obfuscate passwords in databases by hashing them, so if anybody ever got a hold of the database they do automatically know what those passwords are. So we definitely use it, just not necessarily in the scenario that I've given.

Encryption, Deletion and Masking Out

You can do encryption. This isn't a great way for de-identification, and certainly not anonymization, because now we've encrypted — let's say we encrypt the names — they can be decrypted, and so that's problematic when we're going through this process of de-identification. But there is certain information we could encrypt to obfuscate.

We could also just delete or null the data. In this scenario, maybe we just remove altogether the names within here. This might work for one of our scenarios where we're trying to retain some of the data for maybe business analytics and be able to go and pull this data again, but the data might end up looking a little funny, or might not pull right, if we just start deleting certain pieces of information. More critically, one of our scenarios was that we wanted test data for our test environment, and if we just start deleting some of this then we don't have a good data set for our test environment. So this may or may not be a good solution.

Then there's masking out. This is a little different than data masking — the term data masking essentially is the same thing as data obfuscation, but when we're talking about masking out, this is where we show a partial. Maybe it's the last four of a social security number, or maybe, in this case right here, it's a credit card. What's on a credit card, we mask out the numbers so it can't be seen by others. We could do this on reports, and maybe we even do it within our database. This is an example that we can mask out certain pieces of information.

Obfuscation in Code

There's also this term called obfuscation when it comes to software and coding. The idea behind obfuscation of code, whatever we're developing of software, is making it so it's really hard to interpret what it's trying to do. A lot of viruses are an example of this: a virus will go through and obfuscate the code so that if you were analyzing the code and trying to interpret what it was trying to do, it would be a little bit elusive to you. So this is another type of obfuscation.

We covered quite a few different ways that we could go about this data masking or data obfuscation, but essentially the result is that we're covering up this data so that if anybody were to ever take this data it would be pretty much useless to them, although it still serves our purpose.