Recently I was setting up access to just another high security system and I was presented with this list of questions (I had to select three) as my security questions so that I can get my account back if (when) I forget my password.

Lets review these questions for a minute.  Now some of these Facebook actually asks you to provide them so that they can put them on your profile (that annoying part at the top of your Facebook profile).   The rest you can probably figure out about most people just by looking at the information that they provide during the course of using social media in their daily lives and with the groups on Facebook that they belong to.  You add access to a public linked in profile and a little searching in public records and getting most if not all of these answers shouldn’t take you more than a couple of hours.

As the people that build these applications we need to take more notice of just how easy it is to figure out these questions.  The questions that we are putting into the applications shouldn’t be so annoying as “Favorite Teacher’s Last Name”, which I’ve actually seen but they need to be stuff that is at least a little harder to figure out if these are the things that we are going to use to ensure that people are who they say they are.

Things like drivers license number or state ID number (for those without a drivers license) are a good start.  They don’t change all that often (except when you move between states).  Social Security Numbers basically never change so those aren’t a bad number to use (granted there are other issues with using a persons tax ID here in the US).

When you are designing these sorts of authentication systems, don’t assume that just because your paranoid ass doesn’t upload your entire live to Facebook, LinkedIn and Twitter that no one else does that either.  People do, do that and they will continue to do that.  If you want to actually provide a level of security for your customers, which I sure hope that you do as that is kind of your job, then assume that the customers will be posting the easy to figure out questions online for all to see so you might want to use some slightly more complex questions.

Denny

In this technique we’ll encrypt the data using just a single column.  This technique requires butting some additional logic within the application to figure out if the value is encrypted or not, but other than that logic, which you can leave in and strip out later the changes to the application are pretty minimal as the column stays the same, so that means that the stored procedures don’t need to be changed.

The first thing to remember is that the encrypted data will be larger, possibly much larger than the plain text version of the data.  Because of this you’ll need to increase the size of the field which you’ll be putting the data into.  Now the good news is that if this column isn’t indexed this change should be pretty quick and easy as it should just be a meta change which tells the SQL Server that the column size can be bigger without having to actually change the pages.  You can see this by making some changes to the [HumanResources].[Employee] table within the AdventureWorks database.  By turning on STATISTICS IO and using the ALTER TABLE statement we see that there is no IO generated when we change the size of the LoginID column from nvarchar(256) to nvarchar(512).

set statistics io on
alter table MyTable
alter column LoginID nvarchar(512)

Once the column is made larger the .NET code needs to be modified to see if the data is compressed for not.  Now there is no sure fire way to check to see if a value has been encrypted or not, but a pretty good test is to look at the last two characters of the value.  If they are both an equal sign (==) then it is probably safe to assume that the value is encrypted.  To don’t want to just attempt to decrypt the data and look for an error message, and if there is an error assume that the encrypted value is in plain text, throwing and catching error messages in .NET is very expensive, especially compared to simply checking to see if the last two characters are an equal sign.  This isn’t to say that you shouldn’t have TRY/CATCH logic around the code that decrypts the values as someone could easily enough put two equal signs at the end of their password.

At this point either a .NET app or a T-SQL script can loop through the values in the table which aren’t encrypted and then encrypt them, updating the rows which aren’t already encrypted.

Look for more blog posts in this series on how to encrypt data which already exists within your applications database.

Denny

In the basic view, an Index is a subset of columns from a table. This subset of columns is stored in a sorted order so that the SQL Server can more quickly find the records based on the data in the index.

As an example, say that we have a table called Employee with four columns EmployeeId, FirstName, LastName and ManagerId. The EmployeeId column if the primary key of the table, by default the primary key is the clustered index (I’ll write another post about that later on) for the table.

We want to be able to search the table quickly for an Employee based on the last name. If we search the table as it is the SQL Server has to read every record in the table looking for matches. While at most companies this might be ok, if you have a larger employee base, say the US Army for example you’ll have hundreds of thousands of records in this table and searching it one record at a time will take quite a bit of time.

With an index created on the LastName column when we look for the employees last name the SQL Server knows where the records are in the table because the records are sorted in Alphabetical order.

Until SQL Server 2008 was released an index was required to hold all the records of the table. SQL Server 2008 introduces the concept of filtered indexes. When a filtered index is created the index is created on a subset of the data in the table by specifying a WHERE clause in the CREATE INDEX statement. This is good for making the indexes smaller and there for even faster.

For an example lets look back at the Employee table from above. Lets add a column called CurrentEmployee which is a BIT field. When the employee works for the company the value is set to 1 (shown as true in the HR system), when they leave the company it is set to 0 (shown as false in the HR system). Knowing that most of the queries in the system will only need to look for current employees we can create filtered indexes to improve the performance of those queries so that the index only has to hold the current employees.

One thing to remember about indexes, is that SQL Server will only use a single index per table per query. So if you are using more than one column in your where clause (or JOIN clause or COUNT clause) then you will need to have more than one column in your index creating what are called covering indexes.

In SQL Server 2005 we were introduced to another new feature of the index. This gives us the ability to add columns to the index which aren’t actually indexed. This gives us the ability to improve performance yet again, but this time the performance improvement has a cost; this cost is disk space.

Before the include feature was added to the indexes after finding the rows that you need, the SQL Server would then need to go back to the base table to find the rest of the values that you requested. When the SQL Server has to do this it is called a lookup. As an example, lets look at the Employee table from above again. We have an index on the LastName column, but we also want to display the EmployeeId and FirstName values as well so we write our query like this.

SELECT EmployeeId, FirstName, LastName
FROM Employee
WHERE LastName = N'Smith'

The SQL Server scans the index for the records with the LastName value of Smith. But we want the other columns as well, and they aren’t in the index. Because of this the SQL Server now has to do a lookup for each row being returned and get those values from the base table (or the clustered index). This requires that the SQL Server load all the data pages which hold all the records to get these values. Since the employees with the LastName value of Smith probably were not hired at the same time, the data is probably spread over many data pages all over the disk. This means that the results will take much longer to return while we wait for all this disk activity to complete.

Now, if we include these columns in the Index, all the values of Smith will still be kept together, but the EmployeeId and FirstName will be stored in the index along with the Smith values. So when we run the same query with the change to the index the query will run much faster now as the SQL Server can process all the records in a single internal operation.

What’s the difference between adding the columns to the index, and including them you ask? The easiest way to show this to stick an OrderBy clause on the query.

SELECT EmployeeId, FirstName, LastName
FROM Employee
WHERE LastName = N’Smith’
ORDER BY FirstName

By adding the Orderby clause the SQL Server now has to sort the data in a separate operation instead of a single operation. If we change the index to be on the LastName and FirstName columns and include the EmployeeId column the data will already be sorted in the correct order, and the SQL Server shouldn’t need to do any additional processing of the data before returning it to the client application.

I know that this was a long post, but I hope that you found it worth it.

Denny

 Denny