While attacking this problem, the first thing that I did was look into the SSIS package and see what I had gotten myself into.  There were hundreds of objects within SSIS, most of them T-SQL Scripts of Data Pump Tasks.  So simply going through them one by one wasn’t going to be all that helpful.  The client didn’t have any logging to SQL Server or to a text file setup, but they did have the SSIS package setup to log to the Windows event log.

Opening the Application log greated me with one event for the start of each task, and one for the completion of each task.  Not exactly the most useful information ever, but better than nothing, so I’ll take what I can get.  The first thing that I did was export the Application Event log to a CSV file and download it to my desktop so that I could do the processing on my local machine.  The CSV file has a few hundred thousand lines, so I needed to figure out what I had via scripting.  The first problem that I had was that for each SSIS event there were multiple lines within the csv when opened in Excel which you can see below.

Thankfully this made the rows pretty unique looking, so an Excel Macro was able to clean this up pretty easily.  Before I got started I added some column headers into the excel sheet.  The default columns from Event Viewer are Date, Time, Source, Severity, Category, EventID, User, Computer, Description.  This covered columns A through I.  After that I put columns J through P as being Operator, SourceName, SourceID, Execution, StartTime, EndTime and DataCode as these are the specific items that SSIS is logging that you can see above.  Then it was time to bust out a little VBA and flatten out all this data.  The macro shown below looped through the data and flattened everything out so that all the data was on a single line.  Then is went through and removed all the white lines.  This macro took about 3 hours to run.

Sub CleanUpData()
Dim Row As Long
Dim DataRow As Long
Row = 32768
Do While Range("A" & Row).Value = ""
If Range("B" & Row).Value = "" Then
DataRow = Row
Range("A" & Row).Select
Else
If Left(Range("A" & Row).Value, 4) = " Mes" Then
Range("A" & Row).Value = ""
End If
If Left(Range("A" & Row).Value, 4) = " Ope" Then
Range("J" & DataRow).Value = Range("A" & Row).Value
Range("A" & Row).Value = ""
End If
If Left(Range("A" & Row).Value, 9) = " Source N" Then
Range("K" & DataRow).Value = Range("A" & Row).Value
Range("A" & Row).Value = ""
End If
If Left(Range("A" & Row).Value, 10) = " Source ID" Then
Range("L" & DataRow).Value = Range("A" & Row).Value
Range("A" & Row).Value = ""
End If
If Left(Range("A" & Row).Value, 4) = " Exe" Then
Range("M" & DataRow).Value = Range("A" & Row).Value
Range("A" & Row).Value = ""
End If
If Left(Range("A" & Row).Value, 4) = " Sta" Then
Range("N" & DataRow).Value = Range("A" & Row).Value
Range("A" & Row).Value = ""
End If
If Left(Range("A" & Row).Value, 4) = " End" Then
Range("O" & DataRow).Value = Range("A" & Row).Value
Range("A" & Row).Value = ""
End If
If Left(Range("A" & Row).Value, 4) = " Dat" Then
Range("P" & DataRow).Value = Range("A" & Row).Value
Range("A" & Row).Value = ""
End If
End If
Row = Row + 1
Loop

Do While Row 2
If Range(“A” & Row).Value = “” Then
Range(Row & “:” & Row).Select
Selection.Delete
End If
Row = Row – 1
Loop
End Sub

Once that was done I needed to set the start times for each event on the same line as the end time for each event. That way I wasn’t scrolling through looking for stuff while trying to do my data analysis. This I was also able to do with a little Macro. In this case I started on row 14 as that was the first entry that had SSIS data. You may need to change the EndRow=14 line to what ever line your data starts on. This puts the start time and the end time into Columns R and S, but only on the line which is the OnPostExecute line.

Sub FindRunTimes()
Dim EndRow As Long
Dim StartRow As Long
Dim SourceName As String
EndRow = 14
Do While Range("A" & EndRow).Value = ""
If Range("I" & EndRow).Value = " Event Name: OnPostExecute" Then
StartRow = EndRow + 1
Do Until Range("K" & EndRow).Value = Range("K" & StartRow).Value And Range("I" & StartRow).Value = " Event Name: OnPreExecute"
If Range("I" & StartRow).Value = "" Then
MsgBox "Ran out of rows to process for ending row " & EndRow
Exit Sub
End If
StartRow = StartRow + 1
Loop
Range("S" & EndRow).Value = Range("B" & EndRow).Value
Range("R" & EndRow).Value = Range("B" & StartRow).Value
'Range("T" & EndRow).Value = DateDiff("mi", Range("A" & StartRow).Value & " " & Range("B" & StartRow).Value, Range("A" & EndRow).Value & " " & Range("B" & EndRow).Value)
Range("R" & EndRow).Select
End If
EndRow = EndRow + 1
Loop
End Sub

Once that Macro was finished I needed to figure out which of the statements was taking to long to run. Putting a formula into the column Q marked the rows that I needed to look into a little more. In this case I was using anything with a run time of 20 minutes or longer as needing to be looked at.

=IF(HOUR(R595)=HOUR(S595)*60+MINUTE(S595),"", "X"), IF(HOUR(R595)*60+MINUTE(R595)+19>=HOUR(S595)*60+MINUTE(S595)+1440,"", "X"))

Once that formula was in there all the rows that were SSIS objects that took a long time to run had an X next to them. Now I still needed to ignore the rows which were containers, but those were pretty easy to figure out from the names of the objects as most of them had container in them (in the SourceName column which was column K of my Worksheet). At this point I was able to move back to the SSIS package and start digging through the package finding the objects which had long runtimes and getting those looked at.

The whole reason I went through this process instead of just adding logging to the SSIS package was for a few different reasons.

• I didn’t want to make ANY changes to production if I didn’t have to. This SSIS package loads a production data warehouse which the business uses to run their company, so making changes to it wasn’t something I wanted to try.
• If I had simply put logging into place I would have had to have waited another day to being looking at the problems as the ETL process only runs once a day.
• The SSIS package takes 8+ hours to run, so sitting around waiting for it to run wasn’t exactly the best use of my time, or the clients money.

If you get stuck in this same sort of problem, hopefully this approach will help you out.
Denny

Deleting the data from the database is the easy part, a simple delete loop will handle that nicely.  The problem is when you delete rows from a table which contains LOB data the LOB pages aren’t cleared when they are deallocated.  We can see this by running the following code.

CREATE DATABASE Lobtest
GO
use Lobtest
GO
CREATE TABLE t1 (c1 int IDENTITY(1,1) PRIMARY KEY, c2 ntext)
GO
INSERT INTO T1 (c2) VALUES (replicate('a', 20000))
GO
DBCC IND ('LobTest', 't1', 1)
GO
DBCC TRACEON(5201, -1)
GO
DELETE FROM t1
GO
DBCC IND ('LobTest', 't1', 1)
GO
DECLARE @dbid as int = db_id('Lobtest')
DBCC PAGE (@dbid, 1, 231, 3)
GO

You can see that page 231 is a LOB page which is allocated to the table t1. When you look at the actual page using DBCC PAGE after the row has been deleted we can see that there is data in the page, and that the page header shows that the page is still allocated to the table t1. This can be seen by looking in the header of the page for the header value labeled “Metadata: ObjectId = 245575913″.

When you go to shrink the database the SQL Server engine will get to the LOB pages and it will need to figure out if the LOB row is a part of a row which still exists or not. In order to do this SQL Server will need to scan through the pages which make up the table looking for any rows which reference the page it is trying to delete.

When doing shrinks after deleing large amounts of LOB data SQL Server will generate large amounts of IO while figuring this out and the shrink operation will take an extremely long time. (Paul Randle talks more about it here.)

So the question that this person at SQL Saturday had was, how can I reclaim the space from my database within a reasonable time.

The solution that we came up with was actually pretty simple. Do the database deletion as normal. Then backup and restore the database. Then do the shrink, followed by rebuilding the clustered indexes in order to fix the fragmentation issue which the shrink will introduce.

This works for a pretty simple reason, because the PFS page shows that the LOB page isn’t allocated even though the page is full of data (you can verify this by looking at page 1 in file 1 in the sample database created by the script above). When the database engine backups up the database the database engine looks at the PFS pages to figure out which pages to back up. Because the PFS pages show that the pages are empty the database engine doesn’t bother to backup the pages, so when the pages are restored they are restored as blank pages. This means that after the restore the shrink operation can run without an issue.

In the case of this application there was a maintenance window which could be taken advantage of which would allow the backup and the restore to happen.

Another option which we came up with which would require less downtime involved using database mirroring. By configuring database mirroring (which is initialized via a backup and restore process giving us the same basic approach) and then failing over to the mirror we would end up in the same position. We could then shrink the database without issue (probably pausing database mirroring so that we didn’t have to wait for the second server to process the shrink in real time) and then fail back the database to the original server.

As geeky as it was, Mladen, Andre and I had a great time figuring this out, and the attendee had a great time watching us go through all the possible options as we excluded them one by one. And most importantly she got her problem solved.

So if you end up in this situation here’s a solution that will help you shrink the database so that you can reclaim the space that the LOB data pages are taking up without having to wait forever.

Denny

I gave one presentation this year, and it was a session of table partitioning.  The slide deck has been uploaded to the SQL Saturday site as has the sample code.  You can download it from the session page for my session.

I hope that everyone liked the session, and I hope to see everyone at a future SQL Saturday.

Denny

Database chaining is when permissions cascade from one object to another because they are used by the parent object.  The perfect example is a stored procedure which accesses a table.  The user only needs rights to the parent object (the stored procedure) and the rights to access the table exist automatically because the stored procedure accesses the child object (the table).

Cross database chaining uses this exact same concept except that the parent object is in one database and the child object is in another database.  In order to use cross database chaining the feature needs to be enabled on both databases.  This is done by using the ALTER DATABASE statement as shown below on both databases.

ALTER DATABASE A_Database SET DB_CHAINING ON

Once this is done, the login which is mapped to the user within the database which has the parent object needs to be mapped to a login within the database which has the child object.  The user within the database which owns the child object doesn’t need any specific rights other than to be a member of the public role.  Once this is done the cross database permission chain will be made and the stored procedure (or other parent object such as a trigger or function) will begin working.

Denny

Officially the answer is no, however with a little bit of creative configuration you sure can.

The Overall Environment

To setup SQL Server Express in a Windows Cluster I’m building this on a two node Windows Server 2012 cluster using a file share hosted on my domain controller to host the actual databases.  To ensure that the domain controller is rebooted as little as possible the domain controller is installed in core mode. The cluster nodes are Windows Server 2012 standard edition (which now supports clustering) as is the domain controller.

Installation

As SQL Server 2012 express edition doesn’t support Windows Clustering out of the box the installation will be a little different from doing a normal clustered install under standard or enterprise edition.  To install I did a normal SQL Express install on node1.  The only change from a normal install that I made was that I configured the SQL Server instance to start under a domain account.  When I got to the data directories part I configured the data folder to a network share on the domain controller.

Once the installation on node1 was completed I stopped the SQL Server services.  Then I renamed the folder that I installed the SQL Server database files into.  The reason for this is that I need to configure the second instance to put the database files into the same location.  I can then install SQL Server 2012 express edition onto the second node.

The installation on node2 is done exactly like it was done on node1.

Once the installation is done on both nodes configure the SQL Server service to have a startup type as “Manual” instead of disabled or automatic.  Leave the SQL Agent service as disabled as even though SQL Express installs the SQL Agent the SQL Agent isn’t supported on SQL Express.

Configuring Clustering

Once the installation on Node2 is done the cluster can be configured.  To do this bring up the Failover Cluster Administrator on one of the nodes and connect to the cluster.  If the cluster hasn’t been configured yet run through the normal Clustering Configuration wizard.

We’ll now configure a new cluster role on the cluster.  To do this right click on “Role” then select “Configure Role” from the context menu as shown below.

When the wizard opens click next to get to the list of services.  Then select the Generic Service item from the list as shown below.

On the next screen you’ll be asked what service you wish to cluster.  From this list select the SQL Server service as shown below.

On the next screen you’ll be asked to name the resource group.  Give the group a name which is unique on the domain and click next.  The next screen will ask you to select the needed storage.  Simply click next on this screen as we aren’t using any local shared storage.  The next screen asks you if any registry settings need to be replicated between the machines.  We don’t need to replicate anything as SQL Server doesn’t make much use of the registry for the actual SQL Server service so we can simply click next on this screen as well.  The next screen is simply a screen to review the changes which will be made.  You can simply click next on this screen after reviewing the information on the screen.  When the summary screen displays click finish.

Post Clustering SQL Config Changes

The first change that you’ll need to make is to enable the TCP network protocol on both nodes.  By default SQL Express has the TCP network protocol disabled which need to be corrected before uses will be able to connect to the SQL Server service.

The next change that you’ll need to make is to change the local server name in the master database from the name of the last node which was installed to the cluster name using a script similar to the one shown below.  In the case of this script the nodes are named node1 and node2 and the cluster name is clustersql. Once this script has been run the SQL Server instance should be restarted or failed over to the other node.

exec sp_dropserver ‘nodeb’
GO
exec sp_addserver ‘clustersql’, local
GO

At this point the cluster is up and running and applications can have their databases configured on the SQL Server Instance.

Denny

To solve this problem, I added a DDL trigger to the server which will cause the new database to be backed up as soon as the database is created.  The trigger itself is rather simple.  Most of the trigger is setting variables.  Then I make sure that the database isn’t a database snapshot, as database snapshots can’t be backed up.  If it isn’t a snapshot we continue with everything else.

Then I create a subfolder for the backups to be put into (the backups for each database go into their own folder, so as this is a new database the folder needs to be created).  Then I commit the transaction, as database backups can’t be taken within a transaction.  Then we do the actual database backup.  I then throw a message to the end user using the RAISERROR statement telling them that they can ignore the other error that they are going to get about the transaction being closed before the trigger was complete.  This is just an annoyance of my needing to commit the transaction before taking the backup.  Sure I could have setup a job which takes the backup and emails if there was a failure, but that just seems to complex for something so simple.  The code for my trigger is below.

CREATE TRIGGER BackupNewDatabase
ON ALL SERVER
FOR CREATE_DATABASE
AS
declare @database sysname, @event_data XML = EVENTDATA(), @folder nvarchar(4000), @file nvarchar(4000)

SET @database = @event_data.value(‘(/EVENT_INSTANCE/DatabaseName)[1]‘, ‘sysname’)

set @folder = ‘X:\Backups\’ + @database

set @file = @folder + ‘\’ + @database + ‘.bak’

if exists (select * from sys.databases where name = @database and source_database_id is null)
BEGIN
EXEC master.dbo.xp_create_subdir @folder

COMMIT

BACKUP DATABASE @database to disk=@file

raiserror( ‘You can ignore the error message which says that the transaction ended within the trigger.’, 16,1)
END
GO

Hopefully you find this solution helpful if you get into a situation like this,
Denny

All of the keynotes from this years conference not only are the keynotes being streamed live, but many of the sessions are also being streamed live via the SQL PASS website on PASS TV.  The full schedule can found on the PASS TV page.

Bill Grazino told us turning his keynote that pass has sponsored 9 non-PASS events around the world.  Including these 9 events, pass has provided well over 500,000 hours of training to members of the community, much of it like SQL Saturday’s, User Groups, etc. being available to the attendees free of charge.  PASS has created a business analytics specific conference which will be held April 10-12 2013 in Chicago, IL.  More information will be available on the SQL PASS website shortly.

When Ted Kummert took the stage he talked about some of the great products that Microsoft has released this year including SQL Server 2012, the Surface, Windows Server 2012, Office 2013 as well as many others. 

Ted announced that SQL Server 2012 has been released as of today.  This release aligns with Office 2013 and the new SharePoint 2012.  Ted also announced project Hekaton which is a new in memory database engine which will be introduced within the next major release of SQL Server.  Project Hekaton is a fully in-memory database engine which will greatly improve OLTP performance.  While this new engine is a part of the SQL Server database engine, this portion has been totally rewritten to really maximize the hardware under the SQL Server database engine.

Including Product Hekaton is the Hekaton AMR Tool which (when renamed) will help identify bottlenecks which can be resolved by converting the database to a Hekaton database engine.  Because the Hekaton engine is running with the entire database being in memory latching has effectively been removed.  Hekaton also supports recompiling stored procedures from running T-SQL to running as native machine code.  This allows the stored procedures to run much, much faster without any actual code change.  While not all workloads are going to be right for the Hekaton database engine, but those that are should see a massive performance improvement.  Because project Hekaton runs within the same database engine that we have today if you know SQL Server you will already know how to work with a project Hekaton database.

Another great announcement is the clustered column store which will be introduced in the next major release of SQL Server.  Along with the clustered column store index we will also be able to update tables which have column store indexes created on them, no matter if the column store index is a clustered column store index or a non-clustered column store index.  This will introduce massive performance improvements to real time data warehouses and make data warehouses much easier to setup and performance tune; even with the largest workload.

For the non-relational side of things you can today download the Microsoft HDInsight Server CTP which is Microsoft’s Hadoop offering.

Ted also announced a new update to the Microsoft PDW which will be available in the 1st half of 2013 and will be called the SQL Server 2012 Parallel Data Warehouse.  This update will include some of the features which will be included in the next major release of SQL Server such as the updatable column store index.  Another big enhancement of the new SQL Server 2012 PDW is that SQL Server Management Studio will now be the query tool which is used to run the queries against the PDW system.  During the demo against the PDW doing a simple count against a 1PB table the query ran in just a few seconds.  Doing data aggregations against the same table for a real world style reporting query the query again ran in just a few seconds.

The next announcement was Polybase.  Polybase allows you to query data from an external source from within the SQL Server 2012 PDW.  This will allow you to create an external table which simply links to a Hadoop cluster.  This will be done with the CREATE EXTERNAL TABLE  statement which accepts the schema that you want and a hdfs URL to Hadoop without needing to create a linked server to the Hadoop cluster.

The next step of BI is here in SQL Server 2012 SP1 and Office 2013.  Office 2013 includes PowerView natively within the product.  It also supports some new mapping features natively within Office 2013 without any additional downloads or plugins.  This support includes term extraction which is done through HDInsight.  This allows us to view text data and extract names, sentiment, locations, etc. from simple text data and use those extracted strings in order to build PowerView reports.  PowerView also now supports DAX queries to a MOLAP cube straight out of the box.

Stay tuned for more updates from the PASS Summit through out the week.

Denny

With SQL Server 2012′s AlwaysOn feature we have an active server called the Primary Replica and we have up to 4 secondary replicas.  Even if none of those 4 secondary replicas are in use for anything, you will still need to license two of them to be properly licensed.  This is because when licensing SQL Server’s each licensed server gets you only a single free passive server.  So for a 5 instance AlwaysOn Availability Group deployment you’ll need to license at least 3 of those instances which would give you two passive instances.  As long as those two passive instances aren’t being used for read access they are free.

Thanks,

Denny

These sessions are a fantastic deal as you get a full day session, hand out, lunch plus lots of time to chat with the presenter and attendees before the sessions, during the breaks as well as after the session.

Can’t decide which session you want to attend?  No problem there are two days of PreCons so you’ll be able to turn the 3 day PASS Summit into a 5 day conference for just a few hundred dollars more.  If you are heading out to the PASS Summit I’d really encourage you to sign up for one of the pre-cons.  I’d love to have you in my pre-con called “SQL Server 2012 in a Highly Available World” where we are going to walk through all of the high availability options which are available in SQL Server 2012 (as well as in the older versions as well), learn how to get them setup and most importantly figure out when we should be using each one.

If you are already signed up you can modify your registration or contact the registration contact listed on the registration page and she’ll be happy to add a pre-con onto your registration.

I look forward to seeing you at the summit, and at my precon,

Denny

I’m afraid that I’ve got some bad news.  You can no longer pre-order Securing SQL Server 2nd Edition from Amazon.

Instead you have to settle for ordering the book outright and having it shipped to you.  That’s right, no more being a pre-order book, it’s published and available to be shipped directly to you.  Currently Amazon is selling the book at full price which is $49.95, but if you have Amazon Prime it is available for Amazon Prime shipping.  Because it is considered to be a text book you get a $5 Amazon MP3 Credit (what ever terms and conditions that Amazon chooses do apply).

This is a totally updated edition of the book including all sorts of new information about security within SQL Server 2012.  I of course cover things like how to secure AlwaysOn Availability Groups, how to use user defined server roles, contained users, etc. I also dive into how to properly secure SQL Server Reporting Services and SQL Server Analysis Services so they can’t be used to access data that people shouldn’t have access to.

All in all this book is much larger with Amazon showing it at 408 pages compared to just 272 pages for the 1st edition.  If you find someone cheaper to purchase it make sure that you are in fact ordering the second edition.  The ISBN number is 1597499471.

I hope that you pick up a copy of the book and that it is useful for you in securing your SQL Server environment.

Denny