I opened them both up in SSMS and the code for the procedures was identical (as you would expect), but there was something different.  The procedure that had the really high CPU % times was compiled with the SET QUOTED_IDENTIFIER ON setting, while the procedure that had the really low CPU % times was compiled with the SET QUOTED_IDENTIFIER OFF setting.

I have no idea why there was a difference, but I changed the one which was ON to OFF and pushed the procedure into the database.  As soon as I did the CPU % for that procedure dropped down into the single digit range where it should have been.

Let this be a lesson, those setting definitely matter.  And don’t trust that they are correct, even in Microsoft provided code like Session State.

Denny

Some background

For those that don’t know what session state is, let me take a step back and fill you in.  Session state allows the web developer to store values in variables up on the web server, but in a shared location so that it doesn’t matter what web server the end user connects to the values are all there and the web servers understand the session information from each other.  This is very important for shopping websites, or pretty much any site with a login prompt and a load ballancer that is configured to send the users connection to the next available web server.

What is the problem?

The problem becomes that most people when they need to scale out session state opt to put the session state information into SQL Server because that is a nice easy repository that is probably already up and running, not to mention that the session state code in ASP.NET easily supports it.  The issue with putting the session state information into SQL Server is that you don’t give a crap about persisting the session state information to disk.  The information doesn’t get persisted more than an hour or so, and if the information within the database is lost, the only impact is that any values in session variables is lost.

So what are the options?

There are a few options besides using SQL Server for session state.

1. Keep using the in-process option.
2. Use the ASP.NET Session State Service

The easiest option is to keep using the in-process option in IIS for session information.  This means that you will need to configure what are called sticky sessions on the load balancer so that the user always goes to the same web server every time.  In the event that a web server fails, all the session information for those users would be lost, and they would need to start over by logging back into the site, or they would have an empty shopping cart, or whatever the site does.

The ASP.NET Session State Service is a Windows service that provides a memory only session state repository so that session information isn’t ever written to disk, and it isn’t kept on the web server.  You can either stand up a dedicated machine for this, or setup a couple and cluster the service manually so that you have an HA solution for your session state service.  If you have an existing SQL Server cluster you can even use this cluster for it, if you don’t have anywhere better to put it.  Just configure the cluster to have the SQL Server run on one node as the preferred node, and have the session state run on another node as its preferred node.  This way the services won’t ever run on the same node unless the other node is offline.  The session state service doesn’t take a lot of CPU power, and the amount of RAM it needs to be completely dependent on the amount of information that you are stuffing into session variables on the web servers.

In either case, either solution is better than putting the session state information into a SQL Server database.  The information doesn’t need to be written to disk, ever.  The information that is written into the session state database is in a blob binary form, not any sort of relational form so you can’t really do anything with it.

How do I know if sessions state is in SQL?

That is an easy one.  You’ll have some funky database, probably in simple recovery mode usually name aspnetdb (maybe with a prefix and/or a suffix).

What other problems can session state cause?

Well first it’ll take away buffer pool resources from your other databases.  Because the session state database is hit very hard the data from the database will always be in memory.  How ever much data is in the session state database, you are missing that much buffer pool space for your other databases.  Because of the way that session state works, every single time a page is clicked on the website, at least one call is made to the session state database pretty much forcing the database server to keep the data from the database in the buffer pool.

Another problem that you can see if ghost records.  Paul Randal describes ghost records perfectly on his blog post Ghost cleanup in depth.

When a record is deleted, apart from it being marked as a ghost record, the page that the record is on is also marked as having ghost records in one of the allocation maps – the PFS page (post coming soon!) – and in its page header. Marking a page as having ghost records in a PFS page also changes the database state to indicate that there are some ghost records to cleanup – somewhere. Nothing tells the ghost cleanup task to clean the specific page that the delete happened on - yet. That only happens when the next scan operation reads the page and notices that the page has ghost records.

He also talks about how to fix the problem in the same blog post, so I’ll leave you to click through for all that information.

Another problem goes back to the fact that the SQL Server will be persisting everything to disk, and with the information in this database changing all the time, as the website grows you’ll need faster and faster disks under the session state database just to keep up.  As the site grows more popular you will end up spending more and more money on faster and faster disks, just to keep the session state working, much less anything else on the SQL Server.  You may even get to the point where you actually need a dedicated SQL Server just to run the session state database.

Another problem is good old locking and blocking.  SQL Server likes to take page level locks when it does insert, update and delete operations.  Well, unless each page on disk only has a single row in it you are going to have processes being blocked for short periods of time as other users session state information is updated.  You can work around this to some extent by hacking the session state database’s stored procedures and forcing row locks, but now you are taking more locks (and more memory for locks), etc.

How do I change to the session state server?

First you need to install the service on the server that will be your session state server.  On Windows 2008 just install all the .NET components and that should do the trick.  You’ll probably want to start the service as well.  When you bring up the list of services it is the one called “ASP.NET State Service”.

Then on each web server you’ll want to change the session state information. You can either use IIS Manager to do this, or change the web.config. However you configured session state the first time, that’s how you’ll want to change it this time.

To use the IIS Manager select the website (or application) to configure and double click on the “Session State” icon on the right (shown below).

From there select “State Server” from the list and change “localhost” to the server you’ll be using for session state. If you have changed the TCP port number from 42424 to something else you can adjust that here as well.

To set the session state setting via the web.config file find the existing session state information and edit it, or add in the session state information. Set the “mode” to “StateServer” and set the “stateConnectionString” to the same value that goes in the IIS config setting.

<configuration>
  <system.web>
    <sessionState mode="StateServer"
      stateConnectionString="tcpip=SampleStateServer:42424"
      cookieless="false"
      timeout="20"/>
  </system.web>
</configuration>

Hopefully I’ve convinces you to move your session state information out of SQL Server and into a repository that it actually belongs in.

Denny

If you’re like me you probably upgrade when you have the chance.  If you’re like my company you don’t run just one website per server.  In our case this cluster of web servers runs 6 different sites, and the session state for one of them keeps resetting just about every page view.

Apparently one of the values which is used to create the session id is the ID number of the website in IIS.  Up through IIS 6 this number was a hash of the name of the site, so as long as the sites all had the same name you were set.  In IIS 7 this ID number is now just a running number based on the order you create the websites in.  So if you create the websites in a different order on one or more machines then your ASPState information will keep expiring since the hash values don’t match correctly.

The good news is that you can change the ID number by clicking the Advanced Settings link on the right hand side.

Denny