This session will be a two part session in which we will be focusing on two of the biggest topics in the DBA field. How to properly design your SAN storage solution and how to properly design your virtualization solution.
The storage portion of this session will focus on SAN storage, but most of the material will apply to direct attached storage as well.
In the first half of the session we’ll be focusing on the storage array. Storage can be one of the biggest bottlenecks when it comes to database performance. It’s also one of the hardest places to troubleshoot performance issues because storage engineers and database administrators often do not speak the same language. In this session, we’ll be looking at storage from both the database and storage perspectives. We’ll be digging into LUNs, HBAs, the fabric, as well as the storage configuration.
After going over the components we’ll dig into some advanced storage configurations. This includes RAID groups, multi-pathing software, and proper redundant storage network design. We will also be digging into some advanced storage array backup techniques including taking storage level clones and snapshots. After going over these advanced techniques we will dig into how these can best be used to backup the SQL Server environment to provide maximum redundancy with no recurring tape costs.
In the second half of the day we’ll be looking into the pros and cons of moving SQL Servers into a virtual server environment. Specifically we’ll be looking into when it’s a good idea and when it’s probably not a good idea. Like everything in the database world there are no hard set answers as to if virtualization is a good idea or not, but there are some times when virtualizing a SQL Server is a good idea, and can save you some money. There are some other times when you will be shooting yourself in the foot and virtualization isn’t a good idea. We’ll be focusing on when how to make this decision, and how to gather the metrics that you need in order to come to this decision.
We’ll look into how tie the virtual platforms to the storage array so that you can maximize the storage performance for your SQL Servers and the virtual environment.
You can sign up for the normal SQL Saturday event (I’ve got several slide decks submitted including one brand new one) at the SQL Saturday webpage. The pre-con has a separate sign up page as there is the fee that has to be paid. There are only 45 seats available for the pre-con (as well as another 45 seats in the BI pre-con which Stacia Misner is presenting). I haven’t been told that I can’t being goodies to give away, so I’ll see what sort of goodies I can get my hands on between now and then.
Hopefully I’ll see you on Friday at the pre-con session, but if not I’ll hopefully see you on Saturday at the SQL Saturday event.
Some perfmon settings that you’ll want to look at include the seconds per operation, operations per second, disk queue. Now if you are in a SAN environment you will also want to look at the metrics which the array presents to confirm that the problem is actually on the array side and not just a problem of the server.
However if the server has its own disks on the storage array, and the array numbers match what the server is seeing then it may be time to look at EFDs or SSDs for your disk solution.
In the case of the transaction log, every write into the log is put down to disk immediately, and then written into the database file. SQL tells Windows to do the write, and Windows tells the disk to do the write. When the write is written to the write cache the disk tells Windows that the write has been completed, and Windows passes that information to SQL which tells SQL that the write to the log is done, and that it can now write to the database files as well. If the power fails in the middle of this, after the transaction has been committed to the log, but before it has been committed to the database the data still has been written to the disk. When the array sees that the power has gone out it’ll write the write cache to the disk, then power down.
When the power comes back up the array will load up the write cache back into memory, and flush the write cache down to where it needs to be written (in the similar way that SQL does a roll forward when it restarts). Once the writes have been completed the array will allow the hosts (the servers) to see the LUNs and the SQL Server can then fire up and do its normal roll back and roll forward of the data within the transaction log.
Hopefully this helps fix some confusion.
P.S. Yes I am well aware that I have greatly over simplified the process of how SQL writes to the log and the database, but that isn’t the point of the article. I’m sure that Paul has some great articles on the internals of how this works over on his blog.]]>
Creating a Windows cluster under the hypervisor gives you some additional protection. If you only have the host HA solution while you can live migration or vMotion from one host to another easily you aren’t protected from a guest OS failure or from a host failure (sort of). If the host fails, yes the platform will restart the VM on another host pretty quickly, but pretty quickly may not be good enough for you. You may need it to restart within seconds of a host failure.
What happens on patch Tuesday when all the security patches are rolled out? The Windows OS within the guest still needs to be rebooted after the patching and the HA solutions of the host won’t do anything for you here.
However if you need to ensure that very high up time of a SQL Server database (or any other cluster-able service) then building a Windows Cluster under the hypervisor of choice is a viable solution for you to use. Just make sure that you use the rules engine of the host to ensure that both nodes of the cluster are never running on the same host server, because if they are then the whole point of the cluster will be lost in the event of a host failure.
P.S. This post and the next bunch of posts are all questions and answers which I have received from online sessions at either SQL PASS Virtual Chapters or the 24 Hours of PASS virtual event.]]>
These all have the same effect, to tell the Windows API to issue a stop command to the service which the service takes and uses to shutdown. None of them are the correct way, as they all do the same thing.
For the first time I was able to deliver a pre-con session on the day before the first regular conference day. For those that haven’t gone to a pre-con they are typically full day sessions (8:30-4:30) and come with an extra cost to the attendee, $400 per person in this case. My pre-con was a great success as far as I could tell. There were lots of questions (which means that people were listening and wanted deeper information than I had in the slide deck) and we had some great discussions during the session. Another great thing was that the room was pretty full with 82 attendees being the final count.
During the regular session days I was able to make a few sessions, less than I would have liked to have made it to, but more than I went to last year. I end up spending a lot of time socializing with old and new friends, building my professional network.
As a testimonial to how much better things went this year, on Friday night after PASS last year I sat down with Bill Graziano (Twitter) and he wrote down hundreds of suggestions, problems, etc that we spotted with the 2009 summit. This year there were about a dozen (not counting the keynote opening act) and most of them were pretty small. Bill apparently considers this feedback to be valuable, as we have actually scheduled some time for next year for this same conversation.
The biggest issue which I saw was that PASS wasn’t doing a very good job with the dietary needs of some of the attendees. The group which came to mind was a small group of a dozen or so people who didn’t have much of anything to eat during breakfast and lunch because they have an allergy to wheat. Without having access to food which they can eat, those attendees would need to head out to the convention center vendors who did have options. Bill agreed that attendees shouldn’t be forced to buy food, and that hopefully we could get that fixed easily. The nice thing about this problem is that it is easy to fix, and should be cheap to fix (Bill really likes things which are cheap to fix as he’s the guy in charge of all the money for PASS).
Other things were pretty small:
I think that PASS made some great additions to the conference this year. The biggest (for both the attendees and the PASS budget) would be the recording of the pre/post-con sessions and making them available for sale. There were so many excellent pre/post-con sessions that picking the one to attend can be tough. With the sessions being recorded this year you could (after attending a pre-con) buy all the pre-cons on DVD for the cost of a second pre-con ($395) or if you wanted just one pre/post con track it was like $150 or something (also an excellent deal).
After all this I had a great time, and I would (as many have done) call this the best summit so far.
Don’t forget that the summit is in October next year (scheduling issues caused the move) and there’s no time the like present to start planning and working your boss to get money in your company budget to get up there. Once I get home and get my pictures downloaded (and I steal some of Pat’s) I’ll do another post with some pictures from the summit. Until then you can find them on Flickr by looking at the sqlpass tag.
After Rick gave a lucky winner an XBOX Kinect we were treated to a presentation by David Dewitt on how the optimizer works within the SQL Server database engine. Now there isn’t really anything that I can post about David’s presentation other than it was awesome to get this sort of insight into the query optimizer other than the fact that optimizer’s are very hard to write. One really cool thing that David did was to accept questions from the audience both online and in real life via email which were culled by people from PASS HQ and were passed to him after the power point was completed.
After listening to David’s keynote my head hurts. Tonight there will be more singing and drinking at Bush Garden to make the pain go away.
Today was also the first time that I used my iPad to blog for the day. I went totally laptop free for the day to see how well it works for me.
Quentin Clark was our next keynote speaker for today. Quentin started by talking about the high level features around the new Denali product (also called SQL 11). SQL Server Danali will be bringing in an “Always On” concept which will allow ever database to have the ultra high availability requirements that many database applications have. This new “Always On” solution doesn’t require any sort of shared storage to work. It will replicate data to multiple servers with the ability of having some servers available for automatic failover ads well as having some servers available for reading the data as well as for backup so you don’t have to take the backups from the active server (not in the current CTP). The awesome part of the setup for this feature doesn’t require that you copy backups around. It will handle the moving of those backups automatically. I have a lot of questions about how this happens and how it effects the system cache.
On the T-SQL side we now have paging, sequences, and much better error handling. Paging allows you to specify the size of each page to display and which page to display with the database engine doing the hard work. Sequences and some more flexibility into the engine by allowing the developer to use the same sequence numbers for multiple tables.
The new filetable feature is a feature which I really don’t see the use for. This feature allows you to create a table which is exposed as a network share so that applications can write files into the database without any application changes. Under the hood it uses filestream feature to handle the IO. We have spent years keeping this blob data out of the database because of the performance issues of having TB of data within the database. Some questions which need answering about the filestream is how does this affect the buffer pool when pulling files in and out of the database engine? Is there a point at which this feature can’t handle a file load, in other words how many files can I shove into a single filetable table before it’ll slow down performance?
(Sorry this was so late coming out, I’m such a slacker.)]]>
Ted Kummert was our keynote speaker for this mornings keynotes. Before Ted come onstage there was a very interesting video which gave us some history of the SQL Server product. The thing that I found the most astonishing was that SQL Server 6.0 and 6.5 was a $100M a year business for Microsoft. Today the SQL Server product is a $3B a year business. That is a massive amount of growth over just 12-13 years or so.
Parallel Data Warehouse
The first announcement was that the Parallel Data Warehouse (PDW) has been released, and the first appliances will be shipped from Dell and HP starting in December of 2011. When you create a database without specifying where the database is stored. When you create a table you specify the column which is used to distribute the data across the servers, as well as how the table should be partitioned. The admin console for the PDW is a web based system which allows you to view how all the cores are running and really see what is happening with the system, this is in addition to the SQL Server Management Studio of course.
The next thing which we saw was Yahoo telling us about their SSAS system which they have. It processes 3.5 billion rows a day which is 1.2 TB of new data per day being processed into a 12 TB SSAS cube. Truly an amazing scale.
Microsoft has announced the new project “Atlanta” which is a new self service configuration assessment service for Microsoft SQL Server which will hopefully reduce the number of support calls. Atlanta is a Azure based system which analyzes your system looking for problems with the SQL Server configuration. The Atlanta system will scan the system configuration configuration and upload the configuration to the Atlanta application nightly. Atlanta will then process the configuration and show the knowledge base articles which show how to resolve the potential issues. When systems settings are changed Atlanta will track those changes to allow the DBA to more quickly find changes which have been made to the SQL Servers.
SSRS is now available as an Azure product where you can simply upload your SSRS reports to the Azure SSRS website and it’ll get its information from your SQL Azure databases. These reports are edited and published using the same BIDS application which we already use today.
The new Azure Marketplace allows you do purchase a variety of known good data from a variety of sources such as local weather. The data is available to be imported into SSRS reports via a data source using an ODATA URL from SQL Server Reporting Services, either in the cloud or a local SSRS report. This published data can then be used side by side with the business data.
Denali (aka SQL 11)
The new Denali CTP (the first public one) has been released to the public. It will be shown during tomorrows keynote, so today was just the announcement. You can download the package today from the Microsoft website. I’m sure there will be much more information tomorrow about the product tomorrow. Be sure to watch Twitter (specifically the #sqlpass hash tag) to watch the information in real time.
Check back tomorrow for more keynote information.