I’m thrilled to be able to announce that I’m presenting two pre-cons at the Data Platform Summit 2019 in Bangalore, India. The first all-day session that I’m giving is “Azure Infrastructure” which I’ll be presenting on August 19th, 2019. The second that I’m presenting is “HADR – SQL Server & Azure” which I’ll be presenting on August 21st, 2019.
In this daylong session, we’ll review all the various infrastructure components that make up the Microsoft Azure platform. When it comes to moving SQL Server systems into the Azure platform having a solid understanding of the Azure infrastructure will make migrations successful and making support solutions much easier.
Designing your Azure infrastructure properly from the beginning is extremely important. An improperly designed and configured infrastructure will provide performance problems, manageability problems, and can be difficult to resolve without downtime.
With the introduction of multiple Azure data centers now available in India, many companies will begin moving services from data centers into the Azure, and a solid foundation is a key to successful migrations.
HADR – SQL Server & Azure
In this session, we’ll walk through the needs and process to set up a Hybrid Always On Availability Group using servers on premises for production and servers in Azure for Disaster Recovery.
We’ll be looking at high availability and disaster recovery tuning requirements, troubleshooting steps and various best practices for Always On Availability Groups in SQL Server 2017.
Why you want to be there
These two all-day sessions will both be a fun-filled day of learning that you won’t find anywhere else. Be sure to register now, as prices go up each month (the next price increase is in just a couple of days).
Blockchain is the new hot thing in IT. Basically, every company out there is trying to figure out where Block Chain fits into their environment. Here’s the big secret of blockchain; your company doesn’t need it.
Blockchain is simply a write one technology that allows you to change records, but it keeps track of every change that was made. Most systems need some auditing to see when specific changes were made for example thing about an order system that your company may have. You probably have auditing of some sort so that you can see when the new order comes in (it’s probably the create date field on the table), and there’s probably some sort of auditing recorded when the shipment is sent out. If the customer fixes their name, you probably aren’t keeping a record of that, because odds are you don’t care.
Think about what systems you have at your company. Do you need to keep a record of every single change that happens to the data, or do you care about what happens to only some of the tables? Blockchain is a great technology for the systems that need that sort of data recording. But that’s going to be a small number of systems, and we shouldn’t be fooling ourselves into believing that every company needs a system like this.
I’m not g0ing to argue that there are no systems that need this; there definitely are some systems that due. But those systems are going to be in the minority.
Executives are going to read about how blockchain is this great new thing, and they are going to want to implement it. The thing about blockchain is that there’s one major thing that building a system on blockchain requires, and that’s lots of drive space. If you want to purge data from the system after 5-6 years, that’s great; you’ll need more drive space as deleting data from a blockchain database just means that you need more space as you aren’t actually deleting those records.
A friend of mine described Blockchain as a database in full recovery mode, and you can’t ever back up (and purge) the transaction log. That’s how the database is going to grow. Remember those lovely databases that were on the Blackberry Enterprise Server back in the day? The database would be 100 Megs and the transaction log would be 1 TB in size. That’s precisely what blockchain is going to look like, but it’s going to be a lot worse because all your customers and/or employees are going to be using the application. If you have a database that’s 100 Gigs in size after a few years (which is a reasonable size for an application) the blockchain lot for this could easily be 15-20 TB in size, if not 100TB in size. And you’ll have to keep this amount of space online and available to the system.
So if you like buying hard drives (and the nice car that they get from their commissions) then blockchain is going to be great. If you don’t want to spend a fortune on storage for no reason, then blockchain is probably something you want to skip.
Recently Intel announced some major upgrades to their Xeon CPU line. The long and short of the CPU announcement was that Intel was releasing their 56 Core CPUs for public release. That’s just a massive amount of CPU power that’s available in a very small package. A dual socket server, with two of these CPUs installed, would have 112 cores of CPU power, 224 with Hyper-Threading enabled. That’s a huge amount of CPU power. And if 112 cores aren’t enough for you, these CPUs can scale up to an eight-socket server if needed.
With each one of the processors, you can install up to 4.5TB of RAM on the server, per socket. So a dual socket server could have up to 9TB of RAM. (That’s 36TB of RAM for an eight-socket server if you’re keeping track.)
For something like a Hyper-V or a VMware host, these are going to be massive machines.
My guess is that we won’t see many of these machines are companies. Based on the companies that Intel had on stage at the keynote (Amazon, Microsoft, and Google) we’ll be seeing these chips showing up in the cloud platforms reasonably soon. The reason that I’m thinking this way is two-fold; 1. the power behind these chips is massive, and it makes sense that these are for a cloud play; 2. the people who were on stage at the Intel launch were executives from AWS, Azure and GCP. By using these chips in the cloud, the cloud providers will be able to get their cloud platforms probably twice as dense as they have them now. That leads to a lot of square feet being saved and reused for other servers.
As to how Intel was able to get 56 cores on a single CPU, is through the same technique that they’ve used in the past. They took two dies, each with 26 cores on them and made one socket out of that. In the olden days, we’d say that they glued two 26 core CPUs together to make one 56 core CPU. The work that Intel had to do, to make this happen was definitely more complicated than this, but this thought exercise works for those of us not in the CPU industry.
These new CPUs use a shockingly small amount of power to run. The chips can use as little as 27 Watts of power, which is amazingly low, especially when you consider the number of CPU cores that we are talking about. Just a few years ago, these power numbers would be unheard of.
I’ve seen a couple of conversations recently about companies that want to be able to script out their database schema on a daily basis so that they have a current copy of the database; or systems that have to change permissions with the database frequently, and they need to export a copy of those permissions so that they have a copy of those settings.
My question to follow up on these sorts of situations is, why aren’t these settings in Source Control?
Pushing these changes to production requires a change control process (and the approvals that go with these). That means that you have to document the change in order to put it into the change control ticket, so why aren’t these changes pushed into your source control system?
Anything and everything that goes into your production systems should be stored in your source control system. If the server burns down, I should be able to rebuild SQL (for example) from the ground up, from source control. This includes instance level settings, database properties, indexes, permissions, table (and view, and procedures) should all be in your source control system. Once things are stored in your source control system, then the need to be able to export the database schema goes away, as does the need to export the permissions regularly. As these have no point in doing them, there is no need to do them.
Think I’m wrong, convince me in the comments.
With Microsoft Azure now supporting Virtual Machines with NVMe storage; things get a little different when it comes to handling recoverability. Recoverability becomes very important because NVMe storage in Azure isn’t durable through reboots. This means that if your shutdown the server, or there is a host problem, or the VM host has to be patched and rebooted than anything on the NVMe drive will be gone when the server comes back up.
This means that to keep data on the VM past a shutdown you need to think about high availability and disaster recovery.
You need to have High Availability built into the solution (with Availability Sets or Availability Zones) which probably means Always On Availability Groups to protect the data. The reason that you need to have Availability Groups is that you need to be able to keep the data in place after a failover of the VM. When the VM comes back up, you’ll see the server is up, but it may not have any data. So what needs to be done at this point? You need to create a job on every node that will automatically look to see if the databases are missing and if they are then remove the databases from the AG, drop the databases, and reseed the databases from the production server.
Because of the risk of losing the data that you are protecting, you probably want at least three servers in your production site so that if one server goes down, you still have redundancy of your system.
You need to have Disaster Recovery built into your solution as well as high availability. Because of the risk of losing data if a set of VMs fails you need to plan for a failure of your production site. The servers that you have in DR may or may not need to have NVMe drives in them; it all depends on why you need NVMe drives. If you need the NVMe for reads then you probably don’t need NVMe in DR; if you need NVMe for writes, then you probably do need NVMe in DR.
While a full failure of your production Azure site is improbable, it is possible, and you need to plan for it correctly.
If you have NVMe in DR, then you’ll want to the same sort of scripts to reseed your databases in the event of a SQL Server restart.
But this is expensive
Yes, it yes.
If the system is important enough to your business that you need the speed of NVMe drives, then you can afford the extra boxes required to run the system probably. Not having HA and DR, then complaining that there was an emergency and the system wasn’t able to survive won’t get a whole lot of sympathy from me. By not having HA and DR you made the decision to have the data go away in the event of a failure. If these solutions are too expensive, then you need to decide that you don’t need this solution and that you should get something else to run the system.
Sorry to be brutal, but that’s the way it is.
Another year has past, and VMware has decided to make me a VMware vExpect again. I believe that this is the 5th time I’ve been a VMware vExpert (the 4th time in a row, there was a gap year because I forgot to fill out the form, it was a thing).
I’m thrilled that VMware has decided to give me this award for the 4th time in a row. It’s a great honor to be selected for the VMware vExpert award, more so because I’m not a sysadmin by trade, but I’m able to talk to sysadmins about databases and what the best options for hosting them within your VMware environment are.
Thank You, VMware for recognizing all the work that I’ve been doing, and that I plan to keep doing throughout the next year.
Oh, how wrong I was. Back in the day, all I worked on was Microsoft SQL Server. These days I’m doing some Microsoft SQL Server and a decent amount of Microsoft Azure and Amazon AWS cloud work. With all three of those, there’s a lot of Linux in play. Microsoft SQL Server has supported Linux since the release of SQL Server 2017 at Ignite 2017. Microsoft Azure and Amazon AWS have both supported Linux since (I believe) they first supported VMs in their cloud platforms (forever is the world of computers).
Back when I had just a few years expense with SQL Server (and IT in general) I also owned and managed a large (at the time) Oracle database which ran on Unix. Once that was no longer my baby to manage, I assumed by *nix carrier was over. And it was, for a while, but now Linux is back and this time in the SQL Server world.
Looking at the servers that DCAC has in our Azure environment, we have more Linux boxes than Windows. Our website runs off of PHP running on a pair of Linux servers. Our database is MySQL running on a couple of Linux server (eventually we’ll move all this over to Azure PaaS, but still running on Linux). The only production servers in Azure that we have running Windows, our the Active Directory domain controllers, one of which also syncs from Active Directory to Azure Active Directory to handle our sign in, Office 365, etc. That’s it. Everything is Linux.
Our lab environment in our CoLo is also a mix of Windows and Linux. We have a few tools that were built by Microsoft that we run that are running on Windows, but we’ve also got a decent amount of Linux in the data center as well. By the time this is published (I’m writing this on the flight to the PASS Summit in November 2018) we’ll have a Docker cluster up and running as well (unless I get lazy and I don’t get up to the CoLo to rack the servers for it). This Docker cluster is Linux based as well and will let us run a bunch more Linux servers as well.
Your point is?
The point that I’m trying to get to in all of this is that if you are a database administrator that thought they were going to stay in the Windows world forever, think again. You have to be an expert in Linux to manage these systems, but you’ll need to understand the difference between Windows and Linux. SQL Server has a few differences between the platforms, and these differences are significant to the platforms. As a Windows DBA you’ll want to be able to navigate the Linux Operating System, and tell your system teams where SQL Server is storing the database files (they are in /var/opt/mssql/data if anyone asks) so that they know which mount points need to be made bigger.
You don’t need to know everything, but the basics of Linux are doing to take you a long way.
At 11 am Pacific /2 pm Eastern on April 5th, 2019 DCAC will proudly be presenting the webcast “Power BI: Where Should My Data Live?” which is being presented by our own Meagan Longoria and Kerry Tyler.
Power BI provides many options for acquiring and creating datasets. Do you know the difference between an Imported Model, a Live Connection, and DirectQuery? Have you tried Composite Models? Are you aware of the newest features related to Power BI Dataflows and XMLA endpoints? This webcast will review your options for where to store data and explain the factors that should be used in determining what option is right for you. Obvious requirements such as data size, license costs and management, and desired data latency will be discussed. We’ll also talk about other factors such as the desire for self-service BI and avoiding data model sprawl. Join us for this webcast to learn how to determine the most appropriate type of Power BI dataset for your use case.
Click through and sign up for the webcast today. You’ll be able to download a calendar entry to remind yourself about the webcast. See you on April 5th.
If you register for the webcast and you can’t make it, fear not the session will be recorded and available for viewing after the fact as well.
I was recently cleaning the house when I got a phone call from a client that I hadn’t heard from in a while. Their SAN had decided that it was going to corrupt one of the LUNs. This LUN happened to be the Log drive on one of their SQL Servers.
Needless to say, SQL was down.
We created the LUN, and created the folders on the drive, and started SQL. All the databases on the server showed “Recovery Pending” as their state in Management Studio.
Manually bringing the databases online wasn’t going to work as that caused error 5181 which reads:
Could not restart database MyDatabase. Reverting to the previous status.
Apparently, that wasn’t going to work. I was going to need a bigger hammer, in this case, a much bigger hammer. That bigger hammer was emergency mode and checkdb.
The users would be back on the system in about 8 hours, and the users, in this case, were lawyers, some of who might need to be in court in the morning, and the system was down wasn’t an excuse that was going to fly in front of a judge.
Since I had 169 databases to bring online, a script would be needed to handle this. That script was pretty rough, but it got the job done. Thankfully no users would be using the database, so there “should” be no transactions that were in processes. We wouldn’t be able to recover them anyway.
declare @name varchar(max)
declare @sql varchar(max)
declare cur CURSOR for select name from sys.databases where state = 3
fetch next from cur into @name
while @@FETCH_STATUS = 0
set @sql = 'alter database ' + @name + ' set emergency, single_user
dbcc checkdb (''' + @name + ''', repair_allow_data_loss)
alter database ' + @name + ' set online, multi_user'
fetch next from cur into @name
The script ran, all the databases came online (it took a while to run, the databases on the server are about 9 TB in size) and all was well.
Employers and employees have by their nature a very trusting relationship. The employee trusts that the company will keep paying them, and the employer trusts that the employee will do the job that they are being asked to do. The employee also trusts that the employer will act professionally. This trust includes NOT selling the employees personal data to the highest bidder (or any bidder for that matter). Companies out there do this to their employees, and they shouldn’t be doing it. The small amount of revenue is not going to offset the loss of trust that employees have in the company they work for.
Selling customer data is a horrible enough practice, but companies can stand behind their EULA which probably says something about the person using the service is the product to be sold, and the actual customer is the company that is buying data (you didn’t think Facebook wasn’t selling every bit of data they gather?). Selling employee data is a whole-nother mess to deal with as employees are going to have no way to opt out of it, and no way to stop it. Even leaving the company isn’t going to help, because once the data has been sold, it’s gone, and you have no idea what the purchasing company will be doing with it (here’s a hint, they’re probably going to sell it).
If you’re someone who’s in a position to approve or deny these sorts of sales of employee data, the answer is “no” — every time, with no exception. Your employee data shouldn’t be for sale, and I’m guessing that your employee base is going to be some pretty low-value data to purchasers, as once people find out how/why the purchasing company got their data, I’m guessing the people won’t be purchasing anything any time soon.