Automated provisioning

Jim4522

820 pts.

Tags:

provisioning

Virtualization

Part one. I heard someone the other day suggest that in a virtualized world one could use automated server provisioning as a tool to automatically move an application from a down server to a bare backup server. In the context that the speaker was talking this allowed IT to insolate those being service by the problem server from being impacted by the problem and reduced the need for quick maintenance response to fix the down server. But the speaker was vague about how or who would decide when a server was down or even having a problem. From my experience recognizing that a server was having a problem was often the job of the person being serviced by the server, and it was that person working through the help desk that caused the problem to come to the attention of the central server site. Question continues in part two

Asked: September 14, 2009 11:38 AM Last updated: September 16, 2009 2:52 AM

Answer Wiki

Last updated: September 14, 2009 4:36 PM GMT

Denny Cherry68,390 pts.
History
Contributors
- Ordered by most recent
- Denny Cherry68,390 pts.

Thanks. We'll let you know when a new response is added.

When working with VMware this is handled by DRS which will automatically restart the virtual machines that were running on the host machine that failed on hosts which are still running.

DRS is an automatic process so that the downtime to the virtual machines is minimal.

The sysadmin controls the priority that the virtual machines start back up in. If no priority is given then DRS will start the machines in the order that it decides.

Discuss This Question: 4 Replies

Jim4522 Sep 14, 2009 11:41 AM GMT

Part two. Often the first reaction of those at the central server site was to reboot the server and if that failed to power cycle the server. When both those two responses failed the problem was evaluated to determine if the problem appeared to be software or hardware if it was hardware the problem was turned over to the maintenance provider (MP) whose job was them to diagnose the problem, about 20% percent of the time the MP found no problem and the hardware was returned to operations management. Finally the question: At what point in the virtualized world is automated provisioning triggered and who triggers it. In the past the end user was the one who noticed that the server was having a problem. Is that end user still the one who starts this process or has some new monitoring and analysis routine triggering this response automatically?

820 pointsBadges:

report
Jim4522 Sep 14, 2009 5:36 PM GMT

Mrdenney, I am not sure if you read the second part of my question. I know that the wokload of a failing server can be sent through resouce pool sharing to another server but how did you know the first server failed? If you know it failed and it failed as a result of a hardware failure moving to another server solves the problem, but if the first server failed because of a software problem moving that software to another server doesn't seem to solve the problem it just sets up the second server to fail. I watched a pool of 10,000 servers function for a year and in 3229 instances in one year it was the user of the server that figured out the server was having a problem and not the people in the central server pool. They only were made aware of the problems after they were contacted by the help desk. Of those 3,229 problems a third were resolved by just rebooting or if that failed powercycling the server, if you automatetically sent all 3,229 problems off to other servers you would flood the pool, of the two thirds that were not resolved by rebooting half of them were found to either be software problems or no trouble found by the hardware maintenance provider. The ability to monitor and diagnose problems on low end x86 servers is almost non-existant, if not existant, In my opinion. Jim4522

820 pointsBadges:

report
Jim4522 Sep 14, 2009 6:58 PM GMT

Mrdenny, I just read the VMWare manual on DRS and you are right that if you wanted to do maintenance on a server the adminstrator could move the workload from one server to another server in the same resouce pool thus insolating the end users from being inconvienced by the maintenance action, but you would have to know you wanted to maintenance that original server before you did that. Now if I get a complaint from someone on the network that he thinks he is having a problem with his server, my first thought would be to reboot the server which I suspect would solve the problem a third of the time. Would I move the workload and then reboot? No, because I won't know the reboot solved the problem until I get the user to try the server after I rebooted it. If rebooting doesn't solve the problem do I move the workload because 10 to 15% of the time I get a complaint from an end-user there is no problem, should I move my workload everytime some end user had too much to drink for lunch and hit shift instead of enter. If it really is a hardware problem and I do move the workload and the vendor does change the most likely component, let's say a Dimm card, I won't know if the problem is fixed until I retry the server again and if that Dimm card didn't solve the problem I have eleven more Dimm cards to go which is eleven more moves of workloads in both directions. Plus everytime I reseat a Dimm card I run the risk of bending some of the pins and now I have two problems. I don't think DRS is my problem or my solution. I think I need better diagnostics and better ways to monitor the health of my servers than depending on some drunk in Hong Cong to be my triggering function. Jim4522

820 pointsBadges:

report
Denny Cherry Sep 16, 2009 2:52 AM GMT

When you need to reboot a server to do a performance problem, 99% of the time it'll be that you have to reboot the guest OS not the host. In the event that the host has run out of CPU power you can configure DRS to automatically move the VMs around between the hosts to resolve the problem. As the administrator you can control how agressive DRS is at doing this automatically. If the applications and services have memory leaks, etc, those would still require a reboot of the guest OS to resolve that issue and DRS wouldn't solve this problem at all. DRS is used to keep the guests up if the host needs to be rebooted. If the guest needs to be rebooted that still needs to be dealt with just as before.

68,390 pointsBadges:

report