update problems

Shortcuts, Nightmares and Reboots

I’ve just recently read a couple of posts in a newsgroup which reminded me of a few nightmares I’ve created for myself as a result of taking a shortcut (or two).  One of the situations was regarding changes made to a working server (…and of course one doesn’t want to reboot said server after simply adding some software — especially when the software doesn’t specify rebooting).  The story I read went something like this:  a few months ago a demo application program was loaded onto the server, the system was not rebooted after install, all programs worked fine - newly installed demo and existing programs - and they continued to work until the system was rebooted. 

It seems that in this case there were “updates” loaded onto the system which upon the reboot replaced the working versions of the dll’s with incompatible versions.  The “demo” program had been reviewed months ago, determined that it wasn’t desired, but left on the system anyway.  By the time of the system reboot the consultant called in to address the problem was of course told “Nothing has been changed!”  Yes, we did have to take the system down for an extended power outage — but it booted fine - “…no errors…”. 

It took a team 2 1/2 days to establish the cause of the problem - that being that at the time the “demo” software was loaded certain dll’s were being “used” and therefore would not be replaced until the next reboot of the system.  By the time that reboot happened nobody ever suspected that the “demo” software load was the cause of the problem.

Had the system been rebooted after the install the “issue” and its cause would have been immediately identified.  Months later, that wasn’t the case.  The shortcut? Not rebooting after the install.  The nightmare?  2 1/2 days of anguish with printing not working.  The solution?  Reboot after loading new software.