A week ago, the latest post to the Windows 8 blog was entitled “Virtualizing storage for scale, resiliency, and efficiency.” And while the whole thing is definitely worth perusing, I want to zoom in on something that catches my particular fancy in this blog (it also caught Word and Windows guru Woody Leonhard’s attention in the latest Windows Secrets newsletter, too, which is what initially brought it to my attention; his piece is called “Storage Spaces might be Win8’s best feature yet“).
But first, I have to explain why I think the Storage Spaces feature is so awesome, by hearkening back to another hobby horse of mine — namely, Windows Home Server, aka WHS (I’ve blogged five times on this platform right here: once in 2008, and twice each in 2009 [1,2] and 2011 [1,2]). Turns out that Windows Server used to support a feature called Drive Extender that essentially allowed you to shrink or grow a multiple-disk logical volume simply by adding or removing space, then doing likewise with hard disks, on the system. No need for migration, recopying, partition re-sizing, Disk Management, and so forth: just a few commands to tell the file system what’s going on, and smooth sailing throughout.
I was incredibly bummed when it became clear that WHS 2011 (the latest and greatest version, out last year) did not continue earlier versions’ support for this feature, and thought perhaps we might never see its like again in the Windows World. I was wrong! In Windows 8, Storage Spaces lets users set up logical volumes that are bigger than the combined disk space provided for them, so you can grow the physical space over time to match the logical definition (Microsoft calls this “thin provisioning”). Here’s how Rajeev Naga, a group manager for Microsoft’s Windows 8 Storage and File System team explains what Storage Spaces can do:
Organization of physical disks into storage pools, which can be easily expanded by simply adding disks. These disks can be connected either through USB, SATA (Serial ATA), or SAS (Serial Attached SCSI). A storage pool can be composed of heterogeneous physical disks – different sized physical disks accessible via different storage interconnects.
Usage of virtual disks (also known as spaces), which behave just like physical disks for all purposes. However, spaces also have powerful new capabilities associated with them such as thin provisioning (more about that later), as well as resiliency to failures of underlying physical media.
And if and when you exceed the original maximum allocation for this storage pool, Storage Spaces simply notifies you in a pop-up window that you need to add more capacity, which you can do by plugging in more disks, then allocating them to that pool. Even cooler, pools can be designated as mirrored so that there will always be at least two (possibly three) copies of data on different physical disks within that pool. And should any disk fail, Storage Spaces automatically regens data copies for all affected holdings as long as enough alternate physical disks remain available within that pool. It’s not RAID, but it is fault-tolerant and resilient, and a great technology for all kinds of SOHO and end-user applications. The parity designation also permits data to be reconstructed in the face of physical disk failures. They work best for big, relatively static files (like music, movies, home videos, and so forth) which are sequential in nature and require little or no updates to existing files.
Here’s how to work with Storage Spaces. You can use Windows PowerShell commands to create a storage pool, and to set up Storage Spaces that reside in that pool, or you can use the Storage Spaces item in Control Panel, to go through that process. This means selecting drives, then naming a storage space, assigning a drive letter, establishing the layout (default, mirrored, or parity), and assigning a maximum size. Afterward, you can always add more drives as needed, or make changes to configuration data likewise. Right now, the Win8 Developer Preview only supports storage spaces up to 2 TB in size, but when the Win8 Beta ships in February (next month, that is) you’ll be able to create storage spaces of arbitrary size.
What a great feature to bring to the Windows desktop. This may indeed be a true “killer feature” that prompts users to to upgrade to the new OS to exploit its capabilities, particularly those with big media collections!
On November 28, I wrote a blog entitled “PSmartDeploy Helps Ease Heterogeneous OS Situations,” in which I promised a follow-up early this year to provide more technical details about how the company makes a science of the slippery art of matching Windows drivers to Windows OS installations. I got my info fix on this topic, and more, in a conversation with SmartDeploy’s Director of Sales, Spencer Dunford (an uncommonly technical sales guy, if ever I ran across one) and his colleague, Senior Systems Engineer, Erik Nymark, this past Monday (January 9, 2011).
The patented technology that SmartDeploy uses for its driver magic makes perfect sense, and probably reduces a lot of painstaking work and serious reverse engineering, to build on the great work that Microsoft has done with the Windows Pre-Installation Environment (aka WinPE) and its boon companion, the Windows Image (WIM) format and container mechanism that Windows Vista, 7, and 2008 Server versions use to manage OS installation and packaging of image information for same.
Prowess SmartDeploy uses WIM as a storage format and data container for all of its packaging and image presentation methods. It has worked its way deeply enough into WIM internals, in fact, that it does the following to deliver WIM data elements for use in actual deployments (where individual components get combined into a valid WIM container that’s used as the focus for image installations):
- Hardware/device drivers are broken out into a separate component file that can be built and managed independently of OS files and components. These are known as “platform packs” and confer some interesting advantages from their use: they are based on device enumeration from a reference image, and therefore need only contain those drivers that some particular Windows installation actually needs, rather than a sizable galaxy of possible device drivers that any foreseeable Windows installation might need. Thus for example, the DriverStore folder in my Windows 7 installation is 1.07 GB in size (which represents possible drivers) while the Drivers folder is under 58 MB in size (which represents the drivers it’s actually using). Essentially, the SmartDeploy approach allows the OS build to zero in only on the drivers it needs, and omit everything else, to achieve considerable image size reductions.
- The OS and other components travels along for deployment as well, and remains logically and physically separate from the driver stuff. This makes it alot easier to mix and match various platform packs with the standard OS portion to support environment where as many as 60 or 70 different reference images must be supported.
- Perhaps most interesting, SmartDeploy can also support “delta WIMs” so that an existing image can be merged with a set of changes (new updates, service packs, and so forth) without having to transport the entire image from a deployment server to its deployment targets. Thus instead of moving typical images that can be anywhere from 5 to 25 GB in size to replace an old image with a new one, the existing image can be updated in situ with the SmartDeploy tools and laid down to replace its previous incarnation entirely in place. This often means that delta WIMS of 500 MB to no more than a couple of GB in size will do the job quite nicely.
Misters Dunford and Nymark also showed me some very cool demos that showed these technologies at work, and performed a live update of a Windows 7 Professional image with a delta WIM that took less than 15 minutes to complete, and even less time to set up and stage for deployment. SmartDeploy makes excellent use of Wizards to structure initial images, and to build deltas upon those images, and also includes some slick automation for creating answer files to perform post-install tasks for adding applications and other runtime components to OS images as they get deployed to target machines.
The biggest benefit of their tools and technology is that, according to Dunford, they “…have seen a change in how frequently and how often images are updated, and patches or changes are deployed in image format.” Anything that makes this kind of job easier and faster to accomplish is definitely worth digging into. Check out the SmartDeploy website for more information.
Among the many elements in my diagnostic and repair toolkit, I have a licensed copy of Steve Gibson’s SpinRite v6.0, purchased in 2006 (I still have a copy of the receipt in my email archives, and I’m glad I did for reasons I’ll elaborate upon later). I’ve been messing around with a very nice little 2.5″ drive cage that parks itself in a standard 5.25″ desktop PC drive bay — see this blog at edtittel.com “Great Product for Recycling 2.5″ Notebook Drives” for more info on that recent adventure. This led to some fiddling about with those recycled drives to transform them from OEM and/or home-built system drives with repair and hidden partitions into standard, single-partition data drives. And, in turn this led to the need for some low-level repair on one of the drives involved (luckily for me it was in the disk area devoted to a repair/recovery partition that I had never used, or it surely would’ve made a bad situation worse). And finally, to close this circle, that’s what led me back to SpinRite.
But, as usual with Windows, there were a couple of interesting flies in the ointment to content with. One is apparently a problem with my production desktop (a prospect that never fails to darken my day), where it simply refused to recognize the FAT-16 bootable UFD upon which SpinRite is installed. When I stuck it into my production system to make sure the UFD was still OK, I ended up wandering into an interesting hall of mirrors. This system doesn’t see the UFD (that’s MS-jargon for USB Flash Drive, in case the acronym is unfamiliar to some readers), except as an anonymous entry in the Disk Management utility:
And although USB gives me the cheerful French horn blast when I stick the drive into that machine, I don’t get the usual drive recognized pop-up window that asks me if I want to do something with it, either (as you’d expect, since the file system apparently doesn’t recognize the drive as a legitimate disk voluime).
But where it gets interesting is that another of my 32-bit Windows 7 systems and a 64-bit system DO recognize the drive, see it as a legit volume, and happily assign it a drive letter as well:
Once I was able to resolve the contents of the SpinRite UFD and ascertain its contents were still complete and correct (easy enough to resolve by downloading the files, and extracting them to disk to compare file names and file sizes, which is where my packrat tendencies to keep all receipts allowed me to grab a new copy from the Gibson Research Website and do due diligence), I was ready to try repairs on my slightly wonky WD 500 GB notebook drive. (Alas, I’m going to use this apparent problem with the file system on my production desktop to spur me into the upgrade to 64-bit that I’ve been putting off for too long now. I’m not sure I know how to solve this kind of file system issue, and it’s probably easier just to wipe the system drive, upgrade from 4 to 12 GB of RAM, and move up onto a more capacious working environment.)
That’s when I hit the next bump in the road: SpinRite would load FreeDOS and get going but it would never get past the “looking for storage devices” phase of its start-up on my test machine. A little quick research quickly revealed that not all BIOSes are created equal, and apparently the one that gets loaded when a PC boots into AHCI (or RAID) mode is not compatible with the BIOS that SpinRite needs to do its thing. With a little trepidation, I made the switch, then booted back up into the UFD. This time, SpinRite ran fine, and after about 6 hours of repair, fixed (or routed around) what ailed my errant disk drive. It seems that the AHCI and RAID BIOS developers cut enough corners in building their versions of the Basic Input-Output System that Mr. Gibson can’t use it to dig deeply and deviously into a drive’s contents with a disk controller that employs such a BIOS. After problems I’ve had with SATA-IDE vs. SATA-AHCI booting issues, I heaved a sigh of relief after switching back to AHCI on the next reboot to my system drive and seeing everything work properly.
But in the end it all turned out to be easy enough to work around, thanks to the presence of multiple machines I could use to figure things out along the way. Now, I just have to find time to upgrade my production desktop to 64-bit Windows and life will be good. Yeah, right!
Earlier this week, I found one of my Windows 7 test machines in what can only be described as an “interesting state.” For whatever reason (I’d just finished updating a bunch of hardware drivers, and I suspect that one of them bit this OS installation in the hindquarters) the OS would no longer interact with Windows Update, and the file system interface through Windows Explorer was decidedly flaky. To make a long and tedious story short enough to be tolerable, I ended up rolling back to a system image dated just before the December 13 Patch Tuesday updates before that balky test PC returned to a stable and workable state. I found myself wishing for some mechanism to put my system in good order that would be better than a trial-and-error march back in time through restore points and system images until something finally clicked.
Desmond Lee, the program manager for the Windows 8 Fundamentals team, must have often entertained those same thoughts as he went about writing this January 4 post to the Building Windows 8 blog, entitled “Refresh and reset your PC.” Of course, this capability is focused on Windows 8 rather than Windows 7, but in the wake of a trying repair and recovery scenario, it was reassuring to read that my recent situation isn’t unique, and that people do wish for a quick and easy way to make this possible. In fact, Lee’s list of key objectives for the “refresh and reset” initiative are likely to strike chords with you readers, as well as resonating forcefully with me (I quote them verbatim):
- Provide a consistent experience to get the software on any Windows 8 PC back to a good and predictable state.
- Streamline the process so that getting a PC back to a good state with all the things customers care about can be done quickly instead of taking up the whole day.
- Make sure that customers don’t lose their data in the process.
- Provide a fully customizable approach for technical enthusiasts to do things their own way.
In fact, the approach to providing this functionality in Windows 8 was to provide what Lee calls a “push-button” technique for repair and restore operations. This is by no means a new concept but no matter how urgently platform and product providers have hyped such solutions in the past, my own personal experience with such tools has been that they’re invariably simpler in concept than in practice. Sigh.
To that end, Windows 8 will apparently ship with a reset option that returns a system entirely to factory-fresh set-ups and settings, and removes all personal preferences, settings, data, and so forth so that it can safely be turned over to a third party without incurring the risk of unwanted data disclosures. The refresh option is more interesting to me at the moment, in the wake of my recent “Where and when did things go wrong?” adventure. The idea here is to “…get the benefit of a reset – starting over with a fresh Windows install – while still keeping your stuff intact…” In this case the Windows Repair Environment (Windows RE) copies “…your data, settings, and apps, and puts them aside…” before installing a fresh copy of Windows, then carefully restores all of that personal and personalized stuff onto a newly-installed and up-to-date copy of Windows. That said, file type associations, display settings, and Windows Firewall setting are NOT preserved as a matter of conscious design to avoid problems arising from mis-configurations in the previous Windows incarnation.
But alas, there’s a catch: where apps are concerned, Windows 8 will preserve only Metro style apps, “… and require desktop apps that do not come with the PC to be reinstalled manually.” In my own recent personal experience, where installing Windows and getting it ready to use may take two to four hours, re-installing all the applications I use (which varies from a low count of around 50 on test and traveling PCs to a high of 110 on my production PCs) can take the better part of a day to chunk all the way through to completion.
So, while this sounds like a partial answer to my recent problems, it’s still not a 100% solution, either. And, of course, it will be very interesting to learn how these goals translate into practice as the Windows 8 builds make their way into GA status. Stay tuned!
On October 29, 2011, in a blog entitled “Win7, XP Reach Crossover Point,” I reported that StatCounter’s tracking indicated that on or about October 14, the number of PCs on the Internet running Windows 7 exceeded the number of PCs running Windows XP for the first time. Of course, not everybody agreed with those numbers (as evidenced by Ed Bott’s discussion in my November 2 blog “Windows 7 vis-a-vis Windows XP“) where many actually prefer the tracking that NetMarketShare does instead because of its global purview and better detail.
Recently, reports indicate that the crossover point for the two operating systems — that is the now-venerable, 11-year-old Windows XP that simply refuses to die, and the latest production Windows 7 OS that is still finding its legs even as the shadow of the next-and-future Windows 8 OS begins to make itself known — are imminent, according to NetMarketShare. Take a look at these graphs and data points:
If I read the graphs correctly, the NetMarketShare trend lines for Win 7 and XP indicate that the crossover point will occur in late February or early March, as the two numbers converge in the neighborhood of 40% share for each one. And with Win7 on the way up as Windows XP is on a slow but steady downward course, this finally puts Windows 7 in the driver’s seat.
What’s interesting to me is that different forms of measurement produce the same results (Windows 7 overtaking Windows XP) but at dates as much as half a year apart. It’s all in how you measure, what you measure, and where those numbers come from!
Imagine my surprise when I sat down to my PC late morning on January 1, to see that Microsoft had pushed a security update to address “Vulnerabilities in the .NET Framework…” (MS11-100, rated Critical). This involved as many as three security patches on some of my PCs, depending on how many version of the .NET Framework were installed on those machines (for Windows 7 patches were released for versions 3.5.1 and 4; for Windows XP, patches also appeared for versions 1.1 and 2.0, as well as 3.5 instead of 3.5.1).
Because this update addresses zero-day vulnerabilities in ASP.NET that can lead to elevation of privilege for rogue software and execution requests, this is a patch that admins will want to fast-track through their internal testing and deployment procedures. Ryan Naraine (of the Zero Day ZDNet blog) explains why this one is worth special treatment in his post this morning entitled “Microsoft ships emergency .NET fix to thwart hash table collision attacks.” See KB Article 2638420 for a list of “known issues” related to deploying this particular security patch (basically, it involves updating all servers that use ASP.NET authentication tickets concurrently, because the pre- and post-patch mechanisms are incompatible).
This past summer, I had to rebuild one of my test Windows machines when the original motherboard went south. When doing the rebuild, the machine crashed during the reboot part-way through the install process, and forced me to switch from AHCI to IDE mode for the disk drives I was using, even though they’d worked fine in AHCI on the previous motherboard. I wrote it off to some issue with AHCI and my combination of parts, and simply stayed with the switch to IDE and figured that would be the end of it.
But when I was finally able to get the proper IDE drivers loaded for that machine this weekend I decided to revisit the AHCI issue on this Gigabyte P43-ES3G. I like to tinker with my systems over the holidays, and have just finished a major patch-fix-upgrade-and-repair pass over all my PCs now; the disk controller stuff all started working when I switched the BIOS from “emulate IDE” to “straight SATA” just to see what would happen. IDE kept working, but DriverAgent was suddenly able to help me find the right, current drivers for the Intel ICH10R chipset and the JMicron JMB36X SATA/IDE controller on the P43-ES3G. “What the heck,” I figured, “Let’s try AHCI now, and see what happens.” It still kept hanging during drive detect while booting.
With some assiduous poking around, I discovered that others have had issues with the BIOS hanging during the drive detect just as I have, including a variety of Asus as well as Gigabyte motherboards. As it happens, the reason I had to switch from AHCI to IDE drive mode was because drive detection would hang on the second of the two drives in that system (a Samsung 1 TB HD103UI 7200 RPM hard disk) after correctly detecting the WD 300 GB Raptor that serves as the boot drive, but before detecting the presence of the SATA DVD burner on the system.
Just for grins, I tried a different drive instead of that Samsung yesterday while fiddling with the machine, after which AHCI booted like a charm. Upon further investigation I came across a posting on social.technet.microsoft.comfrom a gentleman named Ivan Filippov –who just happens to work for German-based disk formatting and partitioning company Paragon Software (whose Hard Disk Manager Suite has long been a favorite of mine) that explains a possible cause for this situation. He observes that when the disk geometry data in the first partition on a drive gets munged, it can cause disk recognition at the BIOS Level to fail. The two other drives I tried to replace the original Samsung didn’t have this problem, apparently, and the drives were recognized and the AHCI BIOS loaded successfully–and so did the Samsung itself, after I popped it into a SATA drive caddy on another machine (after backing it up, of course) and repartitioning and reformatting that drive.
Problem solved, and I learned something both interesting and valuable. I should’ve just tried another drive (I’ve always got at least a couple of spares around, sometimes more than that) when I first hit this problem and it pretty much would have solved itself. And so it goes! Another obscure but interesting Windows lesson learned, and another pesky annoyance rubbed out at last…
[Note to readers: I’m taking the rest of the year off, so you won’t see me post again until January 2, 2012. Let me take this opportunity to wish everyone a happy holiday season, and a festive and prosperous New Year!]
Secunia posted Advisory SA47327 earlier this week, which explains that a specially-crafted Web page instantly crashes 64-bit Windows 7 Professional running the Apple Safari Web browser for Windows. According to the advisory “The vulnerability is caused due to an error in win32.sys and can be exploited to corrupt memory via e.g. a specially crafted web page containing an iFRAME with an overly large “height” attribute viewed using the Apple Safari browser.” The bulletin goes onto say that “successful exploitation may allow execution of arbitrary code with kernel-level privileges, presumably in the form of some post-crash recovery executable. The original discovery (and an instance of an “overly large value” appears in a Twitter post from 12/18/2011 by @w3bd3vil aka “webdevil”).
Here’s a novel workaround for those concerned about this vulnerability until Apple comes out with a fix: turning on Developer Tools in Safari apparently eliminates iFRAME support (see this Apple Support Communities discussion: “iFrame works until I turn on developer tools in Safari…” for more information and instructions). OTOH, hackademix.net states that “Safari can’t be secured 100% against clickjacking” so one had better hope that this workaround truly turns off iFRAME altogether (hurry up testing on my three PCs with Safari installed appear to confirm this, albeit in a very small sample).
OK, so lately I’ve been both a little miffed, and more than a little curious, because my Windows 7 problem reporting hasn’t been working properly. The error message information to explain why not is pretty darn cryptic and turned up a big, fat goose egg — precisely nothing, that is — when I attempted to run the problem down. Here’s what I mean:
The diagnostics program helpfully offers the options to change error reporting settings, but they’re all greyed-out because they’re not user accessible. So off I go, haring into gpedit.msc, to see if jacking around with Local Computer Policy will help. Following tips on the Internet apparently related to this problem (‘Windows Error Reporting Doesn’t Work” or “Group Policy error on Windows error reporting”) I try changing settings in Administrative Templates \ Windows Components \ Windows Error Reporting. Nothing helps.
More Internet research tells me that the start-up and troubleshooting management tool, Soluto (which I use and have blogged about here repeatedly), might be at fault. Because a new update to the Soluto software is now available — which makes it smart for me to uninstall the old version before trying out a new one, as I’ve learned to do from previous major revs of their software — I try running the “Check for Solutions” item in System Center \ Maintenance with Soluto installed, and then again, without it installed.
Bingo! That difference also makes the difference between Windows error reporting (and Check for Solutions) working when Soluto is absent, and presenting the Group Policy error screen when Soluto is present. Very interesting! I certainly hope Soluto is aware of this issue and planning to fix it soon–hopefully in their upcoming first commercial release, scheduled for later this year.
[Follow-up note 12/22/2011: Checked a couple of 64-bit Windows 7 machines also running Soluto, but not the latest version: one rev back. No issues with MS problem reporting or solution lookup on any of these machine. Even more interesting! I’m going to update one of these machines and see if the problem is specific to the 32-bit version only, or affects both 32- and 64-bit machines.]
[Second follow-up 12/23/2011: Just installed the latest rev on two of my 64-bit test machines. Prior to the rev, MS Problem reporting and solution lookup worked fine. After the rev, it showed the same group policy error that also appeared on my 32-bit machine as well. Ladies and Gentlemen: we appear to have a genuine culprit!]
On December 15, 2011, Paul Thurrott of Supersite for Windows appeared on Windows Weekly 239 with Mary-Jo Foley (another regular commenter on Microsoft and Windows operating systems and technologies). My specific area of interest here is the company’s release of iOS apps for Skydrive, Lync, and an update for OneNote 1.3 for the iPad (iOS apps for Kinectimals, Bing, and an iOS connector for email are also available as well).
Of course, it’s still the case that the deepest and most effective support for MS applications occur on Windows Mobile platforms (but because this occurs at the OS level) sometimes the non-Windows mobile platforms get such support sooner than major Microsoft releases can manage. As Thurrott points out, Microsoft is not to tightly aligned that it can push out mobile OS releases in synch with features under development in various part of the company. Some Windows enthusiasts (bigots?) have expressed displeasure that iOS in general (and iPad in particular) have gathered features and functions that might not yet have appeared on mobile Windows platforms. That’s why it’s still the case that Xbox Live is better integrated into Windows mobile platforms, as is the entire Windows Office suite.
It’s going to be interesting to see how this unfolds. Given the increasing importance of mobile apps, and the proliferation of platforms (and the inescapable popularity of iOS) it’s ever more likely that MIcrosoft will reach out to competitive platforms at the same time as it makes apps to run on those selfsame platforms.