Fast Deletion of Large Data Sources
I'm trying to delete 100gb - 1 terabyte here and there and haven't found a good utility to do this in any reasonable amount of time. Has anyone out there found a speedy app for this purpose? I've tried "Delenda" and "Remove Old Files" with very blah results. It wouldn't be so bad, but data is generated at a very high rate around here... then archived... then the bottleneck of deletion.
Software/Hardware used:
Software/Hardware used:
ASKED:
November 20, 2006 1:15 PM
UPDATED:
December 1, 2006 10:45 AM
Answer Wiki:
Not enough information given to formulate a reply.
Is it ONE file that is in the terabyte range? Is it part of a dataset? Why is the data being collected then archived then deleted and not kept online? Hard drives on a cost/GB is very cheap and probably one of the cheapest storage mediums available. Rather than deleting the file.. dedicate a drive for the file and then quick format the drive instead of using the delete command.
To see all answers submitted to the Answer Wiki: View Answer History.
The data being deleted is comprised of thousands of small Tif files. Drive space is cheap on the low end, but when you move into the realm of NetApp’s, more drive space becomes not so cost effective. The data is only needed for a few months, and always coming in. We create enough that we fill it up as fast as we archive and delete it.
I would like to think that someone in the Storage realm has dealt with the deletion issues I face. There must be an app that is multi-threaded and starts without a scan.
Not necessarily. Most low-level file commands include a stat() or an open() call. Anything that needs to delete a file will need to, at the very least, stat it – which takes up so much time.
If you need to quickly delete files on a NetApp volume, I’d suggest deleting/recreating the volume. It’s easy to script with NetApp.