martoncik |
Is it an LTO1 or 2 drives in what type of library? When you say small files, what is an average size? What type of servers are you running on the network. Is it an SAN environment? Stuff like that
ve3ofa |
So you are looking at 12 hours to do a full backup.. So do a full backup once a week or once a month and then do incrementals inbetween..
Investigate the problem.. is it network traffic (collisions?) is it the particular server having problems? Is it file access overhead?
Are you doing a volume image backup or a file backup? I’ve found that with a huge amount of small files a volume image type backup is a lot faster than a file by file backup (many orders of magnitude)
For example, if I get the freedb database and unpack it (millions of small files) it may only take up 10G but take hours to unpack.. or even to delete the directory once it is unpacked but an image backup takes minutes compared to hours.
RoyatRCDC |
I suspect the root problem cause is the high number of files. As others have noted, backups perform much better with a small number of large files than vice-versa.
You could rule out the tape subsystem by doing a (partial) backup to disk, (running it long enough to determine the thruput rate).
If the problem is the number of files, the task is harder. I had one instance when backups were impacted by thousands of EMPTY files (a bad Websphere app that didn’t delete it’s TEMPS). You may need to analyse the mix of all those files. I don’t know of a solution for making Windows navigate through it’s file system faster. Possibly massive cache?
poppaman2 |
Further to what Jim R. said - you might also try slowing down the SCSI or SATA throughput speed on the SCSI. SATA or RAID controller: If you watch your system boot process, just after the system BIOS loads, the controller BIOS(es) will initializem, and you will be presented with a message such as Adaptec’s “Press ctrl + A to enter SCSI Select Utilities” (the message differes model to model and/or brand to brand, of course). Take a look at the SCSI setup (usually in the “advanced” section: you will be presentes with a matrix of options, among them the transfer rate of the individual channels. Set the transfer rate for your backup device(s) to NO MORE THAN the maximum burst throughput figure for the device (available most times from the vendor…). This may help.
Also - check and verify cabling and termination. I know it sounds stupid, but you would not believe how many strange issues are resolved by replacing a simple cable or terminator…
RoyatRCDC |
You didn’t indicate the nature of the files, but if they represent MS Exchange data the following tip might help.
Try concurrent Exchange backups
Rick Cook
05.27.2003
Rating: -3.12- (out of 5)
Microsoft Exchange is one of the most popular business messaging and collaboration tools in today’s enterprise. However when it comes to backing up Exchange, the Microsoft Exchange Server has some limitations that produce significant bottlenecks.
The problem lies in the Exchange Backup API, which all third party backup products must use to access the Exchange databases. This API is single-threaded, which can throttle performance of a fast backup system.
Given a sufficiently fast backup system, there are several ways to improve backup performance with Microsoft Exchange. One of the most straightforward is to back up multiple Exchange databases simultaneously. The backup operation for each database runs its own instance of the Backup API, which considerably speeds the backup — assuming the other components, such as tape drives, disk performance and communications links are up to the task. Given the appropriate backup software, you can use a single tape drive by interleaving the data on the tape.
Hewlett Packard discusses the problems of backing up Microsoft Exchange and the possible solutions in the context of its Ultrium tape system in a white paper titled “Performance Tuning Microsoft Exchange 2000 using HP StorageWorks Ultrium 460 and Veritas NetBackup 4.5″.
Storage, Commvault, SAN, Tape drives/Libraries
I am having problems with backing up a 2 volumes from my HP MSA1000 SAN to an LTO tape library. The volumes are 80 & 90 GB in size and contain around 600,000 small files in each set. The 2 volumes are RAID 5 and I am getting about 5-8 GB per hour back-up. I am also backing up other RAID 5 sets and getting much better performance. I am running Commvault V5.0 and back up through a dedicated Gig LAN. Can anyone offer any advice please? 
0
