Bank Overcomes 50% Failure Rate with Backup Exec with Dell AppAssure’s Reliable Continuous Disk Imaging Solution, Gaining a 140% ROI in the Process

Environmental Snapshot:

• Exchange 2010: 250 mailboxes to backup
• SQL 2005 servers: 4
• Windows 2008 R2 file servers: 25
• Virtualization: 9 servers running Hyper-V
• Data total: 4 TB

Business Challenges

It’s one thing for a bank to have to deal with laborious tape backup schedules, manual rotation of tapes, and a growing backup window due to ever-increasing data. It’s quite another when the best you could hope for each month was a 50% success rate for a good backup. But that was precisely the situation that Bob Thomas, Vice President of Information Technology at Village Bank, faced month after month.

“We’re a midsized bank which also owns a mortgage company,” explains Thomas. “We have 18 locations around the Richmond area and a staff size of about 230 people. Our infrastructure includes mortgage loan origination applications, Fiserve’s bank imaging system, and client account opening software platforms all running on our SQL servers. Plus we’re a service bureau bank, which means we work with Fiserve, who does the heavy lifting of core processing of accounts payable, account management, and so on. So there’s a lot going on at any one time. We were using Symantec Backup Exec 12 and our backup jobs were continually failing, leading to questionable recovery scenarios.”

Thomas estimates that his three-person IT team had to deal with unacceptable failure rates of up to 50% – regularly.

“We would do a daily backup which ran three to six hours a night, and then a full backup on Friday, plus a monthly,” says Thomas. “We don’t have a staff to come in on Saturday, so we’d get the backup on Monday and about 50% of the time, find an error. There wasn’t much we could do about it because we couldn’t afford to rerun the backup, so we’d just note we had had a failure and hope we wouldn’t need to recover anything, because if something failed, we’d be out two weeks of data. We thought about redoing the backup schedule, and looked at another tape mechanism, but it was just too expensive for us. We’d call Symantec on the failures, but they could only recommend a patch or have us waiting for the next version.”

This nightmare scenario was made all the worse by the fact that the bank had grown so much in the last couple of years that it was expanding faster than Backup Exec’s solution could keep up with it. The backup jobs began to take longer and longer, while the increasing data took up more and more tape.

“The failing backup jobs put the company at risk when a recovery is required,” says Thomas emphatically. “Since the number of tapes and time required to do a full backup was increasing, we knew that we needed a more secure and reliable process for the bank. Not only that, we were committed to developing a comprehensive disaster recovery (DR) process, and we couldn’t do that with what we were using.”

Solution

Village Bank had its mission critical servers set up on separate tape apparatuses and kept close tabs on it for failure, yet all its other servers were still vulnerable. “If we had a disaster, the mission critical servers were ok,” notes Thomas, “But the mail server and others were very vulnerable. We were looking at 2-3 weeks of recovery time if there was a disaster, because we had to make choices about what could come back online when needed.”

“We spent a lot of time reviewing the products that are out there,” notes Thomas. “Since we run n-Central, a monitoring system from n-Able, we did a trial on the solution they suggested. But just as we were about to make a decision, we had a tape machine failure and I decided that was it. I didn’t want to reinvest in tape any longer. It was just too expensive for repeat problems.”

Thomas’s team came across Dell AppAssure, and did a trial. Within 15 minutes, Village Bank’s servers were all up and protected by Dell AppAssure in a test mode. Thomas chose Dell AppAssure.

“Our backups are running much quicker and the time to recovery is cut from hours and days to just minutes,” says Thomas. “Not only that, but the ability to go into Exchange store and restore one email with Dell AppAssure has been eye opening.”

Results

Thomas can easily make before and after comparisons:

Recover in Minutes. “Before Dell AppAssure, if a file was lost or accidentally deleted, we would have to request the tape from an offsite storage facility and wait for the courier to deliver it. This meant that it wasn’t at our data center until the next day. Once we received the tape we would have to run the restore job, which could take another hour or more. Now this process literally takes only minutes. This is like night and day.”

Server-Level Recovery. “While we haven’t had a complete failure – I’m new to the virtualized world – we did experience a glitch with a host server having disk space taken up as we were virtualizing. Luckily we had some Dell AppAssure image snaps ready and available. I blew up the virtual image and restored from the last good recovery point. This took less than an hour and I did it from my house in the evening. I didn’t have to be in the office swapping tapes like before.”

More Storage Capacity. “Based on the current readings of our Dell AppAssure console, we are at 27.19x storage savings. But this isn’t quite accurate, because now we’re able to back up much more data than we were before! Now it’s not just our one mission critical SQL server, but all our machines!”

Great Return on Investment. “Calculations are showing a total savings of $88,859 with a 140% ROI. But that’s not what’s so important to me. I said to my boss, ‘Here’s the cost to stay in 1980s technology now that our tape drive has failed, and we’re in the 21st century.’ The ROI is just a bonus.”

Increased Staff Productivity. “Dell AppAssure has freed up staff time by eliminating the manual process of regularly logging, couriering, and reviewing tapes, which took about 1-2 hours a day. We now have time to think about other things – we can better plan, execution goes a lot easier, and we’re not always fighting a fire. We no longer think about backups because Dell AppAssure just works.”

Village Bank can now feel confident that its data is protected as it looks to the future. “Dell AppAssure has completely changed our DR strategy because we have the time to focus on the future,” says Thomas. Village Bank can now meet its goal of doing replication off-site, which is what is up next on Thomas’s team’s plate.

Recently, a user deleted a large number of folders, representing about 15GB of data, off the Bank’s shared drive. With Dell AppAssure, Thomas recovered the data in less than five (5) minutes. “The bottom line,” says Thomas, “is that Dell AppAssure lives up to the hype – the literature is point on. I would encourage anyone who is looking for a good backup and disaster recovery solution to take those words to heart. Dell AppAssure does what it says it does.”

About Village Bank

Headquartered in Midlothian, VA, outside of Richmond, Village Bank is a midsized community bank which has served the Richmond metro area for over 11 years. In addition, Village Bank’s mortgage company, Village Bank Mortgage Corp., has offices in Midlothian, Manassas and Hanover, VA.