NetBackup OST Delivers Unexpectedly Good Results; User Testifies to Amazing Gains in Backup Performance

By now most enterprise backup users have heard about Symantec's new Open Storage (OST) API that was included as part of the Veritas NetBackup 6.5 release in August 2007. However the full benefits of OST are still largely unknown mostly because so few users are taking advantage of them. This is due, in part, that to realize the benefits of OST, disk vendors had to add an OST software component to their systems in order for NetBackup to manage them. Now that more systems have added this feature, Symantec recently held a round table discussion during which it shared some of the progress that has occurred around OST with one early adopter on the call sharing a pretty amazing story about the performance gains that he has seen in his production environment using OST.

A recent round table moderated by ESG's Lauren Whitehouse and attended by representatives from Data Domain, FalconStor, Quantum and Symantec took some time to explain both what purpose that OST serves and how companies are leveraging OST on their devices to improve the performance of their backups.

Mathew Lodge, Symantec's Senior Director of Product Marketing, explained that OST is a means by which organizations can use "intelligent data protection devices" in their Symantec backup environment to reduce costs and improve performance and recovery capabilities. When used in this context, "intelligent data protection devices" refers to disk-based storage systems like virtual tape libraries (VTLs) that have capabilities that extend well beyond just acting like a physical tape drive or tape library.

These disk-based systems may have different features (different tiers of disk, replication, etc.) that organizations would like to use but unless these features are exposed to NetBackup, it cannot manage or take advantage of them. Instead, it must revert to treating them as the lowest common denominator - tape. Further, if NetBackup does take advantage of features like replication found in these disk-based data protection devices, it needs a mechanism to update its catalog based upon the data movement going on in these devices. Lodge says, "This was the genesis and the drive behind OST - it was a way to connect intelligent backup devices to the Symantec backup environment and stop pretending that they were tape."

The initial adoption rate of OST in high end accounts is notable considering that the technology is still fairly new and that organizations need to pay a licensing fee to both Symantec and the provider of the disk-based storage system. The disk-based vendors participating on the round table had the following to share regarding the adoption rate of OST in their accounts:

• Data Domain finds among its customers that its OST plug-in has been adopted more widely than its VTL interface on its high-end systems.
• FalconStor has also been shipping the OST plug-in for its systems since 2008 and it has been widely adopted among its large enterprise customers. These customers deal with multiple sites and have very large NetBackup installations that are looking to take advantage of the features that its VTL has but making NetBackup aware of them. They want to ensure NetBackup knows about the multiple copies and the offsite copies that their VTL creates.
• Quantum sees three use cases for OST: tiered backup, DR and tape consolidation, and uses it as part of its edge-to-core configuration on its DXi solution. OST has helped Quantum's customers create a single unified view of all of these events and move data through their environments. Currently Quantum has about 20 customers that are using OST to do everything from DR to tiered backup to tape consolidation.

Yet the most compelling story regarding the use of OST came from the end user on the call, Richard Nosal, the server administrator at High Point Regional Health System in Greensboro, NC. Nosal uses NetBackup 6.5.3 which is configured with one Master server and one Media Server.

Nosal had previously purchased a VTL solution but was still having a problem with tremendous storage growth - in a 6 month period he had to buy 21 TB of storage to keep up with his backup needs. To combat that, he looked toward a solution that provided both deduplication and replication across different storage units so he ended up selecting a Data Domain solution.

Since implementing OST on a Data Domain unit, he has seen a significant reduction in backup times for some of his applications. Before using OST, his user folders were taking over 10 hours to backup using NDMP.  Now using OST, they only take 5 hours to backup. But more important to note - Nosal was sending his backups to a VTL prior to using OST so he saw this marked improvement in performance when he made the switch from one disk-based data protection device to another.

What were equally interesting were his comments towards the end of the round table about his experiences when he was doing his initial testing using a Data Domain DDR660. Prior to deploying the Data Domain unit in production, he first tested the unit. At first, he did not use its OST feature but instead backed up to the DDR660's file system (NFS/CIFS) interface and saw backup speeds of 25 - 35 MB/sec over copper Ethernet. About 2 weeks later, Data Domain added the OST feature to the same DDR660 and Nosal saw his backup throughput jump up to 120 MB/sec - essentially a 4 to 5X improvement by just switching from NFS/CIFS to OST. Granted, this was just during testing but that is an amazing performance jump considering just one variable in the backup equation was changed.

So the real question is what does OST do behind the scenes to facilitate such a jump in backup performance? OST's benefits in relation to helping keep NetBackup's catalog up-to-date by tracking replicated and offsite copies of backup data are already pretty well documented and were discussed at length during the call. But this is the first documented story where I had heard that OST could replace a file system interface during backup and deliver such a jump in backup performance times. This left me puzzled as to why this showed such a jump in performance so I went back to Symantec to get a more technical explanation as to how OST accomplishes this. The results of that conversation I had with Symantec and some of its partners I will cover in more detail in an upcoming blog entry.