Virtual Tape Principles

As discussed in the previous page, there are two fundamental types of virtual tape; Virtual Tape Augmented (VTA), where data is written into a disk cache and then consolidated onto physical tape, and Virtual Tape Elimination (VTE), where data is written to disk and stored there until it is deleted.

The basic principles of VTA are illustrated in the gif below.

Virtual Tape Demonstration

Virtual tape vendors use different names for their virtual tape components, but the components are all basically the same, though they are implemented in a different way.

The diagram shows 16 virtual drives which are software emulated. The operating system sees these as real drives, mounts them and writes data out as virtual tapes. The data is not written directly to tape, but is staged, usually in compressed format, to a disk cache. When there is enough data in the cache, the vtape system copies the data from cache to a real tape drive, onto high capacity real tapes. It either frees up the space in the cache, or marks it as available for use when required, depending on the implementation and your settings. Its usually best practice to keep the disk cache full, and reuse space as required. That means that if you require to read a tape which was written previously, it may be already in the cache

The principle behind VTE follows the same initial process, but the data remains on disk and you need a lot more disk space of course..
While VTA uses deduplication, deduplication is essential for VTE to lower the disk space requirements. Deduplication works by checking large sequences of bytes and looking for duplicates. This byte sequence can be by entire files, by a fixed chunk blocksize or by a variable length sliding window. The last example is the most efficient at finding duplicates, but it also uses the most CPU. Deduplication can happen at the client, so that less data is transmitted to the backup server, or all the data could be transmitted and stored on the backup server, then it is deduplicated later to cut down on disk usage. These are called pre and post deduplication. Vendors claim that deduplication can reduce storage use by 90% or more.

The individual components, and some tips on how to use them, are described below.

Virtual tape drives

Virtual tape drives are defined to the operating system just like ordinary drives. From the viewpoint of the host operating system, they look just like real drives, and will accept all the commands, and return all the conditions that a normal drive would.

Virtual tapes

Virtual tapes need to be 'initialised' or given a logical label to identify them just like real tapes. The tape labels are held in a VTS catalog, so a virtual tape does not need to be recalled from physical tape to scratch it. All the data required for label checking exists in the Vtape catalog.

On an IBM mainframe, data is directed to an IBM VTS by DFSMS constructs and this can cause problems with foreign tapes. Suppose you define a virtual tape range from V00000 to V49999. Your favourite supplier then sends you a fix tape with a volser of V23456 and you submit a job to read it. DFSMS will intercept your request, divert the allocation to a virtual tape drive and mount its own virtual tape, not your real physical tape. This will happen even if you code in a specific UNIT=xxxx override to try to force the allocation to a specific non-virtual drive. However, nil desperandum, you can force your allocation, but you need to use a special DFSMS storage class, STORCLAS=DUPT@SMS

//   VOL=SER=V23456,UNIT=CART,

Disk cache

On a VTA system, the disk cache is a buffer which holds the virtual tapes, at least until they are written out to physical tapes. A virtual tape is usually held in the cache even after it has been written to a physical tape, then if it is required for read again, there is no need for a physical tape mount. On an IBM VTS, the disk cache is best kept full, with older, copied virtual tapes overwritten by new ones, as space is required. An Oracle/STK VSM will issue warning messages once it hits a capacity warning threshold. At this point, automated space reclamation should kick in, but don't ignore the warnings. NEVER fill up the cache on an STK VSM. If you do that, the VSM will grind to halt, and will require a service call to fix.

Physical tape drives

Physical tape drives are usually high capacity IBM TS1140 a SUN T10000C or D or an LTO-6 or 7. The Physical Tape drives are not defined to the operating system, they are only known to the Vtape system.

Physical tapes

Physical tapes cannot be ejected from the virtual tape library until all the data has been removed from them (IBM). If someone asks you to eject a virtual tape, send them an empty envelope (yes, it does happen).

Controlling software

The controlling software looks after the disk cache, and also cleans up deleted data from real tapes.

Reconciliation: checks for invalidated volumes, i.e. virtual volumes which have been re-written. Reconciliation is best run before Reclamation

Reclamation: applies to VTA systems and consolidates stacked volumes which contain expired virtual volumes. Reclamation requires scratch volumes, and requires free drives, so it must be run at a time when recalls are minimal. Keep the reclamation percentage low, between 10 and 30%, otherwise reclamation will run for too long, and not achieve much

Data restrictions

The consensus now is that you can put any kind of existing tape data onto virtual tape. In practice, some kinds of data are better than others. Also, virtual tape will provide a lot more drives, and that can benefit all applications. As always, the choice is up to you. Some things to consider are

  • DFDSSHSM data.
    DFHSM is already good at filling up large tapes, and has an internal mechanism, recycle, to free up deleted data. The HSM recycle function can conflict with the VTS reclaim function, so it is best to set the HSM percent valid parameter low, maybe 10%, and let the VTS do the work.
    HSM Backup and HSM Migration data have totally different expiration patterns, so keep them on separate logical tapes using SMS management classes. Ideally, backup data should be flushed straight through the VTS cache as it is unlikely to be needed again. If you keep migrated data resident in cache then you could reduce the size of your ML1 pool.
    If you want to recall a single file from an HSM tape, you first have to copy the entire virtual volume back into the VTS cache, which could mean that recalls from ML2 will take longer, depending on the size of the virtual tape. Also, HSM can run several recalls in parallel, and you may not have enough real drives to satisfy them all.
    On the plus side, it takes several hours to run an HSM audit against a 9840 or 3590 ML2 tape. Virtual tape usually emulates 3480 or 3490 tapes and so audits will not take so long.
    HSM duplexing is prone to error, especially when you have spanned datasets. The solution is to use VTS duplexing instead. This also removes the need to swap in alternate cartridges, as the VTS does that for you automatically.
    You really want to avoid having any spanned datasets. As the HSM tapes are now virtual and do not need to be filled up, set the tape percentage to 80% to avoid spanning.
    ABARS by definition is for disaster recovery, so you should not write ABARS data to a local VTS. However, you could write ABARS to a remote VTS. The IBM software team recommend that you run ABARS with the STACK option to consolidate data onto one tape, even with a VTS. The IBM VTS team recommend that you specify the NOSTACK option to avoid unnecessary multi-volume files, and let virtual tape do the stacking. The answer is that it depends on the size of the aggregates. If your aggregates are large, then it probably does not matter whether you use STACK or NOSTACK, as the VTS emulates small 3490 volumes so the aggregate will go to multi-volume files anyway. However, for small aggregates, NOSTACK means that DFHSM will write to at least two virtual tapes, both of which need to be retrieved back to cache before recovery can start. So on balance, it is probably best to specify ABARS SETSYS ABARSTAPES(STACK)
  • Database backups are often small files, and so JCL stacking is used to put several backups onto one tape. Virtual tape eliminates the need for JCL stacking
  • For Spectrum Protect / TSM backups, most people write their TSM backups to a disk storage pool initially, then copy the data off to tape, which is just the virtual tape principle. To then send this data to a VTS seems to be intuitively a waste of resource. TSM also has its own recycle mechanism to recover unused tape data, and again, recoveries can take longer, as the entire virtual tape has to be written back to cache before the restore can start. Thus it might seem that TSM is not a good virtual tape candidate, but in fact a lot of people use TSM with Virtual tapes, both VTA and VTE. The big attraction is the extra drives and the parallelism that brings. Because TSM works on an incremental forever philosophy, it does not deduplicate as well as other backup products.
  • Netbackup works on Weekly Full / Daily Incremental backups and so get a lot of benefit from deduplication. They also benefit from the large number of drives that Virtual tape provides.
  • DFDSS physical dumps, and FDRABR volume dumps move large amounts of data and could fill up the virtual tape DASD cache. They may not be good candidates if your cache is limited. DFDSS logical dumps, and FDR dataset dumps are good candidates.
  • Transient batch data, which is written out by one job, then read back again soon after, is a very good candidate, and should speed up a batch run considerably.

Performance Monitoring

Virtual tape requires a different type of performance monitoring, than real tape. There are three areas to watch

  1. The 'front end' channel usage. Because you will be typically sharing up to 256 virtual drives between 4 to 8 FICON channels, these channels can become a bottleneck
  2. The internal throughput, basically the speed at which the VTS or VSM can transfer data internally, and the residency time of virtual volumes in the disk cache.
  3. The 'back end' performance, which is, how busy the real drives are, and how often the system has to wait for a free drive to service a recall.

There are tools available to help with this. An example is Perfman for Tape Libraries, which will collect virtual tape performance data and report on it.

back to top