Volume Shadow Copy Services

Volume Shadow Copy Service (VSS),often called Volume Snapshot Service, was introduced in Windows 2003. It is used in co-ordination with applications to provide point-in-time copies of single or multiple volumes. VSS is equivalent to hardware 'snapshot' functions like Flashcopy or Timefinder, but it is an operating system component that works at software level.
VSS works by making a block-level copy of any changes that have occurred to files since the last shadow copy. Only the changes are copied, not the entire file. VSS backups will not require the same size disk space as the original data, unless all the files are completely overwritten. However the amount of disk space used for changes can vary, depending on whether the application rewrites the entire file when a change is made, or just changes parts of the existing file

VSS can be used to improve backup and recovery processes.
Traditionally, we backed up data to tape in an overnight window, which could take several hours. This meant that the backup was not be consistent, as some files were backed up several hours apart, and if the backup fails, it was usually left for fixing the next day. Another problem is that backup software sometimes cannot cope with open files, so either all the applications must be stopped to get a good backup, or open files are skipped.

VSS fixes most of these problems by taking a snapshot of the data to disk at a consistent point. It can work with backup applications to get consistent backups of open files, so good backups can be scheduled to run several times per day, with the applications available 7*24. The backup process usually takes a few seconds, so if it fails, it can be fixed immediately, so ensuring a consistent backup every night.
This was expanded in Enterprise Vista and Windows 2008 with the 'previous versions' function that allows any user to looks at previous snapshot versions of a file, provided VSS is enabled for their file space.

The VSS freeze function can be used to take instant, and consistent backups, but for it to work effectively it needs to work with VSS aware applications and backup utilities. If an application is VSS aware, then the backup utilities will be able to process open files. A working VSS system has four basic components, some of which can be provided by third party vendors.

  1. The VSS Coordination Service provides control and communication services between the other three components and is part of the Windows operating system
  2. The VSS Requestor is typically a third party backup tool and it issues the VSS commands that create and manage the VSS snapshots. diskshadow.exe was introduced in Windows 2008 as an in-the=box VSS Requestor.
  3. The VSS Writer is the tool that manages the snapshot capability of VSS and it will make sure that all the files in a snapshot are consistent. The Windows system contains a VSS writer for basic filesystems, while third party database management systems will typically provide a specialised VSS writer for their databases.
  4. The VSS Provider maintains the shadow copies after they are initiated. The Windows VSS Provider is a copy-on-write software solution but it is possible to replace this with a SAN based hardware solution.

The process basically works like this

  • The Requestor, or backup utility asks VSS to tell the Writers involved in the backup to gather their writer metadata documents (XML files that contain instructions for the backup) and send them to the Requestor.
  • The VSS writer contacts the Provider responsible for managing the volumes involved in creating the shadow copy and freezes all IO to the snap disk at a consistent point, by interfacing with all applications and the Windows operating system. Any writes in memory are flushed to disk
  • The VSS writer then creates a snapcopy, using SAN hardware capability if that is available
  • Once the snapshot is complete, I/O is resumed. If the original data is updated, the VSS provider copies that data to the snap before any updates are applied to the source

This process works fine, providing all the applications are VSS aware. If some applications are not VSS aware, then their IO will not be frozen, and you will not be able to guarantee data integrity at the snapshot time, and open files may be skipped. You will need to take a view on how important this is. 'Thick' Client-Server applications will always be a problem, as the data is stored on the server, but processed on the client. Database applications that are not VSS aware will be a problem too, as you could backup corrupt data from partially completed transactions.

If the administrator enables Shadow Copies for Shared Folders (SCSF), then users can recover their own files of folder from a VSS backup, without any need to find and mount a tape.

In a NAS/SAN environment, once a shadow copy is taken of a disk on ServerA, that disk can be quickly made available to ServerB with appropriate LUN masking. This can all happen under VDS control.

Shadow Copies for Shared Folders

If the administrator enables Shadow Copies for Shared Folders (SCSF), then users can recover their own files of folder from a VSS backup, without any need to find and mount a tape.

There are three major reasons why data would need to be recovered -

  1. Loss of all or a major part of a building
  2. Loss of a hardware subsystem or disk - this could include a major virus attack
  3. Loss of a file or folder due to deletion, accidental overwrite or corruption.

Shadow Copy can be used for the third type of recovery, but not the other two, so it will not completely replace backup or remote mirroring solutions, nor is it intended to be used for long term archives. When storage area limits are reached, the oldest shadow copy will be deleted to make room for more shadow copies to be created. Also, there is a limit of 64 shadow copies per volume that can be stored. When this limit is reached, the oldest shadow copy will be deleted and once a shadow copy is deleted, it cannot be retrieved.

Setting up Shadow Copy at the Server

VSS works at volume level, not file or directory, and works on NTFS volumes, not FAT volumes. Microsoft recommend that its used for general purpose volumes that hold user files, documents, spreadsheets or databases. Home directories would be ideal.
Users must use shared folders to access shadow copies and administrators on the local server must also specify the \\servername\sharename path to access shadow copies.

A proportion of the disk is set aside to be used for VSS, and when that space fills up, VSS deletes older copies to make room. The default space is 10% of the size of the volume being backed up. If the rate of data change is high, then this proportion will be too low, and older backups will be deleted too quickly. If VSS cannot fit an amount of changed data into a backup space, even after all old backups have been cleared out, then it will not create any shadow copy. This means it is important to get the size of the backup volume correct.

You need to consider two factors -

  1. How many backups do your users expect to find? (You can only have a maximum of 64 shadow copies)
  2. How much data is changing between shadow copies?

You need to discuss and agree the first factor with your users, then publicise the agreement.

The second factor is a bit difficult, as it is not the number and size of files which are changing, but the number of changed blocks within the files. Microsoft provides tools to assist you with this. This number will also be affected by the timing between Shadow Copies. If you take a Shadow copy every hour, say, then you would expect to see a lot less changed data than if you took a shadow copy every day. However, remember the 64 copy limit. If you do take a copy every hour, you will only be able to recover data back less than three days. Shadow Copy is volume based, not policy based. You are not keeping 64 copies of a file, but 64 copies of changed data on a disk. If you are using Windows Server 2003, then by default, it creates shadow copies at 0700 and 1200, Monday through Friday. You may wish to change these settings.

To enable Shadow copies on Windows Server 2008 R2 or Windows Server 2012 work through the following tasks.

Shadow Copies is still an all or nothing function at a volume basis, that is you cannot select specific files or folders. By default, a schedule task will run at 7:00 A.M. and a default storage area of 10 percent of the available space will be created on the same volume.
To enable and configure Shadow Copies of Shared Folders

  1. Click Start , point to Administrative Tools , and then click Computer Management
  2. In the console tree, right-click Shared Folders , click All Tasks , and then click Configure Shadow Copies
  3. In Select a volume , click the volume that you want to enable Shadow Copies of Shared Folders for, and then click Enable
  4. You will see an alert that Windows will create a shadow copy now with the current settings and that the settings might not be appropriate for servers with high I/O loads. Click Yes if you want to continue or No if you want to select a different volume or settings
  5. To make changes to the default schedule and storage area, click Settings

Windows 2012 introduced the File Server Volume Copy Shadow Service (VSS) Agent Service. Applications can store files on local disks, and on remote SMB File Shares. Windows 7/ 2008 Server only supported VSS on local drives, so you could not get a consistent backup of all the data if some data was stored on SMB shares. Windows Server 2012 supports VSS for SMB File Shares by introducing a VSS Agent on the remote file server. The local server contains the VSS provider, and it talks to the agent using the new File Server Remote VSS protocol. This means that the local providor can manage VSS snapshots on the remote server and take shadow copies of the data.

Microsoft recommendations and best practices for Shadow Copies of Shared Folders.

Use a separate volume on another disk as the storage area for shadow copies.
Select a storage area on a disk that is not being shadow copied. Using a separate volume on another disk eliminates the possibility that high I/O load will cause shadow copies to be deleted and provides better performance. This is the recommended configuration for heavily used file servers. For failover clusters, this configuration also requires that the original volume and storage volume belong to the same cluster resource group.

If you are using failover clusters, two volumes on the same disk cannot be associated for diff area storage. This is because the cluster manages the disk for online and offline operations, but the Volume Shadow Copy Service (VSS) needs to have the diff area and original volumes brought offline or online in a specific order. Instead, the storage volume and the original volume need to be the same volume, or they need to be on separate physical disks.

Consider how your clients will use a shared resource before you enable Shadow Copies of Shared Folders and set scheduling options. Adjust the shadow copy schedule to fit the work patterns of your clients.

Do not enable shadow copies on volumes that use mount points. The mounted drive will not be included when shadow copies are created. Enable shadow copies only on volumes without mount points or when you do not want the shared resources on the mounted volume to be shadow copied. Alternatively, you can explicitly include the mounted volume in the schedule for shadow copy creation. (For previous versions of a file to be available, the volume must have a drive letter assigned.)

Perform regular backups of your file server. Shadow Copies of Shared Folders is not a replacement for performing regular backups. Use a backup utility, such as Windows Server Backup in Windows Server 2008 or Windows Server 2008 R2, in coordination with Shadow Copies of Shared Folders as your strategy for data protection.

Do not schedule copies to occur more often than once per hour. The default schedule for creating shadow copies is at 7:00 A.M., Monday through Friday. If you decide that you need copies to be created more often, verify that you have allotted enough storage space and that you do not create copies so often that server performance degrades. There is also an upper limit of 64 copies per volume that can be stored before the oldest copy is deleted. If shadow copies are created too often, this limit might be reached very quickly, and older copies could be lost at a rapid rate.

Before deleting a volume that is being shadow copied, delete the scheduled task for creating shadow copies. If the volume is deleted without deleting the shadow copy task, the scheduled task will fail and an Event ID: 7001 error will be written to the event log. Delete the task before deleting the volume to avoid filling the event log with these errors. To manually delete the scheduled task, click Start , point to Administrative Tools , and then click Task Scheduler . In Task Scheduler, click Task Scheduler Library , right-click the task to create shadow copies, and then click Delete .

Use an allocation unit size of 16 kilobytes (KB) or larger when formatting a source volume on which Shadow Copies of Shared Folders will be enabled. If you plan to defragment the source volume on which Shadow Copies of Shared Folders is enabled, we recommend that you set the cluster allocation unit size to be 16 KB or larger when you initially format the source volume. If you do not, the number of changes caused by defragmentation can cause previous versions of files to be deleted.

If you require NTFS file compression on the source volume, you cannot use an allocation unit size larger than 4 KB. In this case, when you defragment a volume that is very fragmented, you may lose older shadow copies faster than expected.

After backing up a volume that contains shadow copies, do not restore the volume to a different volume on the same computer After backing up a volume that contains shadow copies, do not restore the volume to a different volume on the same computer as this will leave multiple snapshots with the same Snapshot ID on the system and will cause unpredictable results when performing a shadow copy revert.
You can restore the volume to the original volume on the same computer; you can restore the volume to a different volume on a different computer, or you can restore file and folders to any location on the same computer.

Shadow Copy at the Clients

Shadow Copies for Shared Folders comes ready installed on Windows 2003 clients, and can be installed on Windows 2000, XP Professional, and Windows 98. Once installed, if you right click on any file or folder and select Properties, you will see an extra tab called Previous Versions. If you click on the Previous Versions tab, you will see a list of shadow copies of that file, with the date and time the copy happened. If you click on one of those versions, you get three options, View, Copy, or Restore.

Recovery of Files or Folders

The recovery process is slightly different, depending on whether you want to recover a deleted file, and existing file or a folder. The three scenarios are described below.

Recovering a Deleted File
  • Go to the folder where the deleted file used to be, with Windows Explorer.
  • Right click your mouse on a blank space in the folder (not over a file).
  • Select Properties from the menu then select the Previous Versions tab.
  • Select the version of the folder that contains the file before it was deleted, and then click View.
  • View the folder and select the file that will be recovered.
  • Drag and drop, or cut and paste, the shadow copy to the desktop or folder on the end user's local machine.
Recovering an Overwritten or Corrupted File
  • Right-click the overwritten or corrupted file and click Properties.
  • Select Previous Versions.
  • If you want to view the old version, click View.
  • To copy the old version to another location, click Copy
  • To replace the current version with the older version, click Restore.
Recovering a Folder
  • Position the cursor so that it is over a blank space in the folder that will be recovered. If the cursor hovers over a file, that file will be selected.
  • Right-click the mouse, select Properties from the bottom of the menu, and then, click the Previous Versions tab.
  • Choose either Copy or Restore.
  • Choosing Restore enables the user to recover everything in that folder as well as all sub folders. Selecting Restore will not delete any files.