Windows Removable Storage Services

Microsoft removed support for RSS/RSM with Windows Server 2008 R2 and Windows 7. The information below will be retained as it could be of interest to anyone running older operating systems.
One issue is "How do I restore a file backed up using RSM on an older operating system, to a new system when I cannot start RSM?". The answer seems to be that you can't do this so you need to keep an older system around for these restores.

Removable Storage Management (RSM)

Introduction

One of the issues with Open Systems storage is the lack of an operating system based method to manage tapes. This means that applications that use tapes have to develop their own tape management systems, which can be a considerable overhead, and makes it difficult to share tape libraries between applications.

Microsoft attempted to plug that gap with Removable Storage Management (RSM). RSM made it easy to manage near-line libraries, and the media contained in them.

Support and Compatibility

RSM requires NTFS version 5, which is supported from Windows 2000 up to, but not including, Windows 2008.

If RSM is installed, it can affect existing applications, as it takes exclusive control of all media changing robots. Any non-RSM aware applications that use these robots will stop working, and must be changed to use RSM, so you need to discuss RSM support with your third party vendors.

Media Pools

RSM uses 'Media Pools' to classify the storage. Every piece of removable storage must belong to a media pool. Each media pool holds a single type of media that has common management properties. Media pools can be

• Shared between applications
• Span multiple libraries
• Contain other media pools (but the cannot contain both media pools and media)

This means that you can define a very flexible hierarchy of pools to describe and manage your removable media.

There are two types of media pools, System and Application. The System pools are for data that does not belong to Applications. Every different media type has three system pools

free media pool

basically scratch pools to be used when an application needs an empty tape

unrecognized media pool

media which is not initialised, or which the system is unable to classify. You need to manually move this stuff to a free media pool

import media pools

media which has come from another RSM system. You need to either move this media the relevant free pool or application pool

Applications pools contain data currently required by applications. Each application will create pools for its own exclusive use. However, applications can share media pools, as application pools can be subsets of a media pool.

Managing RSM

You can manage Removable Storage with the management GUI, or from a command line. The Removable Media service must be active for the commands to work. The line commands are all prefixed with rsm, and include

• rsm allocate to allocate media from media pools. You can specify that media will be automatically use media from the free media pool with a /d switch, and automatically return empty media with a /r switch. The default is not to do automatically use scratch media, or return empty media. This could cause you to run out of RSS tapes. If your media is multi sided, you can also allocate each side to a separate pool
• rsm deallocate will deallocate media from media pools
• rsm mount will mount media onto a drive. You can mount the media in either read or write mode.
• rsm dismount will dismount media from a drive
• rsm createpool to create a media pool
• rsm deletepool will delete media pools
• rsm eject will eject media from a library
• rsm ejectatapi will eject media from an ATAPI changer
• rsm refresh will refresh the RSM database so it contains the current status of media, drives or libraries
• rsm view will list out libraries, drives and media

RSM Hints and Tips

RSM problems

RSM has been known to stop responding to media requests, or your backup utility may be issuing error messages stating that it cannot connect to the Removable Storage service. If this happens, check out the obvious first, make sure that the RSM service is not disabled or stopped.
If you see the message 'the removable storage database failed to load. Check the event log' then that could mean the database is corrupt. Database recovery procedures are described below.

Before you start to recover your database, check out the DelayStart value in the registry. By default, the RSM service is delayed by 45 seconds to help boot performance. If you want to RSM to start up faster, you change that registry key.
Standard caution applies, you update the registry at your own risk, and if things go wrong, then worst case is that you need to reload the whole Windows operating system.
So if you are experienced at registry changes, change the DelayStart value in registry key HKEY-LOCAL-MACHINE\System\CurrentControlSet\Control\NTMS. The value is in seconds.

RSM does not recognise device

If your application program does not recognise the removable device, then this is usually a physical problem with the device or maybe the cable that connects it. However if the device is recognised correctly by the Windows Device Manager then it is worth checking it out in the RSM database as this keeps a record of the device and media status. To check the RSM status,

• Right-click My Computer, and then select Manage.
• Navigate to Computer Management, Storage, then expand the Removable Storage section. This will list out all removable media. Any drive types that contain removable media are listed under Physical Locations. If a drive has red X on it, it was detected at one point but is now offline.
• Delete the drive from RSM by right-clicking the drive, and then clicking Delete.
• Remove the device from Device Manager
• Reboot the server
• When the device is detected, it is automatically entered into RSM and placed available for use.

Recovering the RSM database

In extremis it may be necessary to restore or rebuild the RSM database. Your backup programs are probably using RSM to manage the backup media, so this makes restoring RSM a bit tricky. The RSM database is located in the %SystemRoot%\System32\NtmsData folder. If you use Windows 2000 Backup to back up your %SystemRoot% folder then Backup automatically exports the RSM database to both the backup media itself and the %SystemRoot%\System32\NtmsData\Export folder. You can restore the RSM database from the export folder, but it is a manual process. If you backup the RSM database with a third party product then you need to create an empty database before you can start. For example, the TSM command to backup the RSM database is backup RSM. This exports the database to \windows\system32\ntmsdata\export then copies if off to the TSM server. If you don't have a backup, then you can rebuild RSM from scratch. This means that you have three recovery scenarios

For all three scenarios,

• Scroll to the 'Removable Storage' service in the 'Services' window.
• Change the startup type for Removable Storage service to 'Disabled' the 'Stop' it.
• Browse to \systemroot\system32\ntmsdata\, and then rename all files in that folder to *.OLD. You could just delete the RSM database files, but my preference is to keep the old ones 'just in case'.

Windows backup -

• Use Registry Editor to view the following registry key: (edit the registry with care and be aware that you can corrupt it by changing it and that could require a complete re-install of the Windows operating system) HKEY-LOCAL-MACHINE\System\CurrentControlSet\Control\NTMS
• If the ImportDatabase REG-DWORD exists, change the value of the ImportDatabase REG-DWORD from 0 to 0x1.
  If it does not exist, add it with a value of 0x1: value name: ImportDatabase data type: REG-DWORD String: 1
• Change the Removable Storage service startup type to Automatic then start it. This should import the database from the export folder, build a new index file, and be consistent from when the last %Systemroot% backup was performed.

Third party backup -

• Change the Removable Storage service startup type to Automatic then start it. This will create a new database.
• Restore the RSM database with your third party product

No backup -

• Change the Removable Storage service startup type to Automatic then start it. This will create a new database
• Rebuild the database by attaching the remote devices and inserting all the media.
• Remember to take a backup next time

Removable Storage System (RSS)

RSS is an HSM (hierarchical storage management) application that uses RMS for external media management. It migrates data from expensive disk to cheaper tape if it has not been looked at for a while then transparently recalls it back again if it is needed. Microsoft have also enhanced the user interface side of the Windows 2000 operating system. RSS provides the following features

• The migration process leaves a stub file behind, which contains a pointer to the migrated data. This means that the user sees a logical view of the location of the files; they appear to be in the same folder as non-migrated files so migration is transparent to the application.
• The Windows 2000 GUI places a clock icon in migrated files, to indicate that retrieval will take a bit longer.
• The user is notified when the file is being retrieved from Remote Storage.
• Network timeouts are automatically extended for files that are stored on tape.
• Windows Backup will not recall migrated files to back them up.
• The NTFS Change Journal will indicate that a file has been migrated or recalled.
• The Indexing Service is HSM aware and will not try to index a migrated file.
• If an application tries to recall all migrated files, RSS can terminate the application once a configured runaway recall limit is encountered.

Restoring the RSS database

Try the following actions to recover the RSS database. This process assumes that you have a good backup, and that RSM is working correctly

• Scroll to the 'Remote Storage' services in the 'Services' window.
• Stop the 'Remote Storage Engine', the 'Remote Storage File' and the 'Remote Storage Media' services.
• Find the most recent backup of the database. This will be on the last tape used by RSS to migrate files. Go into Removable Storage Manager, Media Pools, Remote Storage Pool. The last tape will be the one with the highest suffix number.
• Move this tape to the NTbackup media pool. You may need to create that pool by running NTbackup.exe
• Catalog the tape using Windows 2000 Backup, If the RSS media spans more than one tape, catalog all the tapes
• Within the tape, navigate to the %windir%\system32\RemoteStorage folder, and find the latest backup.
• Using Restore, select the most recent RemoteStorage folder and all files within this folder.
• By default the databases will be restored to the RSS Managed Volume. You want to restore them to the operating system partition so in the 'Restore files to' box, click alternate location, and then click the drive that contains the Windows folder, then run the restore.
• Make sure that the %systemroot%\system32\RemoteStorage\engdb is empty by moving any existing files to another folder.
• From the engdb folder above, execute the command:

  Restore %systemroot%\system32\RemoteStorage\engdb.bak

• Put the tapes (or other media) that you used for the restore back into the RemoteStorage media pool.
• Restart the 'Remote Storage Engine', 'Remote Storage File' and 'Remote Storage Media' services and all should be well. Try recalling a few migrated files to check.

back to top

---