SCSI

Small Computer Systems Interface or SCSI was developed many years ago to create a standard set of commands and connections to address physical disks. The original technology used parallel ribbon cables but parallel SCSI has now been more or less replaced by Serial Attached SCSI, or SAS. The other main technology that is used to connect disks to motherboards is SATA, but SAS has one major advantage over SATA, it has a 'dual path' facility which means that if one path fails, the disk can still be accessed down the other path.
The big storage vendors all connect disks together under a storage controller and use different classes of storage so that data can be 'tiered' or placed on the correct storage type depending on its current value. All the big vendors now use SAS for the lowest disk tier, and solid state for the highest tier. Fiber channel disk was traditionally used as a middle tier, but SAS is now fast enough to compete with FC disks for performance, and are about one third of the price.
The combination of all these factors means that SAS is competing with SATA in PC and appliance applications, has replaced SATA in enterprise disks, and will almost certainly replace Fiber Channel too. While SAS itself will probably be replaced by the faster protocols used to support SSD devices, it will be around for years yet.

The Speeds of the various SAS release are

SAS-1: 3.0 Gbit/s, introduced in 2005
SAS-2: 6.0 Gbit/s, available since February 2009
SAS-3: 12.0 Gbit/s, available since March 2013
SAS-4: 24.0 Gbit/s, The specification was completed in August 2017, and should be commercially available by the end of 2019

    GFS Advert

Serial Attached SCSI (SAS)

There are three types of SAS devices: initiators, targets and expanders. Initiators attach to one or more targets and form a SAS domain and a SAS domain can consist of up to 128 devices. The initiators also need a service delivery system to be able to communicate with the targets.

Initiator devices can be host bus adaptors (HBAs), controllers or an on-board component on a motherboard. They have multiple ports for connecting to internal and external targets and each initiator port can be narrow ports with a single physical link, or wide ports with two, four, or eight physical links. If you want failover redundancy then you can connect SAS initiator ports to separate domains.

Target devices include SAS disks (HDDs) and solid state drives, SATA disks and SAS tape drives. SAS drives have two narrow ports and so can connect to two different domains for failover redundancy. They use the same electrical and physical connection interface as SATA drives. However, SATA drives have a single narrow port and so no failover. You can mix SAS and SATA devices in the same domain.

   

Accelerate DB2 Write with zHyperWrite and "EADM™ by Improving DB2 Logs Volumes Response Time:

Expanders are low-cost, high-speed switches that are used to increase the number of targets attached to an initiator and so create a larger SAS domain. Expanders come with 12, 24 or 36 port and can connect to initiators, targets and other expanders. They receive commands and data in one port and route them to another port based on the SAS address of the target.
Expanders use three routing methods: direct, table, and subtractive. If the target device is directly attached to the expander then the expander uses direct routing. Table routing is used to forward the command and data to another expander and subtractive routing is used when the expander does not recognise the address. In this case, the expander just forwards the command and data to another expander that recognizes the address.
All these methods use routing tables that are created in each expander during the discovery process known as self-configuration. The size of the expanders' routing tables determines how many initiators and targets you can have in a domain, but in theory you could develop a SAS network that could address as many as 16,256 physical devices.

It is tempting to say that the 'service delivery system' is just the cables that connect the SAS components together but of course it is a bit more complicated than that. SAS-2.1 and above supports active cables, which are thin cables with active circuitry including built-in drivers, repeaters, and an equalizing filter. These components are designed to reduce signal distortion and transmission loss, so thinner, longer cables can be used.
SAS also uses differential signaling (LVD), where the signaling is transmitted over two signal wires, with one wire transmitting the inverse of the signal on the other wire. The receiver reads the differential voltage between the two wires and so filters out noise, and ignores signal voltage drift.
SAS communication operates in full duplex mode, which means that it can send and receive information simultaneously over two wire pairs, and the SAS interface lets vendors scale up bandwidth by installing multiple physical links to create 2x, 3x, 4x, or 8x connections per port.
The actual SAS point-to-point connectors are small as they only need 4 wires, so they can be fitted to 2.5 in. hard disk drives.

SAS devices use port identifiers to ensure that communications are sent to the correct device. Each SAS port in a SAS domain has a SCSI port identifier that is assigned by the device manufacturer and identifies the port uniquely within the SAS domain (and usually world-wide unique too). This is usually enough to identify a device, but every SAS device also has a SCSI device name, which identifies the SAS device uniquely in the world.
Both the SCSI port identifiers and the SCSI device names consist of a 64 bit SAS address. For a SAS expander device, the SCSI port identifier and SCSI device name are the same SAS address. The SAS address is analogous to the WWN address in a Fibre Channel configuration.

It is possible to connect a SATA drive to a SAS interface, though you cannot connect a SAS drive to a SATA interface. This is because the SATA connector is keyed with a small notch between the data and power connections. SAS has no key or notch, so while a SATA cable cannot fit a SAS socket, a SAS cable will fit a SATA socket. It is also possible to mix and match SAS and SATA on the same SAS interface. To achieve this, the SAS designers made the SAS connector form-factor compatible with SATA, and the SATA connector signals are a subset of SAS signals that enable compatibility between SATA devices and SAS controllers. Upgrading from SATA to SAS is as simple as replacing the disk drives.
SAS uses three different protocols to communicate with the different device types, and automatically selects the correct one depending on the device type it is accessing

  • Serial SCSI Protocol (SSP) communicates with standard SAS devices using SCSI commands
  • SATA Tunneling Protocol (STP) wraps a SATA protocol into SAS packets for communication with SATA devices. Check out the SATA page for details.
  • Serial Management Protocol (SMP) communicates management information to expanders

SAS now comes with some of its logic embedded in chipsets. For example, a SAS RAID on chip controller (ROC) which takes data from a PCI or PCI-X host interface and RAIDs it to 4 or 8 SAS connections on a card. A SAS IO controller is similar to the above, but without the RAID facility. A SAS / SATA MUX dual ports a SAS drive to allow connection to two RAID controllers for high availability applications.

The picture below illustrates two SAS initiators connecting a mix of six SAS and SATA disks. The upper initiator is connected to both the fan-out expander and one of the edge expanders, so it has some failover resilience. The lower initiator is only connected to one edge expander so it has no resilience if that expander fails. However the lower initiator can see all the disks. The upper initiator can see all the disks except the one that in connected directly to the lower initiator.

Illustration of SAS initiators and expanders

iSCSI

A Fibre channel SAN requires a dedicated data network, separate from the LAN and this is costly to provision. It also requires that all the devices attached to the SAN be fitted with expensive Fibre Channel HBAs. An IP SAN removes this requirement as it works over the same network as the user traffic and used standard HBAs. This may be an issue as storage traffic will compete with user traffic, so it is best to implement iSCSI on a VLAN to isolate the traffic. Standard SCSI is limited to short distances, typically tens of meters. iSCSI encapsulates SCSI commands in TCP/IP packets and can send them over more or less any distance. This is useful for compliance driven long distance replication as this needs to transmit at block level. Of course, fibre does this but it is more expensive. iSCSI also drives IO at block level and is relatively cheap.
Most SAN switches are multi-protocol, so they will support and combine IP and Fibre channel in a SAN and can convert between SCSI and FC protocols.

SCSI Express

SCSI express is a combination of a SCSI disk interface and PCI express connectivity. It was designed to manage the high I/O requirements of solid state disks. The alternative architecture is NVMe (Non-Volatile Memory express), which will potentially provide a standard interface that will work from notebooks to enterprise servers. NVMe will also support SAS and SATA interfaces.

back to top