The DS8K architecture is discussed in the Enterprise Storage Selection page. Internally, the system consists of device adaptors that connect to the outside world, disk adaptors and enclosures for the data storage and a large memory cache to speed up access. The whole lot needs to be connected together and needs to processing power to run it.
Processing power is supplied by Dual 2, 4, 8 or 16 core POWER7-based controllers, which use 4-way Simultaneous Multi-Threading and so can handle up to 64 concurrent threads. All DS8000 machines have two processor complexes running as a redundant pair, so if one complex fails, the DS8000 will continue to run on the other complex.
Cache size depends on the CPU configuration, with a 16 way processor using a significantly larger cache than a 2 way. Cache is used to speed up IO operations, Sophisticated algorithms are used to detect patterns within read IOs, so required data can be predicted and loaded up into cache memory from disk before it is requested. If read IOs are sequential, then the DS8K will use an Adaptive Multi-stream Prefetching (AMP) caching algorithm, which basicallty streams the data into the cache.
Disks are a combination of (relatively) slow 'nearline' disks, 'day to day workload' SAS 2.0 disk drives and very fast solid state drives. All disks come in 16 drive install groups, except that the 600GB solid state drives can be installed in 8 drive half groups.
'Easy Tier' software is used to dynamically move data within the three storage classes, depending on how active it is at the present time. Data can also be placed manually.
A Storage Tier Advisor Tool exists to give guidance on how much solid state you need to optimise your performance.
All the components are connected together with PCI Express Generation 2 I/O enclosures which use point-to-point PCI Express cables. Inter-processor communication is isolated from I/O traffic, and used RIO-G ports. Storage Disks are contained in DDMs and are connected using switched FC-AL. Each disk drive is connected to two device adapters and each device adapter has four paths to the disk drives. One device interface from each device adapter is connected to a set of FC-AL devices so that either device adapter has access to any disk drive through two independent switched fabrics.
Data is organised within the DS8K as Extent Pools, How you organise your extent pools will depend on how many different types of physical disk you are using, and whether or not you are mixing CKD and FB data on a box. CKD and FB data must be in separate extent pools, and it pretty much makes sense to separate slow disks and fast disks into different pools.
Extent pools contain storage extents of either 1GB for FB or the size of a 3390-1 for CKD. To understand where the extents come from we need to switch to a physical view of the disks.
Physical disks are installed in groups of 16, called disk groups or enclosures. Each enclosure contains two fibre channel switches, and every disk has two fibre connections, one to each switch. These switches are then connected to two device adaptors over two independent RIO-G loops. Disk enclosures are installed into the front and rear of the DMX, and two enclosures are then paired up to form four Array Sites. An Array site contain 8 disks, 4 from a front enclosure and 4 from a rear enclosure, so an array site will also span two RIO-G loops. Array Sites are numbered S1, S2 etc. n.b. there is no S0! This is illustrated in the GIF below
When you format an Array site up as RAID, it becomes an Array. RAID options are either RAID5, 6 or 10. Typically, a CKD formatted RAID5 array based on 146GB disks will contain 826 GB of data, as one disk is reserved for a spare, and the equivalent of one disk is needed for RAID5 parity. For the benefit of you people reaching for your calculators, the missing space is the CKD formatting overhead. The FB formatting overhead is lower, so an FB RAID 5 array with a dynamic spare will contain 836 GB. The rules for calculating how many spares are needed are complicated, but in general the DS8K will require one spare for every Array, until it has four spares per Device Adaptor pair, but it may reserve more if different disk sizes or speeds are combined in an Array Site. The Arrays are numbered A0,A1, etc.
At present, a Rank consists of one RAID array. Each Rank is split into extents of 1GB each for FB format and 0.98GB (the size of a 3390-1) for CKD format. These are the extents that make up an Extent Pool. Ranks names are allocated by the system and are called R0, R1, R2 etc.
Ranks should ideally be allocated to Extent Pools so that the capacity is equally shared between Server0 and Server1. Ranks, Array Sites and Arrays are independent until the Rank is placed into an Extent Pool. It is then bound to the server associated with that pool.
As mentioned above, you need an absolute minimum of two Extent Pools per DS8K, and in theory you can have as many Extent Pools as you have Ranks. In practice, if you place more than one Rank into an Extent Pool, then you can stripe the data over the Ranks to improve performance. The IBM recommendation is to place between 4 and 8 Ranks in an Extent Pool. If you allocate all the space in an Extent Pool to volumes, then add an extra rank for more capacity, you will not get the benefit of striping as there is only one rank to strip across. Always try to add at least 4 ranks to a full pool, or to a new pool, so striping can be effective.
If you want to use space efficient volumes, then you must define a repository for them within an Extent Pool. Only one repository is allowed per Extent Pool and once defined it cannot be resized, except by deleting all the space efficient volumes, then deleting and reallocating the repository. Space Efficient volumes is a chargeable option.
Ok, so now we have a few Extent Pools all containing lots of 1GB extents. How do you present those extents to a server? A logical volume is made up of a set of extents from one Extent Pool. You can define a volume to be almost any size you want, but the space is added internally is fixed increments. This means you should make the total size of the logical volume to be a multiple of 1GB for FB or 3390-1 for CKD to prevent space wastage. Up to 64K logical volumes can be defined
In a mainframe environment, all the extents in a CKD volume must be contained in one Extent Pool, but can be striped over several Ranks. A CKD volume can be any size between a 3390-1 and a 3390-A (1113 to 262,668 cylinders) provided your z/OS release supports this.
The point behind this apparently complicated setup is that it makes the storage environment more flexible, especially for future maintenance. With as ESS, if you wanted to resize volumes, you had to empty out an entire array, delete it then redefine the whole array. The concept of breaking a CKD volume space into fixed size extents makes it possible to delete one volume from within a range, then add it back in again as a different size. You can also convert a 3390-9 to a 3390-27 in place by simply adding more extents to it and running and IDCAMS REFORMAT …. REFVTOC
In an Open Systems environment, a logical subsystem looks like a SCSI controller with up to 256 associated LUNs. Up to 256 Logical Subsystems can be defined with even addresses associated with Server0 and odds with Server1.
The DS8K supports several Open Operating Systems, including Windows, Linux, AIX, Sun Solaris, HP-UX, VMWARE and Open VMS. Each of these operating systems has its own quirks, so the best thing is to look up the latest data for your own OS.
back to top