Storage Vendor News
Hu Yoshidas blog
2018: A Year in Review for Storage Systems.
Wed, 16 Jan 2019
2018 was a very busy year for Hitachi Vantara. September marked the one year anniversary of Hitachi Vantara which was formed by the integration of three Hitachi companies, Hitachi Data Systems, an IT infrastructure systems and services company; Hitachi Pentaho, a data integration and analytics company; and the Hitachi Insight Group, developer of Lumada, Hitachi’s commercially available IoT platform. This new company will unify the operations of these three companies into a single integrated business as Hitachi Vantara to capitalize on Hitachi’s social innovation capability in both operational technologies (OT) and information technologies (IT).
When the formation of Hitachi Vantara was announced, it was clear that combining Hitachi’s broad expertise in OT (operational technology) with its proven IT product innovations and solutions, would give customers a powerful, collaborative partner, unlike any other company, to address the burgeoning IoT market.
For lack of similar capabilities, some of our competitors began implying that we would no longer be focused on the innovative data infrastructure, storage and compute solutions that were the hallmark of Hitachi Data Systems. In fact, the closer collaboration between data engineers and data analysts from Pentaho, the data scientists from Insights with the proven software and hardware engineering skill of Hitachi Data Systems has given Hitachi Vantara even more talent and resources to drive innovation in storage systems.
During 2018 we proved our detractors wrong in many ways with the introduction of a new family of all flash and hybrid VSP storage systems which could scale from a low end 2U form factor to the largest enterprise storage frame with the same high end enterprise capabilities, including Global Active/Active availability, non-disruptive migration, multi-data center replication, and 100% data availability guarantee. A common Storage Virtualization Operating System SVOS RF, enables the “democratization” of storage services. A midrange user now has access to the same, super-powerful features as the biggest banks. Or, as a large company, you can deploy the same super-powerful features in remote, edge offices as you do in the core, using a lower cost form factor.
A Common Architecture from Midrange to Mainframes
Analysts like Gartner acknowledge that sharing a common architecture and management tools from the smallest VSP G200 to the flagship VSP G1500 provides users with an easy to follow upgrade path that leverages their investments in training policy and procedures. It also leverages Hitachi's investments in infrastructure monitoring and management tools, as well as ecosystem certifications and plug-ins. 2018 saw competitive storage vendors follow suit by announcing their intent to consolidate 3 to 5 disparate storage systems just to have a common storage system for the midrange. Since many of these storage systems were acquisitions, this consolidation effort will be a 2 to 5 year journey, with questions about migration between the systems. Hitachi Vantara is ahead of the pack delivering, a common storage platform from small midrange to large enterprise and mainframes system without any limitations in capabilities.
Open REST API
Another area of storage innovation was the introduction of AI and automation tools for the smart data center which was enabled by an open REST API. A REST API is built directly into our VSP storage controllers. We increased the memory and CPU in the controller specifically to support the REST API running natively in the controller. This gives us the opportunity to not only connect with other vendor’s management stacks, but also apply analytics and machine learning and automate deployment of resources through REST APIs.
Analyze Infrastructure Metrics from Servers to Storage
Hitachi Vantara has developed an analytics tool, Hitachi Infrastructure Analytics Advisor (HIAA) that can provide predictive analytics by mining telemetry data from servers, storage appliances, networking systems and virtual machines to optimize performance, troubleshoot issues and forecast when a business may need to buy new storage systems. Based on an analysis of metrics from host servers to storage resources the analytics tool can determine the right actions to take, then launch into an automation tool to invoke the appropriate services to execute that action.
Automate the Delivery and Management of IT Resources
The automation tool, Hitachi Automation Director (HAD), contains a catalog of templates that can automatically orchestrate the delivery and management of IT resources. The analytics tool communicates with the automation tool, through a REST API, to select a template, fill in the parameters and request deployment of resources, which is done automatically. The APIs are open, and Hitachi Vantara provides a design studio and developer community site for customers and third parties to design their own templates to fit their own environment, operation policy and workflow. Our customers love the ability to integrate Automation Director with their ServiceNow tickets for speedier resolution of the client’s requests.
Enhance Flash performance, Capacity and Efficiency
The common Storage Virtualization Operating System, SVOS, has been greatly enhanced from previous versions of the operating system, and has been renamed SVOS RF, where RF stands for Resilient Flash. SVOS-RF’s enhanced flash-aware I/O stack includes patented express I/O algorithms and new, direct command transfer (DCT) functionality to streamline I/O. Combined, these features lower latency up to 25% and increase IOPS per CPU core up to 71%, accelerating even the most demanding workloads. Quality of service (QoS) makes sure workloads have predictable performance for better user experiences. User selectable, adaptive data reduction with deduplication and compression, can reduce capacity requirements by 5:1 or more depending on the data set. A Total Efficiency Guarantee from Hitachi Vantara can help you deliver more storage from all-flash Hitachi Virtual Storage Platform F series (VSP F series) arrays and save up to 7:1 in data efficiency.
If there was any doubt that Hitachi Vantara would continue to be a storage systems leader, that should have been proven wrong in 2018. 2019 will provide even more proof points. Hitachi Vantara will continue to drive data center modernization with high-performance integrated cross-platform storage systems, AI-powered analytics, IT automation software, and the best in Flash efficiency. While these storage announcements were made in May of 2018, they did not make the cut off dates for Gartner's 2018 Magic Quadrant or Critical Capabilities for Solid State Arrays and General-Purpose Disk Arrays. So look for their evaluation in Gartner's 2019 reports. However, even without an evaluation of these new capabilities Hitachi Vantara did well in the Gartner 2018 reports and other Industry recognition reports.
- Gartner 2018 Magic Quadrant for Solid-State Arrays
- Gartner 2018 Critical Capabilities for Solid-State Arrays
- Gartner 2018 Magic Quadrant for General-Purpose Disk Arrays
- Gartner 2018 Magic Quadrant for Industrial IoT Platforms
- Gartner 2018 Critical Capabilities for Object Storage
- IDC MarketScape 2018 for Worldwide Object-Based Storage
- Forrester Wave Industrial IoT Software Platforms, Q3 2018
- 2018 Stratus Award for Cloud Computing: Private Cloud Provider
- 2018 Fortress Cyber Security Awards: Regulatory Compliance
- IoT Breakthrough's 2018 IoT Innovation Award: Enterprise Solution of the Year
- Frost & Sullivan 2017 Hall of Fame Award: Global Media and Entertainment Storage Customer Value Leadership
Hitachi Vantara’s API Strategy for The Smart Data Center
Thu, 10 Jan 2019
In this post I will take a deeper dive into one of the key enablers for Digital Transformation, the REST API. I will cover our strategy for utilizing it in our products and provide some example of how it is utilized to enable the Smart Data Center.
Application Program Interface, API, is software that allows applications to talk to each other. APIs have been an essential part of software development since the earliest days of programming. Today, modern, web based, open APIs are connecting more than code. APIs are a key driver for digital transformation where everything and everyone is connected. APIs support interoperability and design modularity and help improve the way systems and solutions exchange information, invoke business logic, and execute transactions.
Hitachi’s developers are reimagining core systems as microservices, building APIs using modern RESTful architectures, and taking advantage of robust, off-the-shelf API management platforms. REST stands for “representational state transfer.” APIs built according to REST architectural standards are stateless, which means that neither the client nor the server need to remember any previous state to satisfy it. Stateless components can be freely redeployed if something fails, and they can scale to accommodate load changes. REST enables plain-text exchanges of data assets. It also makes it possible to inherit security policies from an underlying transport mechanism. REST APIs provide a simplified approach to deliver better performance and faster paths to develop, deploy, and organize. Restful APIs are available in our Hitachi Content Platforms, Pentaho analytics, Hitachi Unified Compute Converged, Hyper Converged, and Rack platforms, REAN cloud, and LUMADA which is our IoT platform.
A REST API is built directly into our VSP storage controllers. We increased the memory and CPU in the controller specifically to support the REST API running natively in the controller. This gives us the opportunity to not only connect with other vendor’s management stacks, but also apply analytics and machine learning and automate deployment of resources through REST APIs. Here are some examples of how this API strategy brings operational benefits to the Smart Data Center.
Hitachi Vantara has developed an analytics tool, Hitachi Infrastructure Analytics Advisor (HIAA) that can provide predictive analytics by mining telemetry data from servers, storage appliances, networking systems and virtual machines to optimize performance, troubleshoot issues and forecast when a business may need to buy new storage systems. There are 77 performance metrics that we can provide via REST API over IP connections. Based on an analysis of these metrics the analytics tool can determine the right actions to take, then launch into an automation tool to invoke the appropriate services to execute that action.
The automation tool, Hitachi Automation Director (HAD), contains a catalog of templates that can automatically orchestrate the delivery and management of IT resources. The analytics tool communicates with the automation tool, through a REST API, to select a template, fill in the parameters and request deployment of resources, which is done automatically. During the execution, the automation tool may need to communicate with third party switches, virtual machines, containers or public cloud through their APIs. When one considers all the tedious steps required to request and deploy storage, networking, hypervisor, and application services for hundreds or even thousands of users, you can see how automation can reduce days of work downs to minutes.
Policy based Copy Management
Since most data is backed up and copied; a copy data management platform is available to simplify creating copies and managing policy-based workflows that support business functions with controlled copies of data. Hitachi Vantara provides a Hitachi Data Instance Director which the automation tool can invoke to deploy the copy workload and set up and enforce data protection SLA policies through REST APIs.
Hitachi Automation Director’s REST API is open and available for working with third party resources. Enhancements to the software include integration with IT service management (ITSM) tools, including the ServiceNow platform, for better resource tracking and improved REST API integration for working with third-party resources. Hitachi Automation Director creates workflows in Service Now, approval can be administrator driven or driven by the Automation Director. The Automation Director executes the changes and updates the ticket.
Third Party and Home Grown Services
Hitachi Automation Director encourages working with third party services by providing a design studio, and a developer community site. Service Builder is the design studio where users have the flexibility to create their own service template to fit their own environment, operation policy and workflow. They are provided the capability to leverage 3rdparty or home grown tools.
Hitachi Vantara has launched the Hitachi Automation Director (HAD) Developer Community site.
It is available to external users. Here Hitachi Vantara shares sample service templates(more than 30 content packs), prototypes, how to use Hitachi Automation Director, Q&A, etc. Here we will be collaborating with customers / partners to develop more content.
Call Home Monitoring
Other uses of REST APIs include our call-home monitoring system, Hi-Track. which has been re-coded to use our native REST APIs to collect information about the storage systems and report that back to our support teams. Hi-Track provides 24/7 monitoring for early alerting and insight to help your storage system run at peak efficiency. Only authorized Hitachi Vantara support specialists may establish a connection with your site, and only by using the Hitachi Vantara internal network. Secure access with encryption and authentication keeps error and configuration information tightly controlled, and your production data can never be accessed.
We have a Hitachi Storage Plug-in for Containers that integrates us with Docker and thereby with Kubernetes and Docker Swarm. This Plug-in is built on the REST API that is also available to customers to integrate with. This plug- in retains the state of the storage as containers are spun up and down. Without this, the storage for a container would disappear when the container goes away.
The VSP storage configuration tool, Hitachi Storage Advisor, can be accessed through software on an external virtual or physical server via the REST API.
The use of REST APIs is key to the integration of infrastructure, software, and analytics to create an intelligent data center. This is a summary of the primary benefits of our API strategy for an intelligent data center.
A Rest API built directly in our VSP controller provides connection with other vendor’s management stacks, and enables the application of analytics and machine learning for automated deployment of resources
An analytics tool, Hitachi Infrastructure Analytics Advisor, can provide predictive analytics by mining telemetry data from servers, storage appliances, networking systems and virtual machines to optimize performance, troubleshoot issues and forecast when a business may need to buy new storage systems.
An automation tool, Hitachi Automation Director, with a catalog of templates that can automatically orchestrate the delivery and management of IT resources.
A copy data management platform, Hitachi Data Instance Director, which can be invoked by the automation tool to simplify creating copies and managing policy-based workflows that support business functions with controlled copies of data.
Hitachi Automation Director’s REST API is open and available for working with third party resources like IT service management (ITSM) tools, including the ServiceNow platform, for better resource tracking and improved integration with third-party resources.
Hitachi Automation Director encourages working with third party services by providing a design studio, and a developer community site where users have the flexibility to create their own service template to fit their own environment, operation policy and workflow.
Other uses include call home monitoring, container plug-ins, and VSP configuration management from external systems. The list of plug-ins, utilities, and extensions will grow as the digital data center eco system grows.
Reduce workloads from days to minutes
Reduce errors resulting from tedious manual work
Reduce the need for skilled IT staff
Optimize use of IT resources
Increase speed of resolution to customer requests
Effective Outage Management with quicker return to service
Customize to fit their specific environment
Improve forecasting of future resource requirements
Nathan Moffit, Hitachi Vantara senior director of infrastructure, sums up our API strategy as Follows:
“Hitachi’s management strategy is based around the idea of a shared and open API architecture that allows us to simplify transmission of data across our suite of management tools & 3rdparty tools. Everything we have is API-based so that we can draw information in from other sources to create a more intelligent solution, but we can also pass information out if we’re not the master in the environment, so we can make other things smarter. In addition, our goal is to enrich our ‘library’ of 3rdparty device API information so that we can capture analytics from a broad range of devices & interact with them. We are taking a very vendor-neutral approach as we recognize that there is a much broader opportunity to deliver better solutions if we integrate with more vendors and partners.”