VMworld 2018, Tech-Previewing the .Next VSI Plugin

VMworld 2018 is taking place this week and amongst the many announcement, I want to also talk a little bit about something that will come out soon, our vCenter (Virtual Storage Integrator or ‘VSI’ in short) has been our premier plugin to manage your

Dell EMC storage arrays, Virtual Storage Integrator (VSI) is a plug-in for VMware vCenter. It enables administrators to view, manage, and optimize storage for VMware ESX/ESXi servers and hosts, and then map that storage to the hosts.

VSI 8.0 consists of:

  • A graphical user interface (VSI-Plugin)
  • The Dell EMC Integration API which provides communication and access to the storage systems.
  • It’s all new!
  • VSI 8.0 supports the vSphere Client (HTML5) and is a total refresh of VSI
  • SIS (Solutions Integration Service) has been replaced by IAPI (Integrations API)
    • Runs on Docker Container/Photon OS
    • Uses Redis/Cache
  • VSI 8.0 Leverages vCenter Users/Groups
  • Improved user experience

    VSI Administration

    • VSI Home Page (Monitoring the status of the VSI plugin storage including capacity)
    • Registering/Unregistering Dell EMC PowerMax/VMAX AFA, XtremIO, and Unity storage systems
    • Registering/Unregistering VMAX AFA/PowerMAX Storage Groups (including setting max usage), Unity Storage Pools, and Unity NAS Servers
    • Granting/Revoking Storage Access to VMAX AFA/PowerMax Storage Groups, Unity Storage Pools, Unity NAS Servers, and XtremIO storage systems
  • Storage Management
    • Managing VMFS 6 datastores on PowerMax/VMAX, XtremIO, and Unity
    • Managing NFS 3 and 4.1 datastores on Unity
  • Host Management

Managing Host Best Practices for XtremIO

Deploying VSI

One of the biggest changes we are making to VSI 8.X is ease of deployment which means that during it’s OVA deployment, you just need to give it some IP settings and the vCenter credentials and off you go, no need to login to the appliance via a special port and do it from there as well…yep, that’s a big request that came from you, our customers and partners so now, it’s a completely different ballgame where it comes to deployment.

Once the OVA has been deployed, you can check that the plugin is registered in vCenter, under ‘Menu’-> Dell EMC VSI

You can also verify the plugin status and version via the ‘Administration’ Tab -> ‘Client Plug-Ins’

001

  • The Home page/Dashboard is where you view the health of your plugin.
  • In this example you can see the status of the Integration services (IAPI), the number of registered vCenter Servers and Storage Systems, and the capacity of storage

  • Click on vCenter Servers to display the vCenter servers registered to the VSI plugin. You can register and unregister vCenter servers in a linked mode group, as well as editing the IP address being used by the IAPI to connect to the vCenter server.

  • The Storage Systems page displays the storage systems registered to VSI.
  • You can register, unregister, and edit storage system registration from this page.
  • Expand the storage system to view details and any storage pools, NAS servers, and storage groups

  • The User/Groups page is where you manage a vCenter user’s or group’s access to Dell EMC Storage
  • You can view, add and edit access to storage pools, NAS servers, storage groups, and XtremIO clusters.

  • To register a storage system, click +. This opens the Register Storage System wizard.
  • Select the storage system Type XtremIO and enter the required information.

Note that you must register an array using a user with admin privileges

  • The XtremIO storage systems being managed by the XMS are discovered and available for registration.
  • Select the XtremIO storage systems you want to register

Click Next

  • To allow other vCenter users or groups to use the storage system for provisioning, you must grant them access.
  • Select the users or groups to be granted access to the storage system.
  • Click Next

  • The storage systems you registered are now displayed in Storage Systems page.
  • You can expand the Storage System to view storage system details and users granted access.

That’s it for now, as we get to the release date, I will publish another post showing all the features and how to work with them

P.S

I made a reference to the fact the VSI SIS has been replaced by IAPI (Integrations API), more on that later on..

Why VMware have chosen XtremIO for their SAP HANA Workloads

We have just published a short white paper and some videos explaining why VMware have chosen XtremIO to run their SAP HANA workloads, the white paper can be downloaded by clicking the screenshot below

You can also watch Kandy O’Mara (A Solutions Architect from VMware) explaining the SAP HANA use case here

 

The CUBE also had a longer session with Kandy which you can watch here

AppSync 3.8 is GA, Here’s What’s New

We have just released the long awaited 3.8 version of AppSync

AppSync is a software that enables Integrated Copy Data Management (iCDM) with Dell EMC’s primary storage systems.

AppSync simplifies and automates the process of generating and consuming copies of production data. By abstracting the underlying storage and replication technologies,

and through deep application integration, AppSync empowers application owners to satisfy copy demand for operational recovery and data repurposing on their own. In

turn, storage administrators need only be concerned with initial setup and policy management, resulting in an agile, frictionless environment.

AppSync automatically discovers application databases, learns the database structure,and maps it through the virtualization layer to the underlying storage LUN. It then orchestrates all the activities required from copy creation and validation through

mounting at the target host and launching or recovering the application. Supported workflows also include refresh, expire, and restore production.

AppSync supports the following applications and storage arrays:

l Applications — Oracle, Microsoft SQL Server, Microsoft Exchange, VMware VMFS datastores, VMware NFS datastores, and Windows, Unix, and NFS File systems.

l Storage — VMAX, VMAX 3, VMAX All Flash, PowerMax, VNX (Block and File), VNXe (Block and File), XtremIO, ViPR Controller, VPLEX, Dell SC (Block only), and Unity (Block and File).

l Replication Technologies—VNX Advanced Snapshots, VNXe Unified Snapshot, TimeFinder Clone, TimeFinder VP Snap, SRDF, SnapVX, RecoverPoint Bookmarks, XtremIO Virtual Copies, Unity Unified Snapshot, Unity Thin Clone, Dell SC Series Snapshots, and ViPR Snapshot.

The AppSync Support Matrix is the authoritative source of information on supported software and platforms.

What’s New

AppSync 3.8 includes the following new features and enhancements:

Unity enhancements:

Create and manage remote copies of applications that reside on Unity Block storage using Unity replication. Supported only on Unity 4.2 and later.

Ability to run Silver and Gold service plans, allowing copies of the application to be created on the remote array (Silver service plan), or simultaneously on the local and remote arrays (Gold service plan).

Supports both asynchronous and synchronous replication modes.

Ability to repurpose a remote copy.

XtremIO enhancements:

Create and manage remote copies of applications that reside on XtremIO block storage, using XtremIO native replication. Supported only on XtremIO 6.1 and later.

Ability to run Silver service plans, allowing copies of application to be created on a remote array.

Ability to repurpose a remote copy, for making it a useable copy on the remote array.

Service plan copies are now created as immutable read-only snapshots for better protection. Mounting a copy of these, will result in a new read-write copy taken of the read-only copy. Supported only on XtremIO 6.1 and later.

Provides an option to use the modified read-write copy or the original read-only copy for copy restore.

XtremIO X2 Native Replication

•Ability to create local or remote XtremIO X2 copies with an existing Native Replication configuration

–The remote replication session must be in an active state

Lets dive deeper into this use case and why it’s so important for both physical and virtual (vSphere based) environments, lets assume you are running XIOS 6.1 with XtremIO Native Replication (NR) and you are replicating a consistency group (CG) which contains two volumes that were formatted as datastores, say with a 5 minutes RPO, XtremIO NR will copy the data in a crash-consistency manner, ok, that’s great but now lets assume that some of your VMs are running databases or Active-Directory domain controllers, all of these can benefit from using VMware tools to achieve application-consistency snapshots and this is where AppSync comes in


AppSync can be used in conjunction to XtremIO NR to generate application level consistency using either the VMware tools or it’s own agent (for MS-SQL and Oracle DBs), so using my example, you can use NR to take a crash level consistent snapshot every 5 minutes and use AppSync (on the same replication session) to take an application consistent snapshot once a day


You can identify the crash from the application consistent snapshots using the XMS UI


Crash consistent snapshots will look like [Snapshot…] and the application consistent snapshots will have the vCenter tag (identified in purple in the screenshot above), they will also have the session name suffix in them, you can of course use the AppSync UI to see that as well (see the below screenshot)


–AppSync creates a local bookmark and the XtremIO policy ships it across, deleting the local bookmark once complete

Bronze(local) or silver(remote) service plans and local or remote repurposing copies

•The ability to restore from the local side if the application is on an XtremIO CG with a remote protection session configured

–Restore from bronze copies will fail if the source volume has an “active” session at the remote site

–Manual steps required to restore in such a scenario:

Terminate the remote protection session

Perform restore operation from AppSync

Start the remote protection session

The second use case is an highly requested one, use the DR site data as a development environment, so using the logic below, Appsync will use the XtremIO NR snapshots, these snapshots will be at a read-only state at the DR site (unless you of course failover or test failover them), take a snapshot from there which will be a writable one and use these snapshots for your development environment that just happen to act as your DR site for these dark days..this way, your ROI on the DR site is much higher!

Limitations and Restrictions

•At the time of GA release, one XMS managing both local and remote arrays has not been formally qualified –each need their own XMS

•This release does not support “Gold” service plans -simultaneous local and remote copies

•This release does not support restoring the remote copy

•Replication sessions are managed at the CG level, and a CG can have only one replication session

•Only one remote(silver) copy can be mounted per remote protection session, because only one bookmark can be enabled at any one point in time on an XtremIO CG with remote protection configured.

–This does not apply to either generation repurposed copies -they utilize a snapshot set created from a bookmark, not the bookmark itself

•A LUN can be part of only one replication session -If a single LUN is part of two CGs:

Only one of those CGs can have a replication session and that CG with the replication session is selected by AppSync for all remote operations

–AppSync will use volume based API’s for all local operations, which might have a performance drawback with repurpose use cases. Hence it is suggested to have all the LUN’s of an application in a single CG.

•Replication sessions must be configured in a 1:1 fashion –one session cannot have multiple targets

XtremIO XIOS 6.1 Behavior Changes

•AppSync’s Protection/Service Plan Workflows:

–RO snaps are now created when using AppSync’s service plan/protection workflows

›When mounting: AppSync internally creates RW snaps for the corresponding RO snaps, and uses them to mount and recover the application. The RW snap is maintained on array even if the copy is unmounted, and reutilized if the same copy is again mounted. This way, the changes on the mounted copy are retained.

On Expire: Both the RO & RW snaps are deleted on the array.

When restoring: AppSync provides an option to allow for either the modified RW copy or the original RO copy to be used

•AppSync’s Repurposing Workflows:

–RW snaps (using CG APIs from XtremIO) are created when source LUNs are in the same CG

This is similar to how they are created with pre-AppSync 3.8 and pre-XIOS 6.1

›Volume APIs are used for 1stgeneration copies, if not all LUNs are part of the same CG

•Please understand that there is a limitation on the number of CGs per XtremIO cluster supported, which is 512. AppSync creates linked CGs (copy CGs) for the below use cases:

–During Repurposing workflows only when the source is in a CG

–If all the volumes with the application being repurposed fall into a single common CG

–Each remote repurposing copy creates a new linked CG

VMAX3/PowerMax enhancements:

Enhanced clone target rotation using device relinking. This will necessitate one more set of target devices than the set rotation count.

AppSync can auto provision target devices from storage resource pools when appropriate devices are not available in the configured storage group during protection.

Better handling of unsupported multiple storage resource pool configuration.

Provides an option for copy restore from either the modified target device or the original SnapVX snapshots.

Enhanced copy cleanup of stale snapshots on the array.

TCE enhancements:



Enhanced the Automated Log Collection wizard to collect logs from the AppSync server, AppSync agent hosts, vCenter servers, and SMI-S providers.

Provides an option to forcefully delete a host from AppSync.

Provides an option to consolidate UNIX plug-in host logs into a single file.

Provides an option to enable email alerts for on demand jobs.

Enhanced the repurpose email alerts to include the label in the subject of the email.

Email Alert Enhancements


•Email alerts are now able to be sent for on-demand jobs

–Found by navigating to Settings > AppSync Server Settings

–This notification setting is notset by default


–Applies to on-demand Service Plans and Repurposing workflows

–Expiring copies do not fall into this category –no alerts are sent

–Notifications are sent both for success and failure

•Repurposing workflow email alerts will now show the custom ” label, rather than “Service plan:REPURPOSE”

Mounting as Independent Disks


•Ability to mount copies as Independent Persistent or Independent Non-Persistent Disks

Please note:AppSync does not support the protection of applications on independent non-persistent disks. Users will see an error event like EVENT [SPP_000083]: Application F:\resides on independent non-persistent disk.

•Found by navigating to Mount copy settings of a service plan or repurposing workflow

•Applies to VMware virtual disks on ESXi hosts only

You can now schedule a service plan to run on the last day of every month and set the time with the granularity of 1 minute after the hour for At the frequency of and Every day at scheduling options for both service plans and repurposing workflows.

Prompts the user to configure ESRS gateway for collecting configuration information.

Other enhancements:

Provides an option to mount application copies on a virtual disk as independent disks (both persistent and non-persistent).

Supports the protection of applications on partitions created utilizing the GUID Partition Table (GPT) on Linux agent hosts.

AppSync now sorts the default install-plug-in versions from the latest version to the earliest.

You must now explicitly select a mount host in a service plan.

You can now run CLI commands from a path that does not have the CLI software on it.

Ability to run CLI commands from any path, even one that does not have the CLI-software on it

–Alleviates the need to navigate to the specific CLI path to execute commands

•Resolved an issue where Appsync-SIAMLogs.txt was unable to be created in a directory other than where the CLI was installed

•Appsync-SIAMLogs.txt now gets created either under the system temp location, or in the path specified with the APPSYNC_CLI_LOG_DIR environment variable

–APPSYNC_CLI_LOG_DIR is an environment variable created by the user on that particular host system hosting the CLI software, in order to save the Appsync-SIAMLogsin a defined path

–For Windows, the logs get created under %temp% or in the path as specified in the APPSYNC_CLI_LOG_DIR environment variable

–For Unix, the logs get created under /tmp or in the path as specified in the APPSYNC_CLI_LOG_DIR environment variable

Simplified Workflow(s):

No Default Mount Host / User Must Select

•Requires the user to select a mount host by leaving the selection blank by default

•Previously listed the original/source host selected, which can cause mounting failures/conflicts

•This applies to new Service Plans only –existing service plans are not automatically modified

Default Install Plug-In Version

•The default plug-in version to push/install displays the latest (most up-to-date) version

•Previous versions displayed the oldest version from the drop-down, making it easy to accidently downgrade the plug-in, rather than upgrade the plug-in

Ability to Forcefully Delete a Host


•Ability to forcefully remove a host from the AppSync UI, even if the AppSync database maintains records

Flexible Scheduling Options

•Ability to set the At the frequency of and Every day at with a granularity of one minute after the hour

–Applies to both service plans and repurposing workflows


•Ability to set the Specified days of the month setting to a new value called last day of the month

–Alleviates the need to create a new plan and change the schedule based on the month


•AppSync will not check for any copy dependency on the server and deletes all internal meta data related to this server. User needs perform all cleanup tasks, such as unmounting copies from the server, cleaning up copies on the array, and uninstalling/removing the plug-in

This release also contains the following fixes:

You can download AppSync 3.8 by clicking the screenshot below

ESG Lab Review: Dell EMC XtremIO X2 – Delivering High Performance and Advanced Functionality for Virtualized Environments

ESG have just released a new validation paper about how well the XtremIO X2 platform behaves in a virtualized environment.

Abstract

This ESG Lab Review documents testing of the Dell EMC XtremIO X2 array with a focus on high performance and advanced features for today’s modern, virtualized environments.

The Challenges

There are more technology options than ever before, promising levels of functionality that will truly transform business. But while the vision is exciting, actually delivering is not so easy. The consumerization of IT has led to newly empowered users (as well as senior management) driving functionality requests, believing that anything is possible. This often leaves IT struggling to deliver in reality what the vision promises. It is not surprising, then, that in ESG research, more than two-thirds (68%) of senior IT decision makers believe their organizations’ IT environments are more complex today than two years ago.1 For today’s organizations that focus on virtual servers and desktops, private cloud, and hybrid clouds, this complexity can confound even the best intentions for high performance, simple management, and high availability.

The paper can be viewed by clicking the screenshot below

You can also watch Analyst Kerry Dolan discusses how XtremIO X2’s differentiated architecture provide a foundation for IT transformation noting that “XtremIO X2 is chock full of features and integration that deliver simplicity, efficiency, automation, data protection, and user productivity.”

 

Updated White Paper, Best Practices for running Oracle On Dell EMC XtremIO X2

Dell EMC XtremIO is a market-leading, purpose-built, all-flash array that offers consistently high performance with low latency, unmatched storage efficiency with inline, all-the-time data services, rich application, integrated copy services and
unprecedented management simplicity. It is designed from the ground up to unlock flash technology’s instant performance potential by uniquely leveraging the characteristics of SSDs, and uses advanced inline data reduction methods to reduce
the physical data that must be stored. XtremIO’s storage system uses industry-standard components and custom designed intelligent software to deliver unparalleled levels of performance, achieving consistent low latency for up to
millions of IOPS.

XtremIO has always provided simple, easy-to-use management. The XtremIO Management Server (XMS) delivers an HTML5 user interface that is a simple and easy-to-use interface for storage administrators. XMS allows storage administrators the ability to provisions storage with very little setup and planning.
XtremIO is designed and optimized for databases and for DBAs, providing the following benefits.


Predictable Performance
XtremIO provides predictable and consistency low-latency performance. With XtremIO scale-up and scale-out architecture, year-to-year growth is easy. Initial investment is preserved and application performance is improved. The
performance is predictable and provides sub-millisecond response times regardless of the workload and environment – be it production, QA, test or development.


Incredible simplicity
With XtremIO, there is no need for planning and tuning the location and number of database files. XtremIO is a real scaleup / scale-out and N-active-active storage controller architecture in which all the array volumes are served by all array resources. All DBA tasks are fast and simple with 1 to 3 steps.


Agility
The typical enterprise applications require multiple copies such as test/development, reporting or online analytics. DBAs and test/dev engineers often have to spend hours managing the DB creation and refreshing the environments while often
being limited by capacity, performance and number of copies. XtremIO’s Integrated Copy Data Management (iCDM) allows for instant XtremIO Virtual Copies (XVCs) to be created from production with no performance impact. These copies can be repurposed for near real time analytics, test/dev and any other use case- all with complete space efficiency.
Protection


Protecting the database is easy with XtremIO. There is no need for any design covering RAID type, data file capacity, load balancing, and tuning. The data is protected with a proprietary flash-optimized algorithm called XtremIO Data Protection
(XDP). XDP is very different from RAID in several ways. Since XDP is always working within an all-flash storage array, several criteria were important in the design of this protection scheme. XDP benefits include ultra-low capacity overhead,
high levels of data protection in case of double SSD failure, rapid rebuild times, flash endurance, and of course extreme performance.

With XtremIO virtual copies it is easy to protect and recover from any operational and logical corruption; XVC’s allow the creation of frequent point-in-time copies (according to RPO intervals – seconds, minutes, hours) and use them to recover from any data corruption. An XVC can be kept in the system for as long as needed. Recovery using XtremIO virtual copy is instantaneous and does not impact system performance.

You can download the white paper by clicking the screenshot below

VMworld 2018, The XtremIO Sessions

VMworld is coming up soon, below you can see our sessions schedule for the conference.

Chhandomay will talk about mixed workloads consolidation on Monday at 2PM.

Marco will talk about our recently released Metadata-Aware replication on Tuesday at 1PM

Yours truly will give a (condensed) session on some best practices on Sunday at 4PM

In order to schedule any (or all!) the sessions, just click the screenshot below

 

Virtual Storage Integrator (VSI) 7.4 is here

We have just released the 7.4 version of our vCenter plugin, aka, the VSI plugin, this plugin contains some new and highly requested functionality by many of you, our customers and partners.

XtremIO Native Replication / SRA

VSI can now show you all the available point in time (Pit) per a CG AND can instruct VMware SRM to failover / test failover to a specific pit, this is huge! Why? Because VMWare SRM only allow you to failover to the last point in time which may be problematic in cases of logical data corruption, think a virus / malware outbreak, no matter how many points in time you have, you can only failover to the last one..with this release you can simply point it to a specific pit and viola! , please note that it will also require a new version of our SRM SRA which will be out soon.

You can view a demo I created which shows how to deploy it and how to use it with an XtremIO array

DRR (Data Reduction) per Volume

I can’t even begin to tell you how many email I get about this one..so what is it?

By default, vCenter doesn’t have any awareness to array side capacity information, this was all good in the old days where dedupe / compression where rare at the array backend but these days, thin provisioning, deduplication and compression are a must for any all-flash array out there, add to this the complexity of allowing to have thin provisioning at the vSphere level and you got yourself a nice “I have no idea how much capacity is really consumed by a given datastore”, this day is over, yes, you could already view some info from the backend using the array UI (XMS) but in many cases, the vSphere administrator doesn’t have access to it so now you can view this info using the VSI plugin, this require XIOS 6.1 which has been out for month now..

It’s been a while since I recorded a VSI demo and I got many requests to show how to deploy it etc so below you can see a demo I recorded which shows you how to do it all!

As always, you can download the new plugin from support.emc.com or by clicking the screenshot below