Recently I made the decision to temporarily move away from VMware VSAN. Whilst there was a lot of things about VSAN that I liked (close integration with the vSphere platform for example), my home lab exists to enable me to evaluate a number of different solutions.

My lab consists of three hosts with no shared storage. At this point, my options consisted of purchasing some fibre HBAs, a switch and a SAN of some sort, or sticking with Software Defined Storage. But if not VSAN, then what?

Other posts in this series:

1. EMC ScaleIO
2. PernixData FVP 3.5
3. PernixData Architect 1.1 (coming soon)

EMC ScaleIO

ScaleIO was founded in 2011, and later bought by EMC in 2013. To quote Wikipedia:

“EMC ScaleIO converges the storage and compute resources of commodity hardware into a single-layer architecture, aggregating capacity and performance and scaling to thousands of nodes. It combines HDDs, SSDs, and PCIe flash cards to create a virtual pool of block storage with varying performance tiers. It features on-demand performance and storage scalability, as well as enterprise-grade data protection, multi-tenant capabilities, and add-on enterprise features such as QoS, thin provisioning and snapshots. ScaleIO operates on multiple hardware platforms and supports physical and/or virtual application servers”

It consists of a number of components. The first is a block storage server (SDS) which sits on each host and serves storage to clients. The second is the client itself, which can also sit on the same server, although this is not a requirement. The third is a meta-data manager, which monitors the health of the system.

It has self-healing and auto-balancing capabilities to ensure your data is well looked after.

The lab

Each host has a Samsung PM863 and two SM863 SSDs – the former previously used for caching, and the latter for capacity:

ScaleIO has the ability to make use of devices of different size and manufacturer, so mixing the 480GB and 960GB isn’t an issue.

To install ScaleIO on ESXi you will need the following:

• A Windows host (for PowerCLI)
• PowerCLI
• Java 1.8
• The ScaleIO software (obviously)
• Three vSwitch port groups

For the latter I will be using VLAN10, ScaleIO_data1 & ScaleIO_data2.

Please note: if you only use distributed vSwitches, and there are no standard vSwitches defined on your hosts, then the plugin deployment script (shown next) will fail. This is because the template which we will deploy needs a standard vSwitch to attach to.

To work around this, create a standard vSwitch without any uplinks or port groups on your hosts.

Deploy the plugin

Unzip the EMC-ScaleIO-vSphere-plugin-installer-2.0-xxx.x.zip file. Using PowerCLI, run the ScaleIOPluginSetup-2.0-xxx.x.ps1 script:

Enter the vCenter address, username and password:

Choose 1:

Type Y:

As this was my first ScaleIO installation, I chose S (standard):

Enter the IP address of the workstation you are running the script from:

If you have a browser window open which is accessing the vCenter Web Client, close it first. Open it it again, and the plugin should be registered:

At the PowerCLI prompt, press Enter:

Choose 3:

Type the name of your vCenter datacentre:

Type the path to the ScaleIOVM_2nics_x.x.xxxx.x.ova OVA file:

Type the name of a local datastore on each ESXi host. In the following example, I have created DS1 on ESXi1, DS2 on ESXi2 etc:

Type 4 to exit the script.

In my example, I have three hosts each with a local datastore (which consists of the remaining free space on the ESXi boot volume). As a template was deployed to each, I now have three in my inventory:

However for each template, the network is configured for the vSS port group I created earlier:

This needs to be changed.

In the vSphere Web Client, right-click each template and select Convert to Virtual Machine…

Expand the cluster and select the host you would like the converted template to reside on, then click Next

Click Finish.

Repeat the above steps for each template. When each template has been converted to a virtual machine, edit the settings of each one and change the networks to the appropriate port groups:

Finally, right-click each VM and select Template / Convert to Template…

Configure a ScaleIO cluster

In the vCenter Web Client, click ScaleIO on the home screen:

Click Install SDC on ESX:

Select the hosts you wish to install the SDC onto. For each one, type the root passwords in the boxes provided, followed by Install:

Click Finish. Finally, reboot each host.

Configuring ScaleIO

Once the hosts have been rebooted, log back into the vCenter Web Client and click ScaleIO. Click Deploy ScaleIO environment:

Click Next

Click to check the box I agree to licence terms and conditions, then click Next

Give the system a name, and type and password for the admin account. Click Next

From the drop-down box, select your vCenter:

Select the hosts you wish to configure and click Next

Select which system should host a meta-data manager or tie-breaker. As my cluster only has three nodes, I didn’t bother with the standby systems:

Click Next

Select MDM, SDS and SDC, then click Next

Give the Protection Domain a name and then click Add, followed by Next

In the following example, I have created two storage pools, Gold and Silver.

Type a name for the Storage Pool and click Add, followed by Next

Click Next

A ScaleIO Data Server (SDS) resides on each hosts that contributes storage. In this case, an SDS will exist on each ESXi host.

Select all hosts and click Next

Select the disks you would like to use and click Next

The ScaleIO Data Client (SDC) sits on each host that needs to access data served by the SDS. As there are only three nodes in this example, each host utilise an SDC:

Select each host and type the root password in the box provided, followed by Next. From the drop-down box, select Disable, followed by Next

Choose a host to run the ScaleIO gateway, and type a password in the box provided, followed by Next

A ScaleIO virtual machine will be deployed from each template to each ESXi host. Each VM will run an SDS for serving storage and an MDM for managing storage. The SDC, which will consume the storage, is embedded directly into ESXi.

Type a password in the box provided. This will be the admin password for each ScaleIO virtual machine once it has been deployed.

From the drop-down box, select each template and then click Next

For each network, use the drop-down box to select the appropriate network. In the following example, I have created three port groups on my distributed vSwitch:

Click Next

Configure each IP subnet as desired. When complete, click Next

Click Finish.

ScaleIO will begin the deployment task:

Once completed, click Finish.

Configure volumes

One of the last remaining tasks is to create storage volumes inside our pools for our ESXi hosts. In the following example, I will create a 400GB volume in the gold storage pool, and a 1.6TB volume in the silver one.

Using the vSphere Web Client, select EMC ScaleIO from the home page.

In the left-hand pane, click Storage Pools. Here will see any pools you have created. Right-click a pool and select Create Volume:

Give the volume a name, configure the size and select the SDCs (ESXi hosts) you’d like the volume to be consumed by:

Click OK

Click Close

In my example, the end result looks like:

In the vSphere Web Client, select an ESXi host in your cluster and click the Related Objects tab. Click Datastores, followed by the Create New Datastore icon…

Click Next

Here you will see the volumes you created previously. Continue you through the wizard to name and format them as needed.

Install the GUI

Double-click the EMC-ScaleIO-gui-2.0-xxxx.x.msi file to begin setup:

Click Next

Click to check the box I accept the terms in the License Agreement and click Next

Choose a location and click Next

Click Install

Click Finish.

Once installed, open the EMC ScaleIO GUI:

Enter the IP of the primary MDM, a username of admin and the password you specified during installation, then click Connect.

Click Confirm to accept the default certificate. Once authenticated you will see the ScaleIO GUI:

Final thoughts

I’ve only played with ScaleIO for a short time, but its advantages are immediately recognisable. With support for heterogeneous platforms, and its ability to scale up to a thousand nodes it is certainly impressive.

As it’s also block storage (albeit distributed), it opens up the possibility of using best-of-breed acceleration solutions such as PernixData’s FVP, which is not available to those looking to accelerate object-based storage such as VMware Virtual SAN.

And lets not forget its potentially biggest advantage over its rivals… it’s free.

Are there any drawbacks? Well maybe some, but none too big that spring to mind in such a short evaluation.

It could be argued that for a three-node cluster, it takes a lot of components to get the solution up and running. An SDC, SDS, MDM… and that’s all per host. Plus a gateway. That’s a lot of components that need to be provisioned, that VSAN takes care of by itself. It’s worth adding that only one ScaleIO virtual machine is deployed to each host, running the necessary components.

It also requires a Windows host running PowerCLI, which may be seen by some non-Windows shops as just another headache.

Coming up

Now that the storage tier has been provision, the next step is to accelerate it.

In the next part of this series, I will install and configure PernixData’s FVP to do just that.