Tips When Choosing a Cloud Provider

I just finished a short-term project where I VM’s hosted in a VCD backed cloud and ran into a few issues during the process that might help others when choosing a cloud provider.

I started the project out by calculating resource needs. I then talked with numerous vendors about their clouds and if it would work with our process needs. All of them were interested in our project and wanted to help but I soon realized not all vendors are created equal.

After selecting a vendor I discovered some topics that needed to be talked about before a vendor was selected. Here’s some questions/features which need to be communicated with your potential cloud provider before signing any contracts.

  1. Base VM’s – How VM’s will be created within vApps. Use templates from vendor or build your own. Uploading media can count against your network bandwidth quota.
  2. VM Resources – You may not need as much CPU/Memory if not all is needed or guaranteed.  Performance test in-house with different reservation and resource pool limits.
  3. Application/Workflow – In our instance we needed Fast Provisioning enabled. Provider did this but didn’t restrict LUNs to 8 hosts and they were on vSphere 5.0.  Once we had 9-10 instances running we couldn’t power on any more. Had to ship hardware overnight in case vendor couldn’t get this working for us.
  4. Length of contract – make sure you have enough time (and network bandwidth) to get any VM’s or data off before your contract expires. I was close to not getting a couple of things back I really needed.
  5. Thin Provisioning – wasn’t needed in our environment as we really needed linked-clones (Fast Provisioning) but might be able to use this to save disk space.
  6. Normal update outage times – In our case the provider did updates to VCD Thursday nights. I didn’t pay attention to this and was testing my application during this time. The test failed and it took me hours to realize the issue.

There are other things to pay attention to, and I’m sure you can find a ton of posts on them, but these are observations from my journey to the cloud for a specific project.


Lab Manager to vCloud Director Import Script

One major challenge when migrating from Lab Manager to vCloud Director is getting all of your VM Templates, Library entries, and needed configurations migrated across.  When I first looked at this it there was a partially scripted process using perl code that was long and very hard to understand.  I found it was better to do the process manually at that point as I’m not any good with perl and I was still trying to get VCD down.

Once PowerCLI released some VCD cmdlets we revisited the process and through lots of searching, digging through extension data, multiple posts on the communities, etc… we were able to automate the import process.  It was long and ugly, and failed a lot of the time so we didn’t use it.

We have migrated a lot of departments to VCD but still have a few more to go so I thought I’d refresh my script to see if it would work better.  I was thrilled to see the improvements in PowerCLI cmdlets for VCD.  I was able to cut out almost 100 lines of code from the script, and it now works properly, so I thought I’d share a few changes.

First change is thin provisioning the VM.  This script was used to find the datastore with the most free space:

$datastores = get-datastore | select Name,FreeSpaceMB
$LargestFree = "0"
$dsView = $null
foreach ($datastore in $datastores) {
    if ($Datastore.FreeSpaceMB -gt $LargestFree) {
            $LargestFree = $Datastore.FreeSpaceMB
            $dsView = $

Updated code saving eight lines of code:

$dsView = Get-Datastore PD* | Sort FreeSpaceGB -Descending | Select-Object -First 1

Once you get the datastore ($dsView) you want to svMotion to:
Previous script:

Get-VM $vm2Import | % {
            $vmview = $_ | Get-View -Property Name
            $spec = New-Object VMware.Vim.VirtualMachineRelocateSpec
            $spec.datastore =  $dsView.MoRef
            $spec.transform = "flat"
            $vmview.RelocateVM($spec, $null)

With PowerCLI 5.1:

Get-VM $vm2Import | Move-VM -Datastore $dsView -DiskStorageFormat Thin -confirm:$false

As you can see this saved six lines of code. That doesn’t sound like a lot but it all adds up.

One of the major changes is the internal vApp network creation. The vApps all use internal vApp NAT Routed networks.  Using some code from Clint Kitson’s Unofficial VMware vCD Cmdlets the following code was used to create the internal routed network:

$orgNetworkName = "importOrgNetwok"
$orgNetwork = (Get-Org "importOrg") | where {$ -eq $orgNetworkName}
$mynetwork = new-object
$mynetwork.networkName = "vAppInternalNetwork"
$mynetwork.configuration = new-object
$mynetwork.configuration.fencemode = "natRouted"
$mynetwork.Configuration.ParentNetwork = New-Object
$mynetwork.Configuration.ParentNetwork.Href = $orgNetwork.href
$mynetwork.Configuration.IpScope = new-object
$mynetwork.Configuration.IpScope.gateway = ""
$mynetwork.Configuration.IpScope.Netmask = ""
$mynetwork.Configuration.IpScope.Dns1 = ""
$mynetwork.Configuration.IpScope.ipranges = new-object
$mynetwork.Configuration.IpScope.IpRanges.IpRange = new-object
$mynetwork.Configuration.IpScope.IpRanges.IpRange[0].startaddress = ""
$mynetwork.Configuration.IpScope.IpRanges.IpRange[0].endaddress = ""
$mynetwork.Configuration.features += new-object
$mynetwork.Configuration.features[0].isenabled = $false
$networkConfigSection = (Get-Org "importOrg" | get-civapp "importVappName").ExtensionData.GetNetworkConfigSection()
$networkConfigSection.networkconfig += $mynetwork

With PowerCLI this was cut down to one line:

New-CIVAppNetwork -VApp $newVappName -ParentOrgNetwork ($orgInto+"-OrgNetwork") -Name 'vAppInternalNetwork' -PrimaryDns '' -Routed -Gateway '' -Netmask '' -DisableFirewall -StaticIPPool " -"

That’s a 20 line savings!!!

Once a script works I don’t usually revisit it unless there’s and issue with it but these three examples saved 34 lines of code, even more if you add comment lines no longer needed. I may have to go back through some other scripts and see what I can do with them.

vCD Expired VM Cleanup Fails

While renewing a vApp from Expired Items in an Organization I decided to cleanup a couple vApps which expired over 4 months ago. I deleted them and didn’t think much of them until I was working in vCenter an hour later.  I noticed recurring tasks trying to power off the VM’s that belonged to these vApps.

Looking at the VM’s they were suspended.  For this to happen the owners ignored the running lease time at which time the VM’s were suspended and the vApps stopped.  When the storage lease time expired the vApp went to Expired Items but the VM was still suspended.

Now that I deleted the vApp the process to remove it from vCenter must only check if it is not powered on.   Since it is suspended the power off task fails and vCD keeps trying this process. All I had to do to correct this was power on the VM and in a couple minutes vCD powered it off and cleaned everything up.

I was thinking I could write a PowerCLI script or vCO workflow to check for this and fix the VM’s.  You wouldn’t want to fix all “Suspended” VM’s as a user might want to suspend them on their own.

“Fence” vApps or use Internal vApp Networks

Over a year ago we started looking at vCloud Director as it was slated to replace Lab Manager.  The beauty of Lab Manager for us was we could clone the same VM’s thousands of times and have them “fenced” from each other for software development, training, automated testing, and supporting clients. We could do this very rapidly and save a lot of disk usage costs with Linked-Clones as well.

We ran into an issue with some of our software packages that didn’t like the IP address or hostname changing during the guest customization process though.  Another issue was the confusion created by the VM using IP’s from the Physical Network pool and and the virtual routers using the same pool for NAT’d access to the VM’s.  We worked around all of this by using Network Templates for the VM’s and connecting those to the Physical Networks upon deployment.  This way the VM’s have the same IP (192.168.x.x) and MAC address and they are still “fenced” from others VM’s on the external network (10.x.x.x).

It took awhile to figure this out in vCloud Director when we were first testing it.  After a couple days of reading posts and numerous tests I figured out how to mimic this setup.  I thought I’d share as I’ve answered this question numerous times on the communities pages.

Whether modifying a current vApp or manually creating a new one you’ll select “Add Network” to create a new network for this vApp.


You can use the default network if you wish and just add DNS information or you can change the settings to your liking.


Name the new network and finish it’s settings.


The network setup will now look like this. You have the option to automatically use IP’s from the pool you created, manually assign and IP from the subnet, or use DHCP from the vShield system. Click Next when done as we still need to connect this internal network to an Organization Network.


Click “None” in the connection column and select the proper Organization Network. Decide if you want to use NAT only or add firewall settings as well.  You can also have this vApp keep the same IP it gets from the Organization Network if you’d like. We don’t do this so IP’s are not reserved when not in use.


At this point we can capture this vApp to a catalog but there’s a major gotcha.  In order for the internal network to be saved you must select “Make identical copy” on the capture screen. When someone adds this vApp Template to their cloud they won’t have to go through this process.  They may have to change the connection between the internal network and organization network depending on their needs and if deployed across organizations.


vApp Template not capturing NIC’s “IP POOL” address setting

*** UPDATE ***   The fix has been added to the 5.1.2 release that is now GA.  Read the release notes for other fixes here…

While setting up and testing a couple new vApp Templates for a new organization I noticed an issues when capturing to the catalog and using IP Pool for IP addressing of the VM’s.  vApps deployed from the new vApp Templates where not pulling an address from our IP Pool setup within our External Network.  I messed around with the VM inside the vApp I used to capture the vApp Template from.  I thought it had something to do with the new UDEV stuff in RHEL 6.  It looked correct so I tested by converting it to a Template within vCenter and doing a Guest Customization deployment from it.  The VM was working correctly.  We are currently on version

I finally looked closer at the vApp Template and noticed the NIC for the VM was set to DHCP instead of IP Pool.


I checked some vApp Templates I’d captured while on vCD 1.5.0 and they were OK.  I checked another from when we were on 1.5.1 and they were set to DHCP so something changed at that point.  We don’t use these templates very often and weren’t on vCD 1.5.1 very long so no one complained.

Doing a quick search I came across this post on the communities page.  There’s a SQL query to find the NIC for the VM inside the vApp Template based on Catalog and vApp Template name.  I had 11 templates to update so I modified it to use variables for the catalog and template name.

DECLARE @Catalog varchar(80)
DECLARE @vAppTemplate varchar(80)

SET @Catalog = 'catalogName'
SET @vAppTemplate = 'templateName' -- May need % at the end in case vApp Template was copied/moved. May get multiple returns though so be careful.

select nic.nic_id, nic.mac_address, nic.ip_addressing_mode from network_interface as nic inner join vapp_vm as vm on nic.netvm_id = vm.nvm_id where svm_id=(select svm_id from vapp_vm where vapp_id=(select entity_id from catalog_item where name like @vAppTemplate and catalog_id=(select id from catalog where name = @Catalog)))

When running the above query you should get the following output:


Change the last line of the code to the following and run again.

update network_interface set ip_addressing_mode = 1 where nic_id = (select nic.nic_id from network_interface as nic inner join vapp_vm as vm on nic.netvm_id = vm.nvm_id where svm_id=(select svm_id from vapp_vm where vapp_id=(select entity_id from catalog_item where name like @vAppTemplate and catalog_id=(select id from catalog where name = @Catalog))))

It should update 1 row.  If you run the original query again and look at the NIC properties of the VM within the vApp you will now see the following:



Deployments will now pull from IP Pool as intended.

Per to communities post above there is a hotfix coming out soon for those who submit an SR. Otherwise it will be fixed in the next maintenance release.

Powered by

Up ↑