Of Birds and [Machine] Migrations

  • Posted on: 8 April 2015
  • By: David La Motta

That right there, my friends, is a bar-tailed godwit (limosa lapponica).  "And why do I care?" --you ask.  Because those little rascals do migrations better than any other bird on the planet.  They are known to fly from Alaska to New Zealand, all the way over the big blue Pacific ocean, without stopping! That is over 7000 miles of non-stop flight, no water breaks, no sleep, no food.  Simply amazingly brutal, if you ask me.

But back to why you should care, lets focus on our world of IT and put our feathered companion's migrations into context.  With Integra, we can do hundreds of machine (physical or virtual) migrations in parallel, only constrained by the amount of bandwidth your network can provide.  And, we can do this for both Linux and Windows machines alike.  This means you can schedule a migration and then go have dinner, enjoy a cocktail, or simply have the luxury of getting some shut eye.  Integra handles everything on the migration path, sending as many notifications as you'd like if things are going smoothly--or not.  IT infrastructure migrations are rather sensitive, so Emitrom (or one of its partners) will stand along your side to make sure virtualizing your environment goes much like a godwit's flight to New Zealand and back: successfully.  Do contact us if machine migrations are on your radar.

With all this said, lets see how we migrate a Windows 8.1 machine to an .ova using Integra.

Meet the Godwit Provider

Integra's migration engine is aptly named the Godwit provider.  This provider's mission in life is to create an .ova out of a physical or virtual machine.  As a matter of fact, for all intents and purposes every machine that Godwit converts is treated as a physical machine.  This approach has its pros and its cons, with the most obvious drawback being that a machine's disk is fully exported even if only a fraction is being consumed.  Regardless, remember that the whole point of using Integra's Godwit provider is so that you don't have to babysit your migrations, so full vs partial disk exports are of no concern.  Furthermore, the pro that trumps the con is that our migration does not discriminate between a physical or virtual machine, be it Linux or Windows.  With an .ova in hand, you could potentially find ways to import it to:

  • Azure
  • EC2
  • XenServer
  • KVM
  • Hyper-V
  • vCloud Director
  • ...
The first incarnation of the Godwit provider uses VMware tools to do the conversion; as a matter of fact, it uses the same tools you would.  Think of the Godwit provider as merely the mediator between you and hundreds of conversions, automating the use of software that has been used and tested time and time again.  One distinction to be made is that the Godwit provider does all this programmatically, as opposed to doing things by hand.

"Any Sufficiently Advanced Technology Is Indistinguishable From Magic"

Surely you've seen Arthur C. Clarke's famous words... but no magic here, friends, just pure automation bliss.  There are a few broad steps that we take in order to migrate a machine.  We eat our own cake, so to speak, and:

  1. Use Integra's PowerShell provider to deploy and run the Godwit provider on the Windows guest being converted.  No need for this step on Linux, and PowerShell remoting must be enabled on Windows.
  2. Use Integra's Godwit provider to actually drive the migration to an .ova
  3. Use Integra's Notification provider for sending emails and SMS messages in case of failure
An implicit step is not mentioned but we also use Integra's Java JSON API to configure providers, actions, workflows and schedules.  I would feel bad for using Automation Ninja as one of my Twitter aliases if we weren't leading automation by example!  Anyway, leveraging our API comes in handy for as little as one conversion or hundreds of conversions.  If you have read any other of blog posts here, you will know that I am partial to Groovy.  The resulting script takes care of setting everything up and leaves the system ready for you to either click the execute button, or to enable the schedule so that it runs.  It should be noted that we have the ability to perform conversions in batch.  Always remember that you have full control of what you are automating with Integra, so if you want to execute your conversions in batches of 5, or 10, or 100, that is something that can we can certainly do.

Since the Godwit provider first needs to be installed on the guest, the result of the first pass of the script is configuring and running the PowerShell provider and actions.  Lets take a look at that.

First Phase: Godwit Provider Deployment

In the first pass or step of our conversion process, we focus on deploying the Godwit provider on the guests by leveraging Integra's PowerShell provider.  If this approach is acceptable, the owner of the machines being converted must provide hostname or IP address and credentials for all the machines being converted.  We will need those details to deploy and run our software remotely.  If the machine owner accepts the remote installation of software but still has a hard time giving up credentials, there are other paths we can take to reach the desired destination.


We only need one instance of the PowerShell provider running, despite the number of guests being converted.  If the gears in that automation ninja mind of yours are already turning, yes, with the PowerShell provider and with remoting enabled on your Windows guests you can do all sorts of things from Integra.  Again, in our case we are going to use it to copy the Godwit provider and leave it running on that Windows machine.


Here we see 6 actions.  They range from getting a PSCredential object, mapping a new drive, doing a remote copy, copying the Godwit provider, running it and finally disconnecting the network drive.  One of the interesting things that the Groovy script does is that it uses the guest name (or IP in this example) in the action names to make things more readable.  Imagine hundreds of machines being converted, so having the guest name as part of the action name allows you to quickly edit intricate settings that may apply to only one guest.


As you may imagine, the 6 actions previously configured are used to build a single workflow.  The important thing to remember here is that should you have more than 1 machine being converted, you would effectively see as many workflows as you have guests--with 1 deployment workflow per guest.

Second Phase: Configuring Godwit Provider

If the guest owner is happy installing the Godwit provider by him- or herself, then that is fine as we could skip directly to this second phase.  The need for a 2-phase approach is so that we can install and configure the very software that we are going to make use of.  It is like using a key to open a lock: you first need the lock before you can make use of the key.


Once the Godwit provider is up and running on the guest, we can proceed to adding and configuring new migration-related actions, workflows and schedules.  In the image below you will also notice that the Notification provider was added as well.  We are going to rely on that provider to send email notifications when conversions start, finish or fail.


With the Godwit and Notification providers in hand, the Groovy script configures many more actions for driving the conversion of the Windows machine and also for sending email notifications.  As you can see from the image below, the actions to convert a single machine can start to add up.  With a script these actions can be deleted, added and reconfigured in a matter of seconds.


There are two additional workflows that we will need in order for the conversion.  One is a workflow with a single action which sends an email; this workflow is used in a backward chain, effectively being invoked only if there is a failure on the main conversion workflow.  Since it only contains one action, that workflow won't be shown.  On the other hand, the conversion workflow really gets interesting.  See below for the steps we take to convert a machine and you may see why it is that we can run conversions in parallel.  If you don't, strike a conversation with us in our Google+ community and we'll be happy to discuss.


This blog started with the offer of you having ability to go enjoy yourself while the conversion took place.  As such, we have two schedules for things to execute: the first schedule will deploy the Godwit provider on all the guests you wish to convert, and the second schedule will actually perform the conversion.  This maps pretty well to our 2-phase approach, with one group of operations taking place before the other.  If you pay close attention to the image below, you will see that we have deliberately disabled both schedules.  As you might guess, that's because we are daring automation ninjas and we want to execute things on demand.  With that said, we click the green execute button in the menu bar and our conversion kicks off (asynchronously).  At this point all the steps of the workflow are being executed.


In this example the workflow creates an .ova file out of a Windows 8.1 machine.  As mentioned earlier, we could have more actions that would post-process that .ova and take it yet to other destinations.  For the purposes of this blog, however, we see the resulting .ova sitting pretty in our network share and receive and email notifying us that the process finished with success.  In a future post we will take the resulting .ova and import it into one of the locations mentioned above.


We saw that converting both physical as well as virtual machines is possible with Integra, leveraging the same tools you would otherwise use to do conversions by hand.  Even if you are an experienced programmer, managing conversions programmatically is not for the faint of heart.  By leveraging the Integra platform, however, we can parallelize conversions programmatically in a very simple manner.  The workflows above are simply an example of the things we can do to perform a machine migration; we could add more notifications, SMS, etc., as workflow steps before a long-running step starts and ends, for example.

We do not distribute the Godwit provider nor any of the tools it leverages.  Instead, please contact us so we can work with you on your migration efforts.  Migrations are delicate operations, but they are ripe for automation: an area where Integra shines and can help you save time and peace of mind.

Happy migrating!