One of my customers asked for a procedure for cloning Azure environments. An environment is a resource group with several Windows VM on it. Since that is not the first time that I have seen similar requests, I would like to share a script using Azure Automation that could help cloud administrators who may face the same challenge. (For an update on this article with a script to clone only a single VM, go here.)
There are a couple of elegant ways to achieve the goal. The first one that comes to my mind is by adjusting their Azure DevOps pipeline, add some variables, and some massage of the ARM templates, and, voilà, we would have a shiny new environment. A second one would be using Azure Site Recovery (ASR) to create test failover scenarios.
After some discussion, the reality was that all environments in Azure were part of a lift-and-shift migration. Those environments could have several servers, lots of data, and the goal was to replicate those servers as is. On top of that, some cloned environments might stay for a while in the subscription.
Another critical point about the scenario was that the cloned Azure environments can happen very often and they may stay for a few days or even months. The frequency of cloning Azure environments was another critical point. They could be very frequent.
My decision was to create a script and then convert it to an Azure Automation where the customer provides the resource group and a clone name. The script will retrieve the current virtual network and subnet of the source servers and mimic that network infrastructure in the cloned environment.
- Create a new resource group for the cloned environment adding the prefix _<CloneName>_Bubble.
- Create a new virtual network and subnet structure in the target. We will copy all the naming conventions and IP settings from the original virtual network.
- Create a snapshot of all VM disks in the source resource group and create managed disks in the target resource group.
- Create new VMs using the information from the source VMs. Attach the brand new managed disks in the new VMs.
The script process started with a few functions in PowerShell, a lot of validation and tests in different types of environments. When a good working condition was achieved, we realized that the environment (environment and resource groups will be used interchangeably in this article) could have anything between a few to as many as 50 VMs.
When running the first version of the PowerShell script in Azure Automation, the time to clone an entire environment with 19 VMs took around four hours so a new requirement was introduced: The process should be as fast as we could get.
Because of this new requirement, we introduced PowerShell Workflows in the picture, which is supported in Azure Automation and allows tasks to run in parallel, with some caveats.
The complete script is available at my GitHub, and you can click here. We are going over the main areas of the script to help you to understand the code and apply it to your environment.
The script: Environment validation
The script will validate a few items in the environment before moving forward with the creation of a virtual network and VMs. These following conditions are going to be checked as part of the process. If they exist, then the script will not execute, and manual cleanup will be required.
- The target resource group must not exist.
- All VMs in the source resource group must be turned off.
- All VMs (virtual network interfaces) in the source resource group must share the same virtual network.
Using PowerShell Workflows
There are some advantages when using PowerShell Workflow, such as multithreading (parallel) and checkpoint capability, which allows the Workflow to be resumed. But there are also some caveats when using PowerShell Workflows, and we will address some of them in a separate article here at TechGenix.
You may be wondering how the Azure Automation knows the difference between Workflows and regular PowerShell scripts. The PowerShell Workflow has a different structure, as listed below.
Workflow <Verb-Noum> {
<script code including parameters>
}
All code in PowerShell Workflows run on the Windows Workflow Foundation. The only exception is when we use InlineScript, which executes the code in our traditional Windows PowerShell.
The InlineScript activity can retrieve information from the Workflow by referencing variables using the following format $Using:<VariableName> and that helps to bring values between Workflow and regular PowerShell.
When you have a limitation using Workflow, the InlineScript activity can help to overcome the barrier, and we had to use that in our current script. We used it to create the virtual network and load all variables in a hashtable that will be used throughout our script execution.
Here is how the layout for the InlineScript activity was used in our script before we reach the stage to start provisioning cloned VMs.
$tControl = InlineScript {
Functions Area
Script Body area
$tControl #--> last line of the block to return the value to the variable in the workflow
}
Note: Functions created in the Workflow area cannot be consumed within the InlineScript. We must declare the function in the InlineScript block, and that is the reason that you see the same function in the code twice.
We also configured the preference variable $PSRunInProcessPreference to $True at the beginning of the script, and by doing that, we force all activities to run in the same process. For more information about preference variables, the following link can be used.
The functions
The script aims to reduce linear code and address all repetitive tasks using functions. In the Workflow area, we rely on four functions to deliver the consistent output to support all stages of our cloning process. Here is a summary of our functions:
- VMInventory($VMName): This function will receive a VM name, and it will create an array with all the information related to the VM, including disks, VM size, and network information. Use that function to add any additional information that you may need from the source VM.
- VMSnapshot($vmInfo,$HashTable): It will receive the array containing the VM info and an array with all the environment information. It will create snapshots of the existing VMs in the target resource group.
- VMRestoreSnap($vminfo,$HashTable): It will receive the same information from the previous function, and it will create managed disks of all snapshots that were created in the target resource group.
- CloneVM($VMInfo,$HashTable): The final piece of the puzzle, it will grab all the information about the VM, combine with the brand-new managed disks, and create a new VM attached to the cloned virtual network and subnet.
The main script section
Here is where we achieve the multithreading goal and can run multiple instances of cloning VMs by using -Parallel switch in the ForEach statement. Based on my observations, when using Azure Automation, the sweet spot was around six instances at the same time and to throttle the number of instances accordingly, we used -ThrottleLimit switch.
ForEach -Parallel -ThrottleLimit 6 ($vm in $VMs) {
The final action on the script is to remove any existing snapshots and provide how much time the entire execution took place.
Cloning Azure environments: Wrapping it all up
The goal of this article is to demonstrate how to use Azure Automation with PowerShell Workflows to go about cloning Azure environments.
The script is not a one-size-fits-all, and it was created to address some requirements and challenges of a specific customer. However, you can use it as a starting point to address some of the additional requirements that your environment may have if you want to go about cloning Azure environments.
There are a couple of items that you should be aware of when creating the current cloning script:
- We are not taking into consideration other common resources that could be easily part of your application/environment, such as load balancers, Azure Application Gateways, Key Vaults, Storage Accounts, and so forth.
- We are not taking all the VM properties: boot diagnostics, IPs and extensions. We are just making sure that we have the same VM size, same disks, and OS (in our case only Windows).
- The creation of Windows VMs uses a parameter to use existent licenses. (It may not be suitable for your subscription.)
- We haven’t tested the script against Linux VMs and encrypted disks.
- If your domain controllers or any auxiliary system are required in that “bubble” network, your script should be able to clone them as part of the process.
- There is no connectivity on that new VNet, and my recommendation is to use the Azure Bastion service to allow access to that environment.
Featured image: Shutterstock
