Hyper-V Checkpoint Production vs Standard

Hyper-V, Microsoft's virtualization platform, offers users the flexibility to create checkpoints for their virtual machines. Checkpoints are snapshots of a virtual machine's state at a specific point in time, allowing for easy restoration or troubleshooting. However, it's important to understand the two types of checkpoints available in Hyper-V: standard and production.

Production Checkpoints:

Production checkpoints are the default option for new virtual machines in Hyper-V. These checkpoints utilize backup technology within the guest operating system to create a "point in time" image of the virtual machine. This approach ensures full support for all production workloads and makes production checkpoints similar to application-consistent backups.

Key Characteristics of Production Checkpoints:

Backup Technology: Production checkpoints leverage backup technology inside the guest operating system, ensuring that the checkpoint is created in a manner compatible with production workloads.
Application-Consistent: By utilizing backup technology, production checkpoints provide application-consistent backups, meaning that they capture a point in time where applications and data are in a consistent state.

Suitable for Production Environments: Production checkpoints are designed to be used in production environments and are fully supported for all production workloads.

Standard Checkpoints:

While production checkpoints are ideal for production environments, Hyper-V also offers standard checkpoints primarily meant for development and testing scenarios. Standard checkpoints capture the state, data, and hardware configuration of a running virtual machine, making them useful for recreating specific states or troubleshooting issues.

Key Characteristics of Standard Checkpoints:

Capturing Running State: Standard checkpoints capture the complete state of a running virtual machine, including its data, hardware configuration, and current running processes.

Development and Testing: Standard checkpoints are particularly beneficial for development and testing purposes, enabling users to recreate specific states or conditions of a virtual machine for troubleshooting or experimentation.

Crash-Consistent: Unlike production checkpoints, which are application-consistent, standard checkpoints are crash-consistent, meaning they capture the state as if the virtual machine has experienced a sudden power loss or system crash.

Choosing the Right Checkpoint Type:

When creating checkpoints in Hyper-V, it is crucial to consider the intended use of the virtual machine and select the appropriate checkpoint type accordingly. Production checkpoints are recommended for production environments where workloads require consistent and reliable backups. On the other hand, standard checkpoints are more suitable for development, testing, and troubleshooting scenarios, where the focus is on capturing the running state of the virtual machine

Hyper-V deleting a checkpoint when the delete option is missing

Hyper-V is a powerful virtualization platform that allows users to create and manage virtual machines efficiently. One of its handy features is the ability to take snapshots, which capture the state of a virtual machine at a specific point in time. These snapshots serve as restore points, allowing you to roll back your VM to a previous state if needed. However, sometimes you may encounter an issue where the delete option for a snapshot is not available in Hyper-V Manager.

Method 1: Using the Keyboard Shortcut in Hyper-V Manager

When the delete option is missing from the Hyper-V Manager interface, you can use a keyboard shortcut to delete the snapshot. Follow these steps:

Step 1: Launch Hyper-V Manager

Open Hyper-V Manager by searching for it in the Start menu

Step 2: Select the Virtual Machine and Snapshot

In the left-hand pane of Hyper-V Manager, locate the virtual machine that contains the snapshot you want to delete. Click on the VM's name to select it. Then, select the specific snapshot you wish to delete from the list of checkpoints displayed in the centre pane.

Step 3: Press the Delete Key

With the VM and the desired snapshot selected, press the "Delete" key on your keyboard. This action will prompt a confirmation dialog.

Step 4: Confirm Snapshot Deletion

In the confirmation dialog, review the details of the snapshot you are about to delete. Ensure that you have selected the correct snapshot, as the deletion process is irreversible. Click "Yes" to proceed.

Method 2: Using PowerShell to Remove the Snapshot

If you prefer using PowerShell, you can employ the following steps to remove the snapshot using command-line tools:

Step 1: Launch PowerShell

Open PowerShell by searching for it in the Start menu or by using the Run dialog (press Win + R and type "powershell" followed by Enter).

Step 2: Remove the Snapshot

In the PowerShell console, run the following command to remove the desired snapshot:

Get-VM <VM-Name> | Remove-VMSnapshot

Replace <VM-Name> with the name of the virtual machine containing the snapshot you wish to delete. This command retrieves the specified virtual machine and pipes it to the Remove-VMSnapshot cmdlet to remove the snapshot.

Windows Hyper-V 2019 failed merge of snapshot with error 0x8007054F

 This is not a good fix and am not happy with it... seems a little too much like the nuclear option but it works and at the moment Microsoft are not being to forthcoming with answers and we seem to be stuck between the backup company and them pointing the fingers at each other,

I will update if we get a better fix.

But for now if you are seeing phantom snapshots where if you restart the Hyper V machine management server it will try to merge then fail with the following error code

0x8007054F

The issue appears to be the root parent, not being able to merge with the last checkpoint.  so you could roll all the AVHDX files back to the first one and that last one will not merge back with the primary disk.

Update #1 (02/12/2020)
After manually merge a few servers it is clear the issue is only with the System drive / First drive in the VM config.

Not sure if it because its the system drive or its the first one in the list in the VM config, but removing it from the VM config allows it too start merging the other drives as soon as the config is save.  this has allowed us to save a lot of time and cut down the work we need to do in the sort time.

Update #2 (07/12/2020)
Seems update 1 was false.  Although all up to that date had been the system drive / first one in the VM config the last two I cleaned up was data drives.  both at the end of the VM config list.

Still waiting on an update from Microsoft on this.

Update #3 (14/01/2021)
Microsoft are asking for the VM to be deleted and remade so that it makes a new VM ID,  we are testing this and will update if this has an impact.

Update #4 (09/02/2021)
The VM we have remade have not had issue with check pointing but the others have not either so its still up in the air at the moment.

Update #5 (10/03/2021)
Confirmation this week that the issue has returned on guests where we have not remade the VM config but the ones we have are still running fine and merging back even on the same host.

Short term fix
The quickest way I have found so far to resolve this is to do the Following

1. Shutdown the virtual machine
2. Break and remove replication if its enabled
3. Open the virtual machines settings
4. Take a note of the disk and location of the VM has set,  should end in a AVHDX
5. Remove the disk
6. Open Hyper-V and select "Edit Disk" point it to the AVHDX you noted in point 4 and select "Merge"
7. Once on the Merge options select new disk and give it a name and a location
8. Attach this new disk to the virtual machine in the same controller as the one you removed
9. Boot the VM and confirm its working
10. Remove old VHDX and AVHDX files
11. Enable replication if you disabled it.

Doing it this way can be risky so always check your backups but it does work.

Updating Hyper-V Failover Cluster with Cluster-Aware Updating Tool

When you wish to update a failover cluster, I favour the manual process of draining and update each node individually this allows you to catch any issues with the Cluster due to the servers not failover over smoothly.

But there are times when it can be used and should be really as it helps a lot.  The first step is to run the CAU Best Practices Analyzer from within the Cluster-Aware updating Tool with an Administrator account that can admin across all the Nodes of the Cluster.  This will let you know if everything is ready for it to be used.

It should be ran every time there is a change to the nodes that also require Validate a Cluster Wizard to be ran.  it should so be ran if the Nodes updates sources have been changed.

Once everything comes back as ok the following recommendations are also made

1. All other update tools are removed like 3rd part update installers and Windows schedule updates should also be disable.  CAU should be the only tool used for updating on the Nodes and approving updates, this also includes WSUS schedule updating but WSUS can be used as a source
   
   
2. All Nodes must be configured with the same update source
   
   
3.  WSUS must correctly identify the approved updates for the cluster nodes and approve across all Nodes
   
   
4. Sufficient cluster nodes must be online so that the cluster has a quorum and Guest VM can quickly and correctly be drained form a Node with no errors. (If this can not happen it can make CAU fail and get stuck at times)
   
   
5. All cluster nodes must be in the same Active Directory domain and Cluster must be resolvable via DNS
   
   

More Reading 
https://docs.microsoft.com/en-us/windows-server/failover-clustering/cluster-aware-updating

Shutting Down a Hyper-V Failover Cluster / Hyper-V Host

Hyper-V Hosts and Failover Clusters are great way to maximums your hardware utilisation or redundancy.  The Failover Clusters and Hyper-V sit at the Infrastructure level.

Hyper-V gives you the ability to run multiple platforms on a single host where as the Failover Cluster allows you to take two or more Hyper-V hosts and build in redundancy of the infrastructure allowing a single or more platform to survive a failure at the infrastructure level, be it Network / Storage / RAM / CPU.

When it comes down to powering down this type of system off the best recommendation is too start at the Software level, then the Platform followed by the Infrastructure not the other way around as this can lead to issues.

Basically shutting down any Applications then the Guest VM then the Physicals.  If you have a Failover Cluster with a single Guest DC, this server should be left running to allow it to move to a paused state meaning that the Failover Cluster GUI will work correctly when the Cluster powers back on.  failing that you will need to PowerShell the Cluster as the GUI may stop working.

The DC can come out of a paused state normally error free but servers that have custom applications / 3rd party software installed may not react well to coming out of this state and reboot will be needed if they ever paused.