Save to My DOJO
I think that we can all agree that backup software exists for a reason. Well, lots of reasons. Very good reasons. So, if you ask me in the abstract how to make a copy or backup of a virtual machine’s virtual hard disk file while the virtual machine is running, I’m probably going to refer you to your backup vendor.
If you don’t have one, or don’t have one that you can trust, then I am naturally going to recommend that you download Altaro VM Backup. Backup is their wheelhouse and they’re going to have a lot more experience in it than any of us. The outcomes will be better than anything that we administrators can do on our own.
But, I also understand that sometimes you have one-off needs and you need to get something done right now.
Or, you need to script something.
Or your backup software isn’t quite granular enough.
Or you have some other need that’s unique to your situation.
If you need to get a copy or backup of one or more of a virtual machine’s hard disks without shutting down the virtual machine, you have three options, shown in their preferred order:
- Use your backup application, as we discussed.
- Export the virtual machine with Hyper-V Manager or Export-VM. This only works for Hyper-V versions 2012 R2 and later.
- Copy the file manually.
I’m not going to change my mind that a backup application is the best way to get that copy. But, I’m done beating that horse in this article.
Export is the second-best solution. The biggest problem with that is that it exports the entire virtual machine, which might be undesirable for some reason or another. It also locks the virtual machine. That won’t necessarily be a bad thing, especially if all you’re doing is getting a copy, but maybe it’s a showstopper for whatever you’re trying to accomplish.
That leaves us with option 3, which I will illustrate in this article. But first, I’m going to try to talk you out of it.
You Really Shouldn’t Manually Copy the Disks of a Live Virtual Machine
Manually copying a live VHD/X file isn’t the greatest idea. The best that you can hope for is that your copy will be “crash consistent“. The copy will only contain whatever data was within the VHD/X file at the moment of the copy. Any in-flight I/Os will be completely lost if you’re lucky or partially completed if you’re not. Databases will probably be in very bad shape. I’m sure that whatever reason that you have for wanting to do this is very good, but the old adage, “Because you can do a thing, it does not follow that you must do a thing,” is applicable. Please, think of the data.
OK, the guilt trip is over.
Just remember that if you attach the copied disk to a virtual machine and start it up, the volume will be marked as dirty and your operating system is going to want to run a repair pass on it.
Manually Copying a Hyper-V Disk the Dangerous Way
That header is a bit scarier than the reality. Most importantly, you’re not going to hurt your virtual machine doing this. I tested this several times and did not have any data loss or corruption issues. I was very careful not to try this process with a disk that housed a database because I was fairly certain that would break my perfect streak that way.
Use robocopy in restartable mode:
robocopy /z "C:LocalVMsVirtual Hard Disks" Z:Backup server1.vhdx
The format is:
robocopy /z SOURCE_FOLDER DESTINATION_FOLDER VHDX_FILENAME
It is important that you do not use a trailing slash on the folder names! If you want to copy multiple files, just enter them with a space between each.
Pros of the robocopy method:
- It’s easy to remember
- It works on anything that your account can reach — local storage, CSVs, SMB shares, whatever
- Is “good enough”
Cons of the robocopy method:
- Restartable mode (specified by the /z switch) is sssssllllllllooooooooow especially over networks
- There is no guarantee of data integrity. But, there’s no real guarantee of data integrity when manually copying a live VHD/X anyway, so that probably doesn’t matter.
- I doubt that anyone will ever give you support if you use this method
For basic file storage VHD/X files, this is probably the best of these bad methods to use. I would avoid it for frequently written VHD/X files.
Manually Copying a Hyper-V Disk the Safer Way
A somewhat safer method is to invoke VSS. It’s more involved, though. The following is a sample — do not copy/paste!
vssadmin create shadow /for=C:
mklink /d C:vssvolume \?GLOBALROOTDeviceHarddiskVolumeShadowCopy26
xcopy "C:vssvolumeLocalVMsVirtual Hard Disksserver1.vhdx" Z:Backup
rmdir C:vssvolume
vssadmin delete shadows /shadow={e47b5da0-0ae1-415b-b604-e94ff1913586}
This is going to need somewhat more explanation than the robocopy method. We’ll take it line-by-line.
The first line tells VSSADMIN to create a shadow copy for the C: volume. The VHD/X file that I’m targeting lives on C:. Substitute your own drive here. The shadow copy becomes a standard Windows volume.
vssadmin create shadow /for=C:
The second line creates a symbolic link to the newly created volume so that we can access its contents with the usual tools. You can discover what that lines contents should be via the output from the previous command.
VSS Admin Create Shadow
We use “mklink.exe” to create the symbolic link. The /D switch lets it know that we’re going to make a directory link, not a file link. After that, we only need to tell it what to call the link (I used C:vssvolume) and then the target of our link. It is vitally important that you place a trailing slash on the target or your symbolic link will not work.
mklink /d C:vssvolume \?GLOBALROOTDeviceHarddiskVolumeShadowCopy26
Next, we copy the file out of the shadow copy to our destination. I used XCOPY because I like XCOPY (and because it allows for tab completion of the file name, which robocopy does not). You can use any tool that you like.
xcopy "C:vssvolumeLocalVMsVirtual Hard Disksserver1.vhdx" Z:Backup
That’s all for the file. You can copy anything else out of the shadow copy that you want.
We need to clean up after ourselves. Do not leave shadow copies lying around. It’s bad for your system’s health. The first step is to destroy the symbolic link:
rmdir c:vssvolume
The last thing is to delete the shadow. There are multiple ways to do this, but my preferred way is to delete the exact shadow copy that we made. If you look at the output of your vssadmin create shadow command, it has all of the information that you need. Just look at the Shadow Copy ID line (it’s directly above the red box in my screen shot). Since VSSADMIN was nice enough to place all of that on one line, you can copy/paste it into the deletion command.
vssadmin delete shadows /shadow={e47b5da0-0ae1-415b-b604-e94ff1913586}
You’ll be prompted to confirm that you want to delete the shadow. Press [Y] and you’re all finished! If you want to see other ways to remove VSS shadows, type vssadmin delete shadows without any other parameters. It will show you all of the ways to use that command.
Yes, this works for Cluster Shared Volumes. Get a shadow of C: as shown above and copy from the shadow’s ClusterStorage folder. I would take caution to only perform this from the node that owns the CSV.
Pros of the VSSADMIN method:
- It’s completely safe to use, if you do it right.
- It’s not entirely perfect, but some quiescing of data is done. The copied volume is still dirty, though.
- Faster when copying to a network destination than robocopy in restartable mode.
- Works for local disks and CSVs. Won’t work for SMB 3 from the Hyper-V host side.
Cons of the VSSADMIN method:
- Tough to remember (but this blog article should live for a long time, and that’s what Favorites and Bookmarks are for)
- If you don’t clean up, you could cause your system to have problems. For instance, you might prevent your actual backup software from running later on in the same evening.
- May not work at all if another VSS snapshot exists
- May have problems when third-party and hardware VSS providers are in use
If the integrity of the copied data is important and/or changing frequently, you’ll likely get better results from the VSSADMIN method than the robocopy method. It’s still not as good as the other techniques that I promised not to harp on you about.
Not a DOJO Member yet?
Join thousands of other IT pros and receive a weekly roundup email with the latest content & updates!
More in this category
About the Author
Eric Siron
43 thoughts on "How To Copy or Backup a VHD File While the VM is Running"
Eric, I assume that creating a VSS aware checkpoint in Hyper-V 2016 and then copying the parent disk should somewhat result in your later example, also utilizing VSS. This way the clean up after copying should be rather simple by just removing the checkpoint.
What’s your take on this?
I have my doubts, but maybe. When a virtual machine is running from a checkpoint, the source VHDX is still owned and controlled and connected to by VMMS. It might allow you to copy it out, or it might not. I have not tried that. If you ran ROBOCOPY with /zb against it, it would probably work, but it would be sloooooooooow.
Eric, I assume that creating a VSS aware checkpoint in Hyper-V 2016 and then copying the parent disk should somewhat result in your later example, also utilizing VSS. This way the clean up after copying should be rather simple by just removing the checkpoint.
What’s your take on this?
Shouldn’t the xcopy command in the last method read something like “xcopy “C:vssvolumeLocalVMsVirtual Hard Disksserver1.vhdx” Z:Backup”
(note vssvolume in path)
It does! (now)
I was, uh, just trying to make sure that no one copy/pasted. Yeah. That’s it…
But seriously, thank you!
Shouldn’t the xcopy command in the last method read something like “xcopy “C:vssvolumeLocalVMsVirtual Hard Disksserver1.vhdx” Z:Backup”
(note vssvolume in path)
Hi, Eric, is it possible to do a delta copy of vhdx file?
I have a parent patittion vhdx(won’t change) and a delta vhdx file. This delta file is changing but I don’t want to waste time copying this whole file all the time. I can see Hyper-V replica doing this well by copying very little data every 5 min, so can I do this delta copy in a way without Hyper-V replica?
Regards
Andes
Well… yeah it’s possible. But you’d have to hook into all of the backup APIs to do it. I don’t know how. It wouldn’t be a simple thing.
Yeah, I see then. Hyper-V does have new features on resilient change tracking. This should be an starting point for me. Thanks.
What about creating a VM checkpoing and just copying the vhdx file? Since all new writes will go to the avhdx file, it should be safe to do a live copy of the vhdx file? Once copying is done, delete the checkpoint.
Have you tried it? It seems like a solid enough idea, assuming that you don’t have a great deal of disk churn and/or have enough space. I’d be curious if it will consider it to be an open file and block the copy.
What is your opinion on creating shadow copies of VHDs, simply as an added layer of backup protection? I understand the implications of using up hard drive space, but having that quick restore option in the right circumstances seems to outweigh that negative. I’m mostly interested in what you said about “Do not leave shadow copies lying around. It’s bad for your system’s health.” Could you elaborate? Thank you.
All respectable backup applications leverage shadow copies to back up Hyper-V VMs. I don’t have anything against rolling your own solution, but understand that others have already got the wheel fairly round.
Shadow copies use up storage space that is somewhat invisible. It lives in the “System Volume Information” folder where it might be missed. If you don’t keep a close eye on your shadow copies, they’ll chew up your disk space.
Just asking this question because I can’t find an answer anywhere right now. I use VHD files attached over a network to VM’s to run Windows Server Backups that maintain versioning. I want to schedule these attached VHD files to be copied to removable drives as part of the backup routine. I am using Bitlocker to encrypt the attached VHD’s. I am just wondering if it is safe to leave the VHD attached to the VM, while my NAS performs a file copy backup to an external USB drive. The file copy will never occur during the actual Windows Backup to the VHD. Is this safe to do? Or can some type of corruption occur on the copied VHD?
I don’t think that I have a perfectly clear view of what you’re doing, so I’ll speak generically. This situation throws up some yellow flags for me. If the backup occurs elsewhere and the host is not aware of it, then it can write to the VHD while it is being copied. That would leave it in an inconsistent state. If whatever the NAS does treats the file like static, flat data, then it quite possibly will corrupt its copy. The original is probably fine, though.
I have a Windows Server 2012 HyperV host, it will perform a Windows Backup at 10pm to a VHD file (resides on NAS, attached to same Windows Server HyperV). After that Windows Backup completes at say 12:00am, I will schedule that same NAS to copy this same VHD file to a USB drive connected to the NAS at say 4:00am.
During this process, I never “detach” the VHD from my Windows Server VM. It remains attached.
This will occur nightly. The NAS will delete any existing copies of the VHD on the USB drive before copying the new/updated VHD to the USB drive.
My concern is that not “detaching” the VHD prior to copying the VHD to the NAS, could cause an issue. However the VHD shouldn’t have ANY changes or data written to it after the Windows Backup occurs/completes at 12:00am.
Hopefully this explains it a little better. I may have to just let this run, monitor it, and then see if I can do a test restore of the data to be completely sure.
If the file on the NAS is a COLD backup copy made by Windows Server Backup running on the Hyper-V host, then everything will be fine and you don’t need to read any further.
If it is not cold, meaning that the Hyper-V host has mounted the VHD while the NAS is backing it up and the host has no way to know that the file is being backed up, then you have no guarantees around data integrity of any kind. You cannot guarantee that Windows won’t write to a previously-backed up portion of the file. It might be basic house-cleaning metadata, or it might decide that it can rearrange part of the MFT as an optimization. A simple restore will not positively tell you if the backup worked, only that it worked well enough to not throw the restore off. You would need to perform a full end-to-end integrity check within the guest. Even if that works on your test, you will still roll the dice on each and every subsequent backup pass.
Eric, thank you for your posts on Hyper-V! Using Hyper-V on an old 2008 R2 server. It has 4 Hyper-V virtual servers on the physical server. Nightly after everyone is off the servers a drive Imaging software called Terabytes Unlimited similar to Acronis or Norton Ghost Images the entire physical server which contains every single file on the server including all the VHD files. For restoration the files would be pulled out of the Image backup of the server. In this case are the Virtual drives getting backed up so that they can be restored to a new server should this old server die?
It sounds like that’s what it’s doing. The VHDs are the virtual disks. If it’s properly using VSS to capture the entire system at once, everything should work out OK. I don’t know anything about this program, so I don’t want to make any promises.
Hi Eric
What happens if a disk’s contents change during an Export-VM task? Does the export wrap up those changes at the end or instead are they abandoned until the next export?
Thanks,
Jeff Bowman
Fairbanks, Alaska
From your choices, “abandoned” is the closest.
Export takes a backup-style snapshot at the beginning of the process and exports that. It never sees anything that changes while the export is in progress.