How to Use Storage Migration Service for Windows Server and Azure

Storage migration projects are often one of the most daunting tasks that IT administrators face. These projects are risky due to potential data loss or misconfiguring identity permissions. Migrations are unrewarding, as end-users rarely notice a difference, but perhaps more challenging is that many migration tools are of substandard quality with a limited support matrix. In the past, migration was a lower priority initiative by Microsoft, but things are changing with the Storage Migration Service (SMS).

About a decade ago, I worked for Windows Server as an engineer and designed several of their migration technologies, even earning a patent. However, these projects were often rushed, had features cut and used a limited test matrix. This surprised a lot of people since migration should be viewed as an important tool for bringing users to the latest versions. But from the business perspective, a Windows Server to Windows Server migration was generally a one-time operation by already-paying customers. It made more sense to invest engineering resources in building new features and bringing new customers to the platform. If an existing customer had a subpar migration experience, although not ideal, it was acceptable. Migrating customers from a different platform (like VMware of AWS) is different and has always been treated as a high priority by the company as it generates new revenue.

However, Microsoft has recently (Windows Server 1809) released a first-class Windows Server to Windows Server (or to Azure VMs or Azure Stack) migration solution. Storage Migration Service (SMS) provides a new storage migration technology which is managed using Windows Admin Center (WAC) or Remove Server Administrator Tools (RSAT). This GUI-based utility is straightforward and allows file servers (including their data, shares, permissions and associated metadata) to be migrated from older versions of Windows Server to Windows Server 2019 servers and clusters. SMS supports most Windows Server and various Linux distribution running Samba. After the migration, the identity of the file server can also be migrated so that users and applications do not lose access.

This article will review the scenarios, features, requirements and best practices to use Storage Migration Services. This article includes the latest updates included in release 1903 (May 2019) and 1909 (November 2019).

How Storage Migration Service Works

Storage Migration Service is fairly straightforward and follows other migration processes. First, you need to open a few firewall ports as described in the Security Requirements section of this article. Next, you will install Storage Migration Service in Windows Admin Center and open this tool.

To start the migration, you will select your source servers which will be inventoried and display a list of volumes and folders. You will select the storage that you wish to copy and specify some other settings. Next, you can pick the destination servers and volumes, and map them to the source volumes.

You are also given the option of migrating the entire file server, including its identity of the server and its networks, or just the shares and their data. It is possible to use this technology as a basic asynchronous file replication solution. The data will then be copied from the source volumes to destination volumes using the SMB protocol. The copy operations may run directly between the pair of servers or may be routed through an intermediary Orchestrator server which manages the migration operation.

If the identity of the file server is also migrated, users will be able to connect to their storage once Active Directory and DNS records throughout the infrastructure are updated. There may be slight disruptions in service, but all of their files and settings should be retained. The old servers will enter a maintenance state and not be available to users, and they can be repurposed.

Figure 1 – Storage Migration Service Overview

Note that a step-by-step guide for using the Storage Migration Service can be found in Microsoft’s official files.

Planning for Storage Migration Service

This section provides an overview of different considerations based on the hardware and software requirements for the various migration servers. Processing the migration can be resource-intensive, so it is recommended that the Orchestrator and any destination server have at least 2 GB of memory, 2 (v)CPUs and 2 cores. Conventional infrastructure enhancements will also speed up the process, such as providing a dedicated high-bandwidth network and using fast storage disks.

To make the migration faster, you can use hardware which has been optimized for SMB traffic. This can include using multiple NICs which support Remote Direct Memory Access (RDMA), SMB3 multichannel, Receive Side Scaling (RSS) and NIC teaming. On the server, you can try to maximize the memory and QPI speed, disable C-State, and enable NUMA.

Windows Server as a Source Server

The source server hosting the original storage must use one of the following versions of Windows Server:

• Windows Server, Semi-Annual Channel
• Windows Server 2019, 2016, 2012 / R2, 2008 / R2, 2003 / R2
• Windows Small Business Server 2011, 2008, 2003 R2
• Windows Server 2012 / R2, 2016, 2019 Essentials
• Windows Storage Server 2016, 2012 / R2, 2008 / R2

Migration from Failover Clusters running Windows Server 2012 / R2, Windows Server 2016 and Windows Server 2019 is also supported.

Linux Servers using Samba as a Source Server

Storage Migration Service makes it easy to migrate from legacy Linux server using Samba. Samba is a suite of programs for Linux and UNIX which provides file server interoperability with Windows Server. It allows file shares to be managed like they are running on Windows by providing compatibility with the SMB/CIFS protocol. It supports Active Directory, but when migrating from a Linux server you will enter additional Linux and Samba credentials, including a private key or SSH password.

Samba 4.8, 4.7, 4.3, 4.2, and 3.6 is supported on the following Linux distributions:

• CentOS 7
• Debian GNU/Linux 8
• RedHat Enterprise Linux 7.6
• SUSE Linux Enterprise Server (SLES) 11 SP4
• Ubuntu 16.04 LTS, 12.04.5 LTS

Windows Server as a Destination Server

It is generally recommended to migrate to the latest version of Windows Server (currently WS19), as this operating system will be supported for longer and has performance optimizations for SMB file transfers. With SMS, using Windows Server 2019 as the destination server actually runs about twice as fast as older versions of Windows Server as it can function as both the Orchestrator Server and destination. This is because data can be directly transferred to the destination, rather than routing through another intermediary Orchestrator server. However, Windows Server 2016 and Windows Server 2012 R2 are also supported.

Failover Clusters

Windows Server Failover Clusters are supported as host and destination servers, provided that they are running Windows Server 2012 / R2, Windows Server 2016 or Windows Server 2019. It is possible to migrate storage between two clusters, from a standalone server/VM to a cluster, or from cluster to a standalone server/VM. Failover cluster are also supported for consolidating multiple standalone hosts onto a single cluster by having each migrated file server become a clustered file server workload.

Microsoft Azure Stack

Microsoft Azure Stack can be used as a destination server, with the storage being migrated to VMs running on Azure Stack. Azure Stack is deployed as a failover cluster, so it can also be used for consolidating multiple standalone hosts onto a single piece of hardware.

Microsoft Azure

Storage Migration Services can migrate storage, identity and network settings to a file server running inside a Microsoft Azure Virtual Machine (VM). Simply deploy your Azure Active Directory-connected file server and access it like you would any on-premises file server.

Azure File Sync Integration

Azure File Sync is a technology which optimizes how an on-premises file server syncs its data with Microsoft Azure. It allows Windows Server to function as a local cache of the Azure file share. It integrates with Storage Migration Server and can optimize performance after the migration.

Active Directory Considerations

Storage Migration Service requires that both the source and destination server are within the same Active Directory domain. All of the source servers, destination servers and any Orchestrator Server must have a migration account with administrative access to all systems. If you use migration credentials, the domains must be within the same AD Forest. Any Linux servers running Samba are also required to be managed within the same domain.

When using Windows Server Essentials or Windows Small Business Server you likely have your domain controller (DC) on the source server. For this reason, you likely will not be able to migrate the identity settings as the DC must remain online throughout the process. You can still inventory and transfer files from these servers. If you have two or more domain controllers this should not be an issue, and you can promote the domain controller on the source server after the cutover.

Workgroup migration is not supported.

Installing Storage Migration Service

The Storage Migrations Service feature will appear in your Windows Admin Center feed. SMS can also be installed using PowerShell. Installing the Storage Migration Service feature on the management server will make it function as the Orchestrator Server. Install Storage Migration Service Proxy on your destination host(s) to maximize performance as this enables them to directly copy data from the source servers. You can optionally install the Storage Migration Service Tools if you are using an independent management server.

Figure 2 – Installing the Storage Migration Service Features

Storage Migration Service (Orchestrator Server)

This feature is installed on the primary server running the migration, known as the Orchestrator Server. This server manages the migration process. It can run on any server or VM that is part of the same domain. The Orchestrator Server can run directly on the Windows Server 2019 destination server or an independent server. It is a good practice to always copy the migration events and logs from this server to track the migration progress.

Figure 3 – Installing Storage Migration Service through Windows Admin Center

Storage Migration Service Proxy

The Storage Migration Service Proxy is a role installed by Server Manager. This can be installed on the Windows Server 2019 destination server in order to double the transfer speed as this allows the source and destination server to copy data directly between each other. Without the proxy, the files need to be first copied to the Orchestrator server then they are copied again to the destination server. This takes twice as long as the Orchestrator server acts as a bottleneck. Installing SMS on any Windows Server 2019 host will automatically open the necessary firewall ports.

Storage Migration Service Tools

These are the management tools which can be installed in Windows Admin Center or Remote Server Administration Tools (RSAT).

Configuring Firewall Settings

When installing the Storage Migration Service Proxy, the proper firewall settings will be configured. The source and destination servers must have the following firewall rules enabled for inbound traffic:

• File and Printer Sharing (SMB-In)
• NetLogon Service (NP-In)
• Windows Management Instrumentation (DCOM-In)
• Windows Management Instrumentation (WMI-In)

The Orchestrator Server must have the inbound File and Printer Sharing (SMB-In) firewall rule enabled.

Inventory Storage Volumes

One of the first steps performed by Storage Migration Services is to inventory the storage which is selected for migration. This will list details about each of the components which will be copied, including the volumes, shares, configuration settings and network adapters. This information will also be retained in the migration reports.

Figure 4 – Storage Migration Service will Scan a Server to Inventory its Volumes

Map Source and Destination Servers and Volumes

During the migration, you will get to match each volume on the source server with a volume on the destination server. After selecting your source server(s), SMS will scan them and present a list of volumes. You can select any or all of the drives you wish to migrate, and you will map each to a volume on the destination server which has enough capacity. You must also migrate between the same file system type (NTFS to NTFS or ReFS to ReFS).

Figure 5 – Mapping Source and Destinations Servers using Storage Migration Service

Consolidate File Servers on a Failover Cluster

Many administrators want to use Storage Migration Service as a consolidation tool, allowing them to merge several older file servers onto a single destination file server. This scenario is only supported by migrating each legacy file server to a clustered file server. This is permitted because a failover cluster can run multiple file servers as a native cluster workload or as virtualized file servers inside VMs.

Migration Using Storage Migration Service

This section provides details about what happens during the migration.

Validate Migration Settings

Once the source and destination servers are mapped, click Validate. This will run several tests to verify that a unique destination exists, its proxy is registered, the SMB connection is healthy, and that the credentials work with administrative privileges.

Figure 6 – Validate Migration Settings

Migrate Data

Once the transfer begins, data will be copied from each volume on the source server to its mapped volume on the destination server. If there is any data already in a share on the destination server, then this existing content will be backed up as a safety measure before the first migration. This initial backup only happens the first time and not on subsequent backups. If the storage migration is repeated, any identical folders and files will not be copied to avoid duplication.

Migrate Storage Settings

The following settings (if available) are migrated to the destination server.

• Availability Type
• CA Timeout
• Caching Mode
• Concurrent User Limit
• Continuously Available
• Description
• Encrypt Data
• Folder Enumeration Mode *(aka Access-Based Enumeration or ABE)*
• Identity Remoting
• Infrastructure
• Leasing Mode
• Name
• Path
• Scoped
• Scope Name
• Security Descriptor
• Shadow Copy
• Share State
• Share Type
• SMB Instance
• Special
• Temporary

One important component which is not copied during migration is Previous Versions made with the Volume Shadow Copy Service (VSS). Only the current version of the file will be migrated.

Migrate Local Users and Groups

During the migration, you are given the option to copy the account settings for local users and groups. This allows current users to be able to reconnect to the file server without any additional configuration, which would be ideal if the server identity is also migrated. If you decide to migrate the local users and groups, you will be given the option to keep these accounts the same or force them to be reset with a more secure password. You would not select this option if you plan on keeping your existing file servers in production as there would be duplicate and conflicting file servers in your infrastructure.

If you are running the migration to set up or seed a DFS Replication server, you must skip migrating the local users and groups.

Skip Critical Files and Folders

Since the migration process happens on a running operating system, it is important that any critical files or folder which are in use are protected. Storage Migration Service will skip these files and folders and add a warning to the log.

The following files and folders will automatically be skipped:

• Windows files, including: Windows, Program Files, Program Files (x86), Program Data, Users 
• System files, including: pagefile.sys, hiberfil.sys, swapfile.sys, winpepge.sys, config.sys, bootsect.bak, bootmgr, bootnxt 
• Computer-specific files, including: $Recycle.bin, Recycler, Recycled, System Volume Information, $UpgDrv$, $SysReset, $Windows.~BT, $Windows.~LS, Windows.old, boot, Recovery, Documents and Settings 
• Any conflicting files or folders on the source server that conflicts with reserved folders on the destination server.  

Multi-Threaded Migration

SMS allows for multiple copy jobs to run simultaneously as it uses a multi-threaded engine. By default, SMS will copy 8 files at a time within a job. This can be changed from 1 file to 128 simultaneous files by editing the FileTransferThreadCount registry setting for HKEY_Local_MachineSoftwareMicrosoftSMSProxy. It is best to not set this higher unless you have hardware-enhanced for SMB as it increases processing overhead, and network bandwidth or disk speed are usually the limiting factors. 

View Post-Migration Information

It is a good best practice to keep track of your SMS migration errors, transfers and jobs. There are a few different ways which you can track this information with Storage Migration Services.

Error Log

Any files or folders which cannot be transferred will be noted as warnings in the Error Log, such as those being used by the running operating system. This error log will also describe any other types of warnings and errors.

Transfer Log

To keep track of all of the migrations download the Transfer Log as a CSV (spreadsheet) file. Every time you run the migration this information will be overwritten. You may want to create an automated task in Task Scheduler which copies this file every time the Storage Migration Service has completed so this information is always captured.

Jobs Log

There is a log which tracks all of the SMS jobs, however, this is generally not needed by the admin so it is hidden. You can find it under C:ProgramDataMicrosoftStorageMigrationService. If you are migrating a large number of files, you may want to delete this database to reduce the size it takes up on disk. Additional information can be found in Microsoft’s official files.

Migrate & Cut Over the Identity of the File Servers

Once the data has been copied to your destination server you have the option to migrate the file server itself. This allows users to continue to access their files on the new hardware with minimal disruption. After completing the migration, select the Cut Over to the New Servers option and enter your Active Directory credentials. You can rename the server, but most likely you will keep the same file server name. If you do not copy the identity, the users will keep access files on the source server.

Migrate Network Adapter & IP Address Identity

When you migrate the File Server identity you will be given the option to map each of the network adapters from the source server to network adapters on the destination server. This will allow you to move the IP address during the cutover, whether it uses a static IP address or DHCP address. If you use a static IP address, make sure that the subnets on the source and destination server are also identical. You can also skip the network migration.

Figure 7 – Configuring the Network Migration for a Cutover

Migrate & Cut Over a Failover Cluster

If you are migrating to a failover cluster, you may also need to provide credentials which allow you to remove a cluster from the domain and rename it. This is required any time a cluster node is renamed.

Antivirus Considerations

Make sure that the antivirus versions and settings are the same on the source and destination server, particularly for scanning any included and excluded folder. You may need to temporarily disable antivirus scans during the migration to ensure that any files are not temporarily locked while being scanned.

Summary

Storage migration projects can be overwhelming. But if you plan on using Storing Migration Services for Windows Server and Azure, I hope the scenarios, features, requirements and best practices described here, will prove useful. As always, if you have any questions or concerns, let me know in the comments below.