We have a large migration due to occur of several 1000 mailboxes and do not want to migrate this over a long time.  Can guidance be provided with regards to how many hosts, if that’s even possible, and how much CPU and memory these host should have to get the fastest performance?

The Priasoft migrator is a high performance product.  As such, it is important to consider the host elements like CPU and RAM as they certainly do have an influence on overall performance, which in turn determines the overall duration of a migration event.
When a migration batch is executing, it is able to migrate multiple mailboxes at the same time.  This concurrency is adjustable, even during a migration, and with proper testing and resource allocation it is possible to migrate dozens of mailboxes simultaneously.  We have seen customers migrate as much as 70 mailboxes at a time on a single migration host.
Our recommendation is to use as much CPU and RAM as can be afforded.  Higher numbers mean higher concurrency on a single host.
There are many influencers to this however.  For example, if the source Exchange server is slow, having a 32-core host will not serve well since the bottleneck will be the source exchange server and will limit the concurrency.
As a starting recommendation, we suggest this approach:

• Create a single host with 8 CPUs and 16gb of RAM.
• Use the Priasoft dry-run feature to test whether you are able to exceed those resources.
  • Create a batch with a sufficient number of mailboxes for this test – 50 to 100 mailboxes should work.
  • This test is not concerned with seeing mailboxes complete, but is to determine how high the concurrency can be set before seeing a bottleneck.
  • This test should gradually increase the concurrency every 5 minutes, or so, by 2-4 mailboxes.
  • Between each increase a delay should occur to see that an increase in performance is realized – monitor the messages-per-second metric provided in the batch console.
  • If the Exchange servers – on both the source and target side, Office 365 included – are able to keep up with the increases, it may be found that the migration batch starts to report that “resources too high”, which means that CPU and/or RAM have been exhausted on the local host.
• If testing shows that the migration host has too few resources, this test justifies either an increase in the host resources, or the creation of an additional host.

Please be sure to read our document on dry-runs (http://download.priasoft.com/docs/drm.pdf) as this document provide more detail on the above concepts.
Also, there is little difference between one host with 16 CPUs and two hosts with 8 CPUs.  It is often simpler to have fewer migration hosts with greater resources.

It is also prudent – especially for larger migrations – to have at least one extra migration host on standby.  Things can happen during a migration.  If the primary host fails in some way, having a second host can save time as execution can be shifted to this host instead of either delaying the migration or postponing it altogether.
Another note on multiple migration hosts:  be cautious of over-performing.  If a specific source Exchange server is proven to handle a migration with a concurrency value of 30, a second host also running at 30 will mean 60 connections to the same server, which can easily create a bottleneck and slow the entire migration.
It is possible to have many migration hosts running at the same time.  We have seen customers run 8 and 10 migration hosts concurrently, each running many mailboxes such that the overall concurrency is over 100 mailboxes at a time.  However, such was achieved thru the above methods and also by organizing the migration batches so that each migration host was sourcing and targeting specific exchange servers.
For example, if a source environment has 3 mailbox servers, each with active database copies (important!), the batches would be organized so that each migration host (3 of them) only have mailboxes that are on a single server at a time.  Host-1 has a batch of mailboxes that are hosted on Exch-01, Host-2 has mailboxes from Exch-02, etc.
Understand that maximum performance comes from an even distribution of workload across all all the physical resources at play:  Exchange servers (source and target), migration hosts, storage systems, network, etc.  Overloading any one element unexpectedly can create intermittent slow-downs that become difficult to isolate the issue, when the real issue is simply a lack of control over the distribution of work.

Regarding the question about physical hosts versus virtual: technically speaking, a physical host for migration will outperform a virtual one, but if the virtual host is configured for this purpose the difference will be minimal.
Therefore, it is important to setup a VM specific for the purpose of migration, which will be different that “standard builds” of other virtual hosts in use.  Consider that the move to virtualization was an effort to increase the density of server “usage”.  Placing several hosts together on one physical host means that nearly 100% of the physical resources are utilized and provides financial savings by not wasting physical resources on hosts when such a host would likely never consume even 50% of those physical resources.
This paradigm leads to designs that support density, not performance.  This density is achieved, in part, because most servers have a nominal “run rate” with regards to resources consumed on a hourly or daily basis.  If a Domain Controller, as an example, is shown to only use 20% of the resources on a physical host, day after day, then 4 to 5 DCs could run virtualized on one host, instead of 5.
The virtualization platforms then added, over the years, features to further increase density with concepts like pooled memory, and dynamically allocated CPU and resources.  Such features help deal with intermittent spikes in resource demand.
However, a migration host and its pattern of use is not the same as most servers that are virtualized.  The resource consumption pattern of a migration host is typically one where, for most of its life is idle, with spikes of many hours to a couple of days of nearly 100% utilization, then followed again by a long period of idleness.  The virtualization platforms don’t expect this behavior and so do not have heuristic analysis to accommodate this pattern.  Some platforms and configuration will even begin to throttle the resource demand as an expectation that the host is rogue or malicious in some way.
From many years of experience on this topic, we recommend the following with regards to virtual hosts for migrations, regardless of tool vendor:

1. Do not use pre-made templates.
2. Do not use company made templates either.
3. Design the virtual host to be a near to a physical host as possible by “pinning” virtual resources to physical ones.
4. Avoid the use of pooled resources and dynamically allocated resources.
5. Don’t run the virtual host on a physical host that is already heavily committed for other loads.
6. Don’t run a migration on a virtual host at the same time that a backup of the host is occurring (think of Veeam)
7. Don’t v-Motion a host to another physical server while migrating.  MAPI is very sensitive to network drops.  Such WILL cause mailboxes to fail and be restarted.
8. Don’t mis-configure the CPUs – this one is very important.
   1. If a physical host has 4 CPUs each with 4 cores, such equates to 16 available cores for a VM.
   2. However, if a VM is created with “1 socket, 8 cores”, that VM will have 4 of the cores virtualized as there is no physical CPU with 8 cores.
   3. Do NOT EXCEED the physical layer in the configuration of a VM.
9. Avoid using a dynamically expanding disk for the migration host.
   1. The migration does NOT use temporary files for items copied, but still does use the disk quite frequently.
   2. Consider that there is substantial logging, and state tracking data this uses the disk.
   3. If the disk is slow, the migration will be slow because the migrator will constantly be waiting for disk actions to complete.
   4. The migration does not consume much space, but is very busy with creates, deletes, and updates to very small files.  The disk transaction count is high while the disk data utilization is low.

Lastly, for customers that are migrating withing the same physical environment, possibly just a simple upgrade or a consolidation, avoid reading and writing to the same storage system for migrations.  Consider a situation where a company will move from Exchange 2010 to 2016.  There is sufficient free space on the current storage platform, likely a SAN, for the new data.  Initially it is considered to add the Exchange 2016 databases to the same SAN as the Exchange 2010 data.
This setup is not ideal because the SAN will be doing twice as much work as expected.  Each read from the source creates a write to the target, but on the same set of disks!  For traditional “spinning” disks, this means the read/write head must work twice as hard to hand the influx of reads and writes for the same disk.
If possible, attempt to have the new Exchange servers operate on exclusively different disks than the source.  However, modern SANs seem to prevent this capability, so you may have to check with your storage vendor on this.