Exchange Migration Knowledge BaseCategory: General QuestionsCan Priasoft pre-load mailbox data?
Anonymous asked 11 years ago

Is it possible to do a pre-stage where mailbox data is copied to the new environment, but the new mailbox is not converted to a linked mailbox and the old mailbox is still in place and active. Then, in the final stage, we would only have to copy a small amount of data and convert the accounts which would mean we could migrate much more accounts in the same time.

1 Answers
Eriq VanBibber Staff answered 8 years ago

Priasoft does not support pre-loading of mailbox data.
The following is from our FAQ (https://www.priasoft.com/faqs/#Q1)
A: Priasoft does not currently ‘trickle’ or ‘pre-load’ mail data.
This ability is actually EXTREMELY misleading and should not be casually accepted. Please consider the following:

  • All other factors being equal, if it takes 1 hour to migrate 72,000 items and there are 7.2 million items to be migrated, it will take 100 hours to migrate all of the data. Trickling the data does not make a migration perform faster. In most cases it will actually run slower because the work is being done during idle time and has more work to do than just a direct copy of data. Consider the following concepts that are required for any kind of synchronization of data:
    • Source items that exist in the target (previously copied) must be identified
      • Comparison of key information must be analyzed to determine if the source has changed and requires a re-copy
      • Additional logic has to be employed if both the source AND target items have changed, which add additional time to the process.
      • This means that data from both the source and target exchange server must be pulled into memory for analysis, potentially over a WAN connection.
      • If, after analysis, it is determined that no changes were made, time and bandwidth were wasted in the sense that data was pulled for no additional value.
    • Source items that don’t exist in the target have to be discovered
      Once discovered, logic has to be applied to determine if the source item should be recopied, or if the source item should be deleted (to reflect a delete that occurred in the target mailbox)
  • This operation is also time consuming and requires either a search of the target mailbox in order to NOT FIND the item, or requires the use of an external database (like SQL).
  • If an external database is used, this becomes another point of failure in the system and also adds to the infrastructure requirements that MUST be maintained for the duration of the migration project. Additionally, the size of this database begins to grow quite large with historical information due to changes in the environment during the project lifetime and therefore adds even more management burden in the sense of protection and backup of this information.
  • The same 2 above operations then have to then be performed in reverse: target -> source
    • The contents of the target mailbox have to be compared to the contents of the source
    • This has to be done to adhere to the concept of synchronization, which means that changes on one side are reflected on the other.
  • Consider this complexity multiplied by the number of items in a mailbox multiplied by the number of mailboxes for the project. Such should highlight the complexity, dependence, and load that are placed on the ‘trickle’ logic and the server that hosts this logic. Then imagine if this server was lost in some way or was caused to be offline for any period of time.
  • As more and more mailboxes participate in the ‘trickle’ service (a separate service that keeps the mailboxes up to date, possibly on separate hardware) the load on the service increases, which in turn slows down the entire process.
  • Another side effect relates to an increasing quantity of mailboxes requiring synchronization; the potential is for the ‘trickle’ service to become deadlocked or even for it to crash. Priasoft has had
    direct reports from customers of a ‘trickle’ service having to be ‘rebooted’ EVERY day in order to handle the increased load of work to perform.
  • Visibility of ‘when-to-cutover’ is difficult. Products that ‘trickle’ data do not notify you of the optimal time to begin cutting over mailboxes to the new system, and in most cases, it is a guess.
  • Cutting over a mailbox too early puts a burden on the end user. In once scenario, if a mailbox is switched over to the new account too soon, the end user will be missing mail that has yet to be replicated to the target mailbox. The end user will have to wait for that mail to arrive if the user needs that data and the delay can be minutes, hours, or even days and due to lack of reporting, it will be difficult to tell the user how long it will be before they have the data they need. Furthermore, often this ‘last bit of data’ that is missing is usually the most current mail data, not old data. This means that if only 50% of a 2GB mailbox was ‘trickled’ forward and the user is cutover, they will be looking at the oldest 1GB of data, and will have to wait for the other 1GB of data in order to effectively use the mailbox.
  • ‘Trickling’ data also has an unseen side effect with regards to the source Exchange Servers:
    • You cannot effectively remove a source mailbox until the user is cutover to the new mailbox
      AND all the data from the source has been placed in the new mailbox.
    • The knowledge of when you can safely remove the source mailbox is obscure since data could be queued up for deliver to the source mailbox during the cutover.
    • The source Exchange servers will continue to have storage growth since the nearly all of the source mailboxes will still be in use during the migration, even though 90% or more of the data was duplicated into the target mailbox. Log files will continue to grow, storage management is still a high priority, backups are still a high priority.
    • In addition to having to manage all of the source environment, the target environment now has to be managed in much the same way, even when there are possibly little to no users in the target environment.
    • Current Exchange administrators will not receive a release of responsibility for any portion of the source environment until EVERY MAILBOX IS CUTOVER AND ALL SOURCE DATA HAS BEEN VERIFIBLY REPLICATED
  • There is additional infrastructure and management required. In order to synchronize data, regardless of the systems involved, a separate service has to be employed. In most cases, this separate service demands new hardware, licenses, management, and human resources to manage it. As the size of an environment increases, so does the likelihood for having to increase the number of ‘sync’ servers deployed. In addition, often some sort of tracking is required in order for the system to maintain state of objects. This tracking is typically in the form of a database (or multiple databases), which means more infrastructure, licenses, management, and human resources.
  • In addition to the infrastructure ‘build up’, there is often a client deployment that must be implemented. Often a ‘trickle’ will place some of the final burden of the cutover on the client computers, meaning that after cutover, each user that is cutover will have an agent or some sort of outlook add-in that will handle the final pieces of the migration. Imagine several thousand end user desktops demanding the Exchange system’s time and resources for even and handful of ‘last minute’ updates. This could mean tens of thousands of requests for changes at the same time.
  • The complexity of setting up and managing a ‘trickle’ style of migration increases costs like: time, training, licenses, and management of the complex system. These costs are outside of any licensing costs of a product, and can easily cause a budget to overrun, especially if the human resources that are expected to manage the system require advanced training. Imagine paying for training for a product in which after its use will most likely not be used again for many years, and even at such time will probably require more training!
  • In addition to the costs of such complexity is the effort required to remove the system once the migration is completed. Proper protocols have to be taken when removing databases and servers/services from a domain. Proper and verified communication with peers and management have to be taken to make sure that only the migration related items are taken offline and that in doing so, nothing else is affected.

Note that the concerns listed above are often never exposed in a proof-of-concept since the testing protocol is to define whether a product ‘works or does not work’. Be assured that in testing, the work performed is almost always setup such that it is ‘expected’ to work. No vendor, including Priasoft would provide a proof-of-concept that failed. However, there is great difference in a solution ‘technically’ working, and the same being satisfactory in a production environment. It is our experience that the concerns listed above often do not appear until the solution is accepted and is applied to a production environment, where the scale and load come to bear.
Due to Priasoft’s simplistic deployment, ease of use, and safety-by-design, you can perform TRUE proof-of- concept in a test environment through the use of our Dry-Run mode, but also just as quickly and safely in a production environment. In fact, Priasoft always recommends performing a proof-of-concept in the production environment so that the ALL of the factors that affect a migration can be considered. In addition to a POC, our Dry-Run mode allows you to validate the fidelity, performance, duration, and feasibility of the migration before committing to the production execution.
Priasoft’s migration strategy migrates some (perhaps the most recent 30 days), or all of the mailbox data at the time of migration, after which the user is ‘cutover’ and begins using the new mailbox. The migration is done completely ‘on-the-wire’ – meaning no “double copy” of data (once from source to temp file, then again from temp file to target mailbox) – and is the fastest way to move data.
Priasoft’s strategy also means complete understanding of progress and no delays by the end user. Priasoft provides transparency to end-users through the use of mail forwarding, when required. As users are migrated, mail that would have been delivered to their old mailbox is instead forwarded to the target mailbox.
There is an unseen advantage here in that you can choose to delete the source mailbox very soon after the mailbox has been migrated (even immediately) which will create valuable ‘white-space’ in the source server to help control its storage requirements during the migration.
Even if you choose not to delete the source mailbox, the source server’s growth rate begins to diminish immediately with the first mailbox migrated because of the forwarding of mail and the fact that migrated users are no longer generating activity. In addition, because mailboxes are finitely migrated, once all mailboxes have been migrated from a specific Exchange server, work can be started to remove that Exchange server from the environment at the same time as migrations are occurring on other Exchange servers.
As more and more Exchange servers are brought down, several options and benefits become available:

  • Hardware reuse: After an Exchange server is removed, the hardware can immediately be reallocated for other uses, perhaps as a new Exchange server in the target environment.
  • Replication reduction: After an Exchange server is removed, there is less replication of data in the network. Multiple Exchange servers, especially prior to Exchange 2007, replicate Public Folder data
    between servers. Each Exchange server in an environment also reads and writes data to Global Catalog servers very often in order to read and keep the Address Book up to date.
  • Network utilization reduction: Every Exchange server removed reduces large amounts of network utilization, which on large, already congested networks means that data will flow more freely as the environment is reduced.