Exchange Migration Knowledge BaseCategory: Mailbox Migration QuestionsSome users after migration complain they are missing items. How to troubleshoot this?
Anonymous asked 11 years ago

After a migration of mailboxes, some users say they feel they are missing some items.  Reviewing the logs does not really seem to help and there is a lot of information there.
How would I analyze this issue to determine if items are actually missing, or not?  Why would items be missing at all…is that even possible?

1 Answers
Eriq VanBibber Staff answered 8 years ago

While not impossible, it is rather rare to have missing items after a migration unless the missing items are corrupted – but that would mean the user likely could not access the item either, if it is truly corrupted.
One of the first things to analyze is if any item failures occurred during the migration.
The following is an example from a log file that shows item failures occurred for the folder:

 2015-09-16T12:23:30InformationCurrent Folder is: Staffing Issues::Staffing Issues – Total size: 1481425 Total Messages: 2
 2015-09-16T12:23:31 Information Attempting to retrieve item copy history table for folder: Staffing Issues
 2015-09-16T12:23:31 Information Copy history table retrieved and contains 0 items
 2015-09-16T12:23:34 Information Processing of folder ‘Staffing Issues’ completed. Copied 19 items; Skipped 0 items; Failed 4 items.

The last line in the table above (an highlighted in red for ease) shows that 4 items in the folder failed to copy.
However, understand that there is a difference between “Failed” and “Skipped”.  Skipped means that the migrator saw the item before, in a previous pass on the folder, and skipped the item so as not to duplicate the item.  “Failed” means that some issue occurred during copy – no retry of the item was made.  Please refer to this Q&A post for more detail about skipped items: Why do i see “skipped items” for some mailboxes?
Assuming then that the case that presented the question is not about failed items, what other situations could cause a user to feel that items are missing?  To answer that, some detail about how the migrator processes a mailbox may be helpful.
Single Threaded Enumeration
The migration of data in a mailbox is done by enumerating the folder in the mailbox, one by one.  There is no multi-threaded operation of the migration of data.  Starting with the top of the mailbox (the parent folder of the Inbox), a list of subfolders is requested from the Exchange server.  Once this list of folders is retrieved, the list – an in-memory list – is worked based on the order given to us by Exchange, which is in alphabetical order.
A subtlety that should be seen here is that once the request is made to the Exchange server for the list of subfolders, any changes to that list would be unknown.  If a user, just after getting the folder list, created a new folder, the migrator would be unaware of that new folder.  The only way the new folder would be seen is if the folder list was re-queried.
A similar pattern exists with regards to items in a folder.  The migrator asks for the list of items from Exchange for the specific folder that is being worked.  This list is stored in memory and when all the item identifiers are received, the process to migrate each item in that list is performed.  The same subtle case then exists for items in that if a user, after we have already requested the contents of a folder, adds items to the folder, the migrator will be unaware of these new items.
A Narrow Case
While it may immediately seem that the above seems like a large issue, it must be understood that the case of occurrence is actually very narrow.  The possibility only exists for the duration that the mailbox is being migrated, ONLY if changes are made to folders that have already been processed, or to the currently active folder in the mailbox.
If a particular mailbox is shown to take 30 minutes to migrate, then there is only a 30 minute window in which this can occur.  Once the mailbox is completed, the migrator blocks MAPI and OWA access to the source mailbox.
The likelihood of occurrence is further narrowed when a proper migration strategy is used – one that executes migrations during periods where the users to be migrated are normally not active.  Most migration strategies involve scheduling of migrations during off-hours, nights or weekends, when it is well known that most users are away from the mail system.  Even if users have mobile devices, the risk is still minimized by time-of-day choices.  Users will not likely make structural or content changes to their mailbox via their mobile device, nor in the middle of the night (if they are a normal daytime worker).
The possibility is further narrowed again through pre-migration communication.  Some announcement of the scheduled event of migration should be distributed with simple guidelines to tell users that during the migration period – of which the duration should be concretely defined by the prior use of dry-run actions – to refrain from creating new items, importing data, or moving/copying folders around in their mailbox.  It’s safe to read mail, and if absolutely necessary, a reply can be made, but large changes should be avoided.
From experience, we find that communication like the above will cause most users to simply not use mail – users really do not want to be responsible for any damage for their own mailbox, even if it is only a perceived idea and not a reality.
Cutover + Backfill
While the above strategy does greatly limit the possibility, it does not completely eliminate it.  The 2-pass migration model – Cutover+Backfill – is the final eliminator of the probability, for a few reasons:

  1. A Cutover Pass of a few days or weeks of mail means that individual mailboxes are only “in flight” for a short period of time, often a few minutes.  This severely limits the probability of a user being able to interact with the mailbox in such a way that can create the issue described above.  Individual folders may only take mere seconds to complete, especially for user created folders as many are likely to not have any items within the specified date range of the Cutover Pass.
  2. Once the Cutover Pass completes, the user’s source mailbox is inaccessible and, using other features of the Priasoft migration suite like the Profile Updater Manager, users will begin to access and use the target mailbox.  This renders the source mailbox dormant and static.  All new changes will then occur in the target mailbox.
  3. The Backfill Pass analyzes all items in the source mailbox and merely skips over any previously migrated items.  The Backfill Pass will capture any user created items or folders that happened during the Cutover Pass.

Reporting and Troubleshooting
Given all the strategy and design presented above, it can still happen that a user or users “feel” that items are missing, but are unable to fully qualify or quantify what is missing.
There are several pieces of information that can be used to troubleshoot this.
The Migration Log File
The first place to look is in the migration log file.  As shown above, each folder that is migrated has a log event that shows what actions took place with regards to Copied, Skipped, and Failed actions.  Combine that detail with the first log entry that shows how many items the folder had and a story can be developed, but remember that the number of items reported is at the time we requested the list of items.  If a users added items afterwards, the number will not reflect that difference.
The Data Detail Log File
There also exist additional log files, per-mailbox, that provide a folder-by-folder accounting of contents.  The Data Detail Logs exist in a subfolder in the same location as the main mailbox log files.
These logs files are best opened in MS Excel.  There are many columns of information and opening in notepad will cause the column headers to not be aligned with the data.
The Data Detail logs provide detail, for each folder, of item counts BEFORE and AFTER the folder is processed, for both the source and target mailbox.
The data detail logs provides numbers that show:

  • How many items a source folder had just before we starting migrating data
  • How many items a source folder had just AFTER we finished the folder
  • How many items a target folder had just before migrating the folder
  • How many items a target folder had just AFTER we finished the folder

This level of detail can easily show what a user may have done during a migration.  The following is an example that the Data Detail Log can provide:

Inbox:Items in source beforeItems in source after Items in target before Items in target after
 100 120 0 100

If above is analyzed, it can see that when work was started on the folder, it had 100 items.  You can see that the target folder after the end had 100 items.  The result then is that the migrator copied 100 items, the 100 that were seen at the time the folder was started.  However, it can also be seen that when the 100 items completed, the source folder increased by 20 items.  That can ONLY mean that sometime after the list of source items were requested, 20 new items were added.

Hopefully this provides sufficient detail and strategy with regards to missing items.