Step 0 – Analysis, Remediation, and Policies

This step is numbered zero because it is imperative that it occur and complete before all other steps.  Failing to perform these tasks can create resets, progress blocking events, and frustrated end users and loss of productivity after the migration.

The reason and purpose of this step is to identify which, if any, of the source public folders exceed or nearly exceed the limits and restrictions of the target system. If such folders are identified, the common remediation is to split data into new child folders, or to remove data and folders that are no longer needed or required.   Dramatic changes to the folder hierarchy have a downstream influence on the rest of the steps and therefore should be completed before moving to step #1.

Folder Size Limits

According to the published limits for Microsoft Exchange modern Public Folders, an individual folder is not supported with contents that exceed 10GB (ref: Public Folder Limits).  This limit only applies to the contents of a single folder and is exclusive of any child folders.  Priasoft recommends that a review of all folders that are over or near supported limits be made and that a business policy be established to dictate the max size the organization will support.  It is also recommended that a number less than the limit be established for the identification of ‘large folders’ for this remediation step in order to avoid end user impact the day after a migration.  It can be quite frustrating to end users to have a folder that worked before migration suddenly not allow new data simply because the new target folder is at or over the size limit.

In addition to this, there is an importance to consider item counts on folders as well.  Outlook clients by default make direct connections to public folder data and such info is not cached (not in the cached-mode OST file).  Folders with 10s of 1000s of items will cause Outlook to lock up briefly or for many seconds whenever such a folder is clicked and selected.  The delay in responsiveness from the Outlook client increases with the item count and is compounded by latency (physical distance).  A folder with 100,000 items and an Outlook client with low latency can cause Outlook to lock up for 5 seconds or more.  If a user attempts to click or press escape or other action during this delay, Outlook will receive the “white screen of death” and a “not responding” behavior and a user may misinterpret this as a product issue, possibly raising an unnecessary support ticket.  Priasoft has noticed Outlook show the first sign of lock up at around 15,000 items.  Organizations should choose some number at or above 15,000 items as “too large for user productivity” as a part of a business policy regarding public folder usage.

Item Size Limits

Office 365 has a max item size for Exchange Online of 150MB.  This is for all data types in Exchange regardless of user mailbox, shared mailbox, or public folder.  This limit is not adjustable beyond this value and any items in the source that are near or over this size will fail to copy.  It is recommended that an additional analysis of folders be made to attempt to identify folders with very large items in order to take early action and to set new expectations.  Mail-enabled public folders further complicate this as such folders are limited based on the receive limits for the Office 365 tenant and which are often far less than the default limit for posted data.  The default receive size for mail-enabled public folders is 35MB with Office 365.  This value is adjustable but cannot be set above 150MB and any adjustment affects ALL inbound mail delivery.  There is no option to have separate values for public folders and mailboxes.

If the public folder destination is on-premises the above restrictions do not apply.  However, if there is a future where the migrated data could later be moved to Office 365 it is wise to attempt to apply the limits and expectations now.

Once all “large” folders have been remediated the rest of the steps can be processed.

Public Folder Usage Policies

Priasoft recommends that organizations begin work immediately on the creation of a business policy regarding folder usage (data size and item count) and to implement such against the source data as soon as possible as well as through the migration and continue it afterwards.  Note that such a policy is not something that Microsoft Exchange provide thru any technical settings or features.  Enforcement of the policy will require the creation of one or more scripts and processes and human activity.  Priasoft recommends the following concepts and patterns in such a policy:

• First Reaction:
  • A set of numbers that defines a “large” folder for the organization.  The numbers should reflect item count and size at which the first reaction occurs.
  • For example:  15,000 items or 7GB of data.
  • The reaction to a folder reaching this value should be nothing more than a notice sent to data owners indicating that the folder is “large”, and that data should be moved or removed, if possible, to keep the folder from being a problem.  The concept here is to push the management to the data to the data owners.
• Second Reaction:
  • A set of numbers that is larger than the First Reaction numbers above.
  • For example:  30,000 items or 8GB of data
  • If a folder were to reach these numbers, the reaction would be to inform data owners that if not corrected, local IT has authority to take preset action on the data (often movement of older items to a child folder) with 30 days (or whatever duration is approved).
• Third Reaction:
  • A set of numbers that is larger than the Second Reaction numbers above.
  • For example:  50,000 items or 9GB of data
  • If a folder were to reach these numbers, the reaction would be for local IT to immediately take whatever preset action is approved (often just moving old data to child folders) and afterwards to inform data owners of the action.

In the above suggestion, the organization should decide what the reaction numbers are for each event.  The item count triggers can be larger for on-premises environments since such often has zero or very low network latency and no throttling associated with connections.  Office 365, in contrast, has both elements that affect the perceived performance of a public folder.

Note that if a public folder reaches the configured limit, users will not be able to add or modify data and may not be able to delete data; sometimes the act of selecting items to delete triggers updates to items like modified time or other that can get blocked when the folder is at max size.  If the same public folder is also mail-enabled, new mail to that folder will fail to deliver and a non-delivery message will be returned to the sender.

In conjunction with the migration of data, if a source folder exceeds the limit of a target folder, the data sync will copy data up to the limit and all following items will silently fail and will be logged and the sync process will continue on to the next folder.

Mail-Enabled Public Folders

Analysis of existing mail-public-folders can occur at any time through the project and in parallel to other tasks as long as it is completed with sufficient time before the cutover event.  It is introduced here in step zero due to its complexity and possible duration of work needed to complete.

Priasoft has written an introduction to this topic here:  The Secret Challenges of Mail-Enabled Public Folders in Microsoft Exchange.  Please take time to read that article before proceeding.

During the cutover event a task is to be performed to mail-enable the new target public folders.  The Priasoft PF Toolbox has a option that creates a PowerShell script of all of the mail-public-folders that are to be enabled on the target system.  However, this task depends on and expects that all the links between the hierarchy and Active Directory objects be correct and in proper working condition.  Any folders that are not proper will not make it into the script and therefor will not be mail-enabled in the target environment.

The PF toolbox option “Analyze Mail Enabled Public Folders” should be used to start the analysis.  This task will enumerate all of the mail-public-folder objects from Active Directory and will attempt to send mail to each.  Once all the mail has been sent it will then scan the folder hierarchy looking for the sent messages in order to build a concrete mapping between the Active Directory proxy object and the folder in the hierarchy. 

The Active Directory proxy objects never have the folder path property.  As such it is often impossible to get the folder path of a mail-public-folder with PowerShell alone. 

The results of this analysis task will produce several lists that require human review and reaction in order to “clean up” the mail-public-folder estate.  The list of categories for which a mail-public-folder can be placed by this analysis are as follows:

• PERFECT MATCH: A perfect match is one where the folder received the test message and the link between the mail-public-folder and the public folder is correct.
• IMPERFECT MATCH: An imperfect match is one where the folder received the test message, but the public folder’s link doesn’t match the mail-public-folder. This can occur if the mail-public-folder is forwarding to another mail-public-folder or if the mail-public-folder was somehow deleted and recreated and received a different AD ‘objectGuid’ value. Imperfect matches should be corrected by using this tool or by another tool that knows how to handle this case.
• MATCHED FOLDER: A matched folder is one that did not receive a test message but the link between the public folder and the mail-public-folder is correct. The failure to receive the test message may be due to PF replication, transport rules, moderation, or other mail delivery rules.
• AMBIGUOUS FOLDER: An ambiguous folder is one where more than one folder in the hierarchy received the same tracking message. Any of the folders could be the expected target, or none of them. These folders require a review to determine which folder in the hierarchy to keep as the mail-enabled folder.
• UNLINKED FOLDER: An unlinked folder is one where the link property exists on the public folder from the hierarchy but does not match any of the mail-public-folders tested. This can happen when the mail-public-folder is somehow deleted in Active Directory directly without properly mail-disabling the folder.
• ORPHANED FOLDER: An orphaned folder Is one that received a test message but doesn’t map back to one of the mail-public-folders. This can happen if a previous run of this task was canceled before completion and the mail-public-folder was deleted in Active Directory without properly mail-disabling the folder.
• ORPHANED MAIL-PUBLIC-FOLDER: An orphaned mail-public-folder is one where the test message was sent but did not appear in any of the scanned folders. This can happen if the public folder was deleted or moved and somehow did not cause the mail-public-folder in Active Directory to be updated or removed. Alternatively, this can happen if the less than the entire hierarchy was scanned. These objects should be reviewed and removed by manually deleting the Active Directory object if all folders in the hierarchy have been scanned. Office 365 may require Microsoft support to take action on these.
• EXCLUDED MAIL-PUBLIC-FOLDER: An excluded mail-public-folder is one where the during the initial analysis of the mail-public-folders it was found that mail could not be sent due to a restriction, rule, moderation or other blocker and was excluded from sending test messages or matched a user pattern for exclusion.
• NON-DELIVERY ERROR MAIL-PUBLIC-FOLDER: A non-delivery error mail-public-folder is one where the sending of the tracking message was returned due to a delivery issue of some sort.

Each of the categories except for the Perfect Matches must be reviewed and actions taken to either correct the issue so that the folder can become a perfect match or removed from Exchange so that it drops out of all categories.

The script created by the PF Toolbox in step #10 will only capture folders that are Perfect Matches.