Use this cmdlet to create a new item filter that is used to control which items are processed.
This cmdlet and related features were first added in version 1.0.0.250.
PowerShell
New-ItemFilter
- -FilterScript
- [-ItemProps]
- [-AttachmentProps]
- [-RecipientProps]
- [-AttachmentConversion]
Description
The New-ItemFilter cmdlet is used to create an item filter object that can be set on a SyncTask which can be used to control which items are processed or skipped during a session.
When a SyncTask is being processed, and if an item filter is configured, each item that is processed will be evaluated against the filter, except for items which pass the analysis phase as NoChanges.
The item filter works by collecting one or more properties from an item based on the properties requested by the item filter, and allowing those values to be used in a PowerShell script block. This allows for complex item analysis across many properties to determine inclusion or exclusion in the processing.
Every property of an item can be requested and and used in your script, but be aware that many properties are either binary values, or are obscure numeric values with little documentation from Microsoft on the use and purpose.
Note that requesting large properties, like attachment data or message bodies can cause an out-of-memory error there are too many tasks being processed concurrently.
Additionally, using the item filter inherently slows the sync processing since every item has to be retrieved from the source or target store and then the requested properties retrieved converted .NET standard types for use by the scripting engine.
For each item property requested, the same is added to a PowerShell object [PSObject] for which the property name can be access in the script. Before the script block is executed, a PowerShell variable named $item is added to the runspace which will have all the requested properties. In addition, the $item object has a FolderPath and a Skip property. The $item.FolderPath can be used in the script block to do further analysis and decision making and the $item.Skip is a boolean value, which if set to $true, will cause Super-ExMerge to skip that item. Note that the -FolderIncludPattern and -FolderExcludePattern parameters are much more efficient than using the $item.FolderPath property. The $item.FolderPath is only accessible after item properties have been retrieved. In contrast, the -FolderIncludePattern and -FolderExcludePattern parameters are evaluated before item properties are requested.
Specific to attachments, the AttachmentConversion property of the ItemFilter object determines how the attachment data is set in the $item object. This can be used to convert textual attachments, such text files or documents for sting comparisons. However, note that Super-ExMerge is merely converting the raw bytes of the attachment to text. There is no content level parsing of the data. As such, older Microsoft DOC or PDF files that are stored as binary may not yield any usable text on conversion as such would require the application that created the data to parse such, and for which Super-ExMerge has no understanding or access.
An mail item in Microsoft Exchange has 3 versions of the message body, and all item types can have a message body, not just regular mail items. The three body types are: plain text (PR_BODY), html text (PR_HTML_BODY), and rich text (PR_RTF_COMPRESSED). Note that if a body exists, there will always be a plain text body. However, the other formats may not exist even if a plain text body exists. Furthermore, in most implementations an HTML body is actually stored in the rich text body in an RTF encoding scheme. As such, working with the RTF_COMPRESSED body is the most difficult to understand from a structural layer. Super-ExMerge will decompress the rich text body if requested so that the value can be used in text searches, however this does cause additional processing and will further slow the processing rate of items. Use sparingly and judiciously.
When requesting recipient properties, note that a message sender is not typically in the list of recipients. Sender properties are generally found in the main item props and would be retrieved using the ItemProps parameter. Furthermore, note that for an item that is sent between Microsoft Exchange users, the email address embedded in the message is very rarely an SMTP address, but is more commonly an X500 address (legacyExchangeDN). When attempting to filter items based on sender, TO, CC, or BCC values, understanding of the underlying address format is important. An X500 address is commonly in formatted as: /o=OrgName/ou=Exchange Administrative Group (EXCHANGE ROCKS)/cn=Recipients/cn=recipientID. The ‘recipientID’ element is not likely to be an SMTP value, but is often a random set of characters followed by some or all of the recipient’s alias value (mailNickName).
The $item.Attachments and $item.Recipients properties are lists of PSObject objects which each have one or more of the requested properties. For example, to look for items that have an attachment named ‘sales template.docx‘, an enumeration of the $item.Attachements property is to made and each item evaluated: $item.Attachments | foreach { if($_.DisplayName -eq 'sales template.docx'){$item.skip = $true} }. The example assumes that the -AttachmentProps parameter is configured to request the DisplayName property of attachments.
Common attachment properties can be understood from the following Microsoft article: Attachment tables
Common recipient properties can be understood from the following Microsoft article: Recipient table
Examples
Example 1
PowerShell
$filter = New-ItemFilter -ItemProps Subject, PR_SENDER_NAME -FilterScript { if($item.Subject -like '*new amazon pricing!*' -or $item.PR_SENDER_NAME -like '*amazon*') { $item.Skip = $true } }
This example creates a new ItemFilter object that can be used on either the SourceItemFilter or TargetItemFilter (or both) of a new or existing SyncTask. Use the New-SyncTask and Set-SyncTask cmdlets to set the item filter.
The example requests the subject and sender name properties to be retrieved from items and then filter script uses -like operators to compare the values, which if either are true, the item is skipped. Note that the property name used in the -ItemProps parameter becomes the property name of the $item object in the filter script.
This filter will analyze ALL items for every folder processed by the sync session. Example #2 below uses a folder level filter to limit this.
Example 2
PowerShell
$filter = New-ItemFilter -ItemProps Subject, PR_SENDER_NAME -FilterScript { if($item.Subject -like '*new amazon pricing!*' -or $item.PR_SENDER_NAME -like '*amazon*') { $item.Skip = $true } } -FolderIncludePattern 'Inbox|Sent Items'
This example repeats the from Example #1 above, but limits the item level actions to only folder paths that include the words ‘Inbox’ or ‘Sent Items’. Note that the -FolderIncludePattern uses .NET regular expressions to compare against the folder path, and as a case-insensitive comparison. This is the same pattern matching engine as PowerShell’s -match operator.
The use of the -FolderIncludePattern and -FolderExcludePattern help with performance concerns since Super-ExMerge will quickly skip the ItemFilter processing for a folder that doesn’t match. This allows the sync session to limit the CPU and network intensive actions of filtering to only specific folders.
Parameters
-FilterScript
The FilterScript parameters specifies a PowerShell script block that is used to make comparisons against item properties in order to decide on inclusion or exclusion for further processing.
The FilterScript expects to the implementer to use a specially created PowerShell variable named $item that will contain the requested properties of the item. The $item variable will be created an populated before the script block is executed. After execution, the $item variable will be evaluated to see if the $item.Skip property is set to $true. When the script block completes execution, if the $item.Skip property is $true, the item will be skipped and not processed.
When assigning a value to this element, a script block is identified by curly braces { }. For example: -FilterScript { “script detail goes here” }. Note that a literal string is within the braces and that a string is not be passed as the script. The cmdlet will convert the script block to text when the ItemFilter is saved in a SyncTask.
-ItemProps
The ItemProps parameter specifies one or more item properties that should be retrieved from items for comparison and use by the -FilterScript. Specify multiple properties using a comma between each value, or create an array of strings and pass the array to this parameter.
Note that only properties that exist as part of the master list of properties for MAPI are accepted when using a friendly name like ‘Subject’ or ‘PR_SUBJECT’. However, any property may be specified if the numeric value of the property is known, such as 0x3001001E, which equates to PR_DISPLAY_NAME. Refer to Outlook and MAPI documentation for greater detail about properties.
-AttachmentProps
The AttachmentProps parameter specifies one or more attachment properties that should be retrieved from items for comparison and use by the -FilterScript. Specify multiple properties using a comma between each value, or create an array of strings and pass the array to this parameter.
Note that only properties that exist as part of the master list of properties for MAPI are accepted when using a friendly name like ‘Subject’ or ‘PR_SUBJECT’. However, any property may be specified if the numeric value of the property is known, such as 0x3001001E, which equates to PR_DISPLAY_NAME. Refer to Outlook and MAPI documentation for greater detail about properties.
When one or more properties are specified for this parameter, it causes Super-ExMerge to enumerate all attachments for an item and to collect the requested properties for each attachment. The list of attachments and the requested properties are then added to the $item.Attachments property for inspection by the -FilterScript.
-RecipientProps
The RecipientProps parameter specifies one or more recipient properties that should be retrieved from items for comparison and use by the -FilterScript. Specify multiple properties using a comma between each value, or create an array of strings and pass the array to this parameter.
Note that only properties that exist as part of the master list of properties for MAPI are accepted when using a friendly name like ‘Subject’ or ‘PR_SUBJECT’. However, any property may be specified if the numeric value of the property is known, such as 0x3001001E, which equates to PR_DISPLAY_NAME. Refer to Outlook and MAPI documentation for greater detail about properties.
When one or more properties are specified for this parameter, it causes Super-ExMerge to enumerate all recipients for an item and to collect the requested properties for each recipient. The list of recipients and the requested properties are then added to the $item.Recipients property for inspection by the -FilterScript.
Note that SENDER details are not typically found in the recipient list. Sender properties are found in the ItemProps list, with common values being PR_SENDER_NAME and PR_SENDER_EMAIL_ADDRESS. As mentioned in the description of this cmdlet, the a message sent by a Microsoft Exchange sender will have an email address as an X500 value and not as a SMTP address.
-AttachmentConversion
The AttachmentConversion parameter specifies how to render the attachment data, if PR_ATTACH_BIN is requested in as an attachment property.
All attachments on a message are stored as a binary value, even if the attachment is a text file. This parameter determines whether to attempt to convert the binary data to text for comparison.
The following values are available: None, AnsiText, UnicodeText, Utf8Text.
If the value of ‘None’ is used, which is also the default, the $item.Attachments[n].PR_ATTACH_DATA_BIN property will be an array of bytes.
If one of the text options is used, the value will be a string value, but may have embedded NULL, control, and unprintable characters in the string.
The text rendering options simply treat the underlying array of bytes as text and attempt to present them as a string. Attachments that are often seen as textual documents may be encrypted, or obscured and be a mostly binary format. For example, older PDF files and DOC/XLS/PPT files were binary files that had some plain text inter-mixed. Also note that Super-ExMerge is not able to convert a given attachment thru any converter related to the associated application. Such an exercise would be left to the implementer via the -FilterScript.
-FolderIncludePattern
The FolderIncludePattern specifies a .NET regular expression pattern that is used to compare against the folder path of an item being processed to make a quick and low-cost decision on whether to retrieve item properties.
When processing items, if the FolderIncludePattern contains a value, the value is compared against the folder path of the item being processed. If there is match, the item properties are requested and the -FilterScript is processed.
If there is no match, or the value is empty, processing occurs as normal.
This parameter is used to limit the CPU and network intensive work of item analysis to only folders that match the pattern.
-FolderExcludePattern
The FolderExcludePattern specifies a .NET regular expression pattern that is used to compare against the folder path of an item being processed to make a quick and low-cost decision on whether to retrieve item properties.
When processing items, if the FolderExcludePattern contains a value, the value is compared against the folder path of the item being processed. If there is match, the item is skipped for processing by the -FilterScript.
This parameter is used to limit the CPU and network intensive work of item analysis to only folders that do not match the pattern.
Note that -FolderIncludePattern is evaluated before this parameter. If a folder matches the include pattern, the exclude pattern is never evaluated.
Output
This cmdlet places a new ItemFilter object on the pipeline.
