Exchange Online Troubleshooting Tool

The Exchange Product team posted a new useful tool that I started playing around with called the Mail Flow Guided Walkthrough (GWT) - http://blogs.technet.com/b/exchange/archive/2013/09/03/office-365-mail-flow-troubleshooter-now-available.aspx.

You can access the tool here - http://support.microsoft.com/common/survey.aspx?scid=sw;en;3568&showpage=1.

What it will do is ask you basic questions that challenges you may have and help you debug them. There are several scenarios for Office 365, Exchange Online and hybrid. I see this as a very useful tool to help with debugging initial deployments as well.

For instance I followed the path for checking to see if an Exchange Online mailbox is having issues sending an email. First you need to check if DNS is configured. Second check your NDRs. Third check the health of the service. Fourth run a message trace report. Fifth check the users Outbox, connectivity, third party add-ins, OST files, etc. Really found this to be a great tool.

Exchange Online Hybrid Update

Introduction

I recently wrote two blogs recently on SharePoint Online and Lync Online Hybrid. I figured I complete this out and write some notes on Exchange Online hybrid with Exchange 2013.

The reason why I started off with SharePoint and Lync Hybrid was because there were some major advances with the new 2013 platform being released. Exchange Hybrid has already been a fairly robust solution even before the 2013 release. There are still improvements to be talked about and some good resources you should have.

In this blog I will capture some of the new changes to Exchange Hybrid good resources you should be aware of for planning.

Resource

The best resource to reach is the Exchange Server 2013 Hybrid Deployments Whitepaper - http://technet.microsoft.com/en-us/library/jj200581.aspx. This covers all the details for setting up hybrid.

What is Exchange Hybrid?

Exchange Hybrid allows organizations to host Exchange servers on-premise that are connected to Exchange Online in Office 365. Organizations have the ability to share the same domain space across they hybrid environment and route inbound/outbound email securely between both environments. With Exchange hybrid there is a unified GAL, free/busy is shared, unified messaging, mailboxes can be moved between environments, centralized mailbox management across environments, messaging tracking, mailbox search across environments, etc.

Why Hybrid?

In most cases, the goal of going to the cloud is to remove all on-premise mail servers. In many cases, organizations are able to achieve that. However there can be scenarios that organizations may encounter that may require keeping some mail servers running in hybrid. Two most common are running an extended migration where customers want to slowly move to the cloud or the customer may have a server side add-in that needs to be maintained for a period of time which cannot run in Exchange Online. I have seen scenarios where there are scenarios where there are policies that customers must support which does not allow them to have certain mailboxes reside in the cloud, while everyone else can.

Whatever the scenario is, Office 365 and Exchange Online support this allowing customers some real flexibility when moving to the cloud.

Solution Architecture

This solution architecture has been around for a while. To recap at a high level:

• DirSync – This needed to support the unified GAL across both environments. The organization’s AD will be synchronized to the cloud.
• ADFS – Used to authenticate users in the cloud using their on-premise credentials and domain. Single Sign On is not required, but highly recommended when implementing Exchange Hybrid.
• Microsoft Federation Gateway – Is the trust broker between the two environments.
• CAS / Edge Server Connection – On-premise an Exchange CAS or Edge Transport server can be connected to the cloud instance. Here is more information about the server the transport server you would configure to run with the cloud - http://technet.microsoft.com/en-us/library/hh134662.aspx.
• Mailbox / CAS Server - There are pre-requisites you must consider when setting up this hybrid. One important one is ensuring both the CAS and Mailbox servers that are running in hybrid are running at the right level. For instance is possible to have an Exchange 2007 or later org on-premise but will require the correct level Mailbox/CAS server also be added to the farm on-premise. Please review the prerequisites for details - http://technet.microsoft.com/en-us/library/hh534377.aspx.

What’s New?

Now let’s get to what is new. Actually it is listed here in detail - http://technet.microsoft.com/en-us/library/jj200790(v=exchg.150).aspx – but I will summarize.

• Right out of the gate, we are no longer referring to Exchange hybrid as “rich co-existence”, we are just referring it to Exchange Hybrid J
• Hybrid configuration tools have been consolidated and streamlined to make the configuration simple.
• As you know FOPE has been replaced Exchange Online Protection (EOP). EOP has new features that allow for easier connection configuration between on-premise and the cloud. EOP is the endpoint that is connected to and you no longer have to configure a static IP address. We now have forced TLS configured between the two end points.
• New support options for routing inbound and outbound email based on the location of the MX record.
• Etc.

Planning

This entire whitepaper is filled with very important articles. However when doing your planning, especially on how you want do your mail routing, the following articles two articles are a must read: http://technet.microsoft.com/en-us/library/jj659055(v=exchg.150).aspx and http://technet.microsoft.com/en-us/library/jj659050(v=exchg.150).aspx. The big decision you have is where you want to have your MX record reside. You can have it remain on-premise or have it point to EOP. Both have considerations where are discussed in these articles.

Deployment

As I mentioned, there is a ton of information located off here - http://technet.microsoft.com/en-us/library/jj200581(v=exchg.150).aspx. There is information about prerequisites, considerations, routing, server topologies, deployment steps, and management considerations.

Email and Process Automation with Office 365

New Site Mailboxes

I have been asked many times about email enabled document libraries in SharePoint Online. With the new SharePoint Online (SharePoint 2013 in the cloud) there are new Site Mailboxes.

Site mailboxes are a rather interesting solutions as it brings together the ability to both documents and emails into the same user interface, while continuing to leverage both Exchange and SharePoint to store data.

Here are some really good articles about Site Mailboxes:

• Site Mailboxes - http://technet.microsoft.com/en-us/library/jj150499(v=exchg.150).aspx
• Site Mailboxes in the new Office - http://blogs.technet.com/b/exchange/archive/2012/08/22/site-mailboxes-in-the-new-office.aspx
• Prepare for using site mailboxes in Office 365 - http://office.microsoft.com/en-us/office365-suite-help/prepare-for-using-site-mailboxes-in-office-365-HA103834109.aspx - good article that explains all the features that an end user will have.

You can read about many features but the big one is emailing items to the site mailbox.

Initiating a Process Based on Email Arrival

The next question I have come up against is now that we can email items to a site mailbox, can we do any automation when an item arrives (i.e. some code needs to be executed or a process initiated)? That is the part that must be thought through.

As you see in the picture below and read the referenced articles I provided, you will see that some data is stored in Exchange and some is stored in SharePoint. Emailing to a Site Mailbox will store the item in Exchange while dragging / dropping a file through Outlook to a Site Mailbox will store the document in SharePoint. Knowing that, you would have to have code implemented in two places.

What are the options?

For SharePoint Online it is simple enough to create an event receiver or configure the document library to initiate workflow when an item arrives.

For Exchange Online, it is possible to use Exchange Web Services (EWS) is it possible to write code that will listen for arrivals of emails and then execute code to do “something”. Please read following on setting up a streaming subscription to a Exchange mailbox using Exchange Web Services (EWS) Managed API - http://code.msdn.microsoft.com/Exchange-2013-Set-push-82738cc5.

Areas to Get Started

In Office 365, if you really need to build a solution that will initiate automated business processes based on an arrival of an email, it would be better to start with EWS. Create a mailbox that emails would be sent to. When email arrives your code will be connected to Exchange through EWS which can connect with line of business systems across the enterprise, integrate to SharePoint Online through remote APIs/Web Services, etc.

If you just need to support business processes when email arrives but does not be highly automated you have lots of options:

• Site Mailboxes – Which we have already discussed. This is a great solutions to enable group collaboration around shared email and document data. This works great for projects.
• Shared Mailboxes – This is a great solution where an email address can be created and then allow multiple to monitor and then respond from that email address.
• Distribution Groups – Email is sent and stored in each person’s individual mailbox.
• Public Folders – Yes they are now support on Exchange Online. They have typically been used to storage email and allow people to centrally access it. Public folders are not a recommended solution for storing large files nor is it a document management system.

New Capabilities eDiscovery with Office 365

Introduction

I can happily say I am very excited about the new eDiscovery capability that is part of SharePoint 2013. I am even more excited for this capability being delivered with Office 365. Up to this point in Office 365 you have had the ability to do eDiscovery in the cloud but you have had to execute multiple searches. Now with the new features that will be released, there is a unified eDiscovery for Exchange, SharePoint and Lync data in Office 365. Additionally there are some new features for Exchange 2013, which are available in Office 365, which allows more granular way of eDiscovery and then placing a legal hold on that item.

The following are my notes and some additional resources which you will valuable as you start exploring this technology more.

• The new eDiscovery solution is part of SharePoint Online Plan 2 and is part of the SharePoint 2013 Enterprise CAL on-premise.
• This solutions allows you to search, hold, and export content from Exchange and SharePoint. Lync instant messages can be captured when they in Exchange’s conversation history folder.
• The high level process is you create a case, identify locations of where to search, and any filter to find it. You can manage sources, eDiscovery Sets (sources/filter combos), queries and all exports done. All of these operations can be done from a site in SharePoint Online.
• The major steps are to Create a Case >> Create a eDiscovery set to find and preserve content (optional in-place hold) >> Create a Query to find and export content (previewing and filtering) >> Release Hold

General Legal Hold Improvements for SharePoint and Exchange

With SharePoint 2013 delivered in Office 365, there will be some additional new features available.

• The state of the content is recorded, thus allowing users to continue to work with the data. With SharePoint 2010 technologies this was not possible as once the item placed on Hold, it is locked down until the Hold is released. Even though users have the ability to edit or delete, SharePoint will ensure there is no loss of an item that was placed on hold. Discovery managers will continue to have access to all the data that was put on hold. There is a new special SharePoint library created at the site level to handle edit and delete scenarios. Basically when anyone of those transactions occur, the file or item on hold will be stored there.
• Preservation can be done at the site level now. Users can continue to use preserved content.

With Exchange 2013, there is again a several new features that are available.

• With Exchange Server 2010, the notion of legal hold was to hold all mailbox data for a user indefinitely or until when hold is removed. The legal hold was placed at the mailbox level. With Exchange 2013 in Office 365 you can now determine what to hold and for hold long.
• Indefinite Holds – This is the way it was done with Exchange 2010 and it is still available. The entire mailbox is put on indefinite hold; nothing can be deleted and edits are managed until the mailbox is released.
• Query Holds – This is new. If you need the ability to query for items to be placed on hold, this is now supported. When items are found in the query, just the items are placed on hold. Additionally this does support not just finding existing items, but to future email items that have not arrived yet.
• Time-based Holds – is the idea where legal hold and retention policies are used in conjunction with each other. Specifically this allows you to place a hold on items for a specific period of time which is calculated from the date the item is received or when the hold is created. What this allows you to easily do is create a rule to ensure that all items are retained for X days / years. For instance a policy that says all emails must be kept for 90 days. When an item is deleted out of the user’s mailbox, it will be retained for the remaining amount of time required by the policy.
• Multiple Holds – this is new where by a user can be placed on multiple holds. When a user is on multiple holds, all of the holds are applied together using an OR operator.
• In summary, with the addition of these new in Exchange Online you will be able to place an entire mailbox or specific items on hold; Email will be preserved whether a user or process edits or deletes an email; Users can be placed on multiple holds; Items can be held indefinitely; Legal hold can be made transparent to the user; eDiscovery searches can be done on items that have been placed on hold.
• One Small Note – the In-Place hold utilizes the Recoverable Items folder which is a replacement to the “dumpster”. The Recoverable Items folder is used in support of Legal Hold. There are four sub folders which are used to manage items. First there is the Deletions Sub Folder which is used when a user were to shift delete from inbox or when an item is deleted from the deleted items folder in their mailbox. Users have the ability to recover items from that folder using the item recovery feature from Outlook or OWA. Second there is the DiscoveryHold Sub Folder which manages items that were on legal hold but deleted by the user. Third is the Versions Sub Folder which manage email items that edited and all versions of the edits. For both DiscoveryHold and Versions Sub Folders items will be removed once the hold is released. None of the data stored in these three folders count against the end-user’s mailbox size limit either. Finally there is the Purgres Sub Folder which is responsible for deleting items once all rules have passed.

If you are deep in both SharePoint and Exchange, what you may not is some convergence. SharePoint has always had query based holds which has been brought forward into Exchange. As well Exchange has always allowed user to still work with their mailbox even though they were on hold which SharePoint did not allow. Additionally SharePoint only support legal hold at the item level and not the site level (which is similar conceptually to a mailbox). This is great and is needed to support the next part of this discusion!!!

eDiscovery Sites

Now let’s talk a little bit about this new eDiscovery site that will be available in SharePoint Online to do eDiscovery on Exchange, SharePoint and Lync.

First you create a case using a site template; pretty straight forward process. Once the site is created you will have the some major buckets of features. There are Discovery Sets which facilitates define the sources the search and creating holds. Second there are Queries which search those sources, allow for previewing of results and ultimately exporting the data.

The below screen a new discovery set. As you can see you have the ability to identify the sources, filters, data ranges, etc. for information you want to hold. You have the ability to define what type of hold you want, and in most cases an In-Place hold will be utilized.

Then once items have be placed on hold, you can use the Search and Export feature to run queries to narrow down and find items. Once you find those items, you can export them.

Below is a screen capture of shows has a query can be created across your held data sources. You have the ability to run the query, get information about that query and even preview items before you do an export. You can see that when the Exchange tab is selected you can see all the email, contacts, etc. objects that were round.

The following is from the same query above and shows you the SharePoint data that was returned.

Finally you can export a query once you have it fined tuned. The big question everyone always asks is how the data will be exported. Data will be exported conforming to Electronic Discovery Reference Model standard:

• SharePoint Documents – in their format
• Lists - .csv file will be created.
• Pages – will be exported in MIME HTML (.mht) files
• Exchange Objects – email, tasks, contacts, calendar, attachments exported in .pst.
• Additional XML manifest that complies with EDRM is provided that captures all the information exported.

Resources:

• Office 365 Preview Service Descriptions - http://technet.microsoft.com/en-us/library/jj819284.aspx
• Plan and manage eDiscovery cases - http://office.microsoft.com/en-us/sharepoint-help/plan-and-manage-ediscovery-cases-HA102893857.aspx
• What's new in eDiscovery in SharePoint Server 2013 - http://technet.microsoft.com/en-us/library/fp161513.aspx
• Exchange 2013 In Place Hold - http://technet.microsoft.com/en-us/library/ff637980(EXCHG.150).aspx
• Overview of eDiscovery and in-place holds in SharePoint Server 2013 - http://technet.microsoft.com/en-us/library/fp161516.aspx
• Plan for eDiscovery in SharePoint Server 2013 - http://technet.microsoft.com/en-us/library/fp161512.aspx

New Exchange Online DLP

There is a great new feature of Exchange 2013 that will be part of Exchange Online. I am really happy to now have native Data Loss Prevention (DLP) features to share with customers. Up to this point, you have been able to utilize Transport Rules to implement light DLP however if you wanted to implement real DLP organizations were required to manage an appliance on premise to support this. Now organizations have the ability to now remove that dependency and utilize DLP delivered through the Office 365 cloud.
Below are some notes and resources that you should know about. The new DLP capability:

• The goal is to help identify, monitor, and protect sensitive information from leaving the organization.
• DLP can be configured through the Exchange Administration Center.
• You have the ability to start with a pre-configured DLP templates to detect information such as PII. You have the ability to create custom templates with sensitive information types. This will save you a lot time.
• Types – Detect sensitive information in attachments, body text, subject lines and adjust sensitive level to take action rules (transport rules).
• DLP Policies are tied directly into Transport Rules. They are no more that packages of conditions, transport rules, action and exceptions.
• Transport Rules – You have the ability to coordinate DLP rules with Transport Rules and create actions to capture information. Transport Rules look for specific conditions on a message and then takes action on them. Transport Rules let you apply messaging policies, secure messages, protect message and prevent leakage. You can prevent information from leaving, filter confidential information, track / copy messages sent / received by individuals, redirect email for inspection, apply disclaimers, etc. You have the ability to incorporate classifying sensitive information. Additionally you can perform content analysis through keyword matches, dictionary matches, regular expression, etc.
• Testing – There is the ability to test rules before actually enforcing them. This is possible by creating rules but not activating. Email-flow is not affected until they are finalized.
• Policy Tips – This is truly a great feature in that preventive actions can be taken with an end user before they actually send an email that could violate a DLP policy. Policy Tips to show users warnings in Outlook in the same manner as Mail Tips. This does require Outlook 2013 client.
• Reporting – DLP Reports are available and can create own specific reports to monitor issues.

Some additional deeper notes:

• There are three ways to create a template – 1) Create template using an OOB one. 2) Import ones 3) Start from scratch. There are OOB templates like PCI Data Security Standard, US Financial data, U.S. Gramm-Leach-Bliley Act (GLBA), HIPPA, Patriot Act, PII, etc. Types of common modifications could be to make certain types of users exempt from specific policies for specific situations. Or maybe even invoke RMS in certain situations when a DLP policy may be broken. This native integration into Exchange Online itself is really exciting.
• There are Sensitive Information Types like a US SSN, Drivers License Number, etc. It is the common rule to find that type. You have ability to create XML files that can be imported through PowerShell to do customer ones. You have the ability to create Entity Rules which define identifiers like SSN. Then there are Affinity Rules which are targeted towards documents. Built of a multiple evidence rules when aggregated together and matches happening in proximity to each other can constitute a DLP policy being triggered. So depending on how many times a rule it tripped in a single item can create a DLP policy to be tripped.
• Sensitive Information Rules can be used to with transport rules to create hard and soft rules. There is a new “If this message contains…Sensitive Information” transport rule. This can be used with existing transport rules and Boolean logic. For example: Limit interaction between recipient and senders – between internal groups and external groups, Applying separate policies for communications internal and external, Prevent inappropriate information from entering or leaving, Filter confidential information, Tracking or archiving messages sent / received by specific individuals, Redirect inbound / outbound message for inspection before delivery, and Disclaimers.
• DLP Supported File Types – All the core file types are supported (including zips and cabs). However if there is an unknown file attached that must go through DLP evaluation an exception will be raised to allow you to take action. For Exchange Online you cannot extend this like you can on-premise because you need to create your own IFilter packages which is not supported in the cloud.

References:

• Exchange Online Preview Service Description - http://technet.microsoft.com/en-us/library/jj819264.aspx
• TechNet Data Loss Prevention - http://technet.microsoft.com/en-us/library/jj150527(EXCHG.150).aspx
• File Types That Are Supported In Transport Rules - http://technet.microsoft.com/en-us/library/jj674307.aspx
• DLP Policy Templates Supplied in Exchange 2013 - http://technet.microsoft.com/en-us/library/jj150530.aspx