SharePoint Online Storage Calculation Updates

Introduction

There have been new modifications to SharePoint Online storage. All for the better. If you have been following SharePoint Online and OneDrive for Business, there has been some truly exciting things going on as of late with the announcement for the ability to support 1 TB site collections and 1 TB OneDrive for Business sites.

I have literally seen this as a complete game changer for enterprise organizations to get rid of expensive legacy file drives and move all user data to the cloud. This allows end users to have access to their data where ever they go across devices. It literally changes the way people work.

So What Is Changing?

In this new blog there are some new announcements - http://blogs.office.com/2014/08/18/sharepoint-online-simplifies-storage-management/

Updated Usage Model

SharePoint Online Site Collection storage will now only calculates on actual storage utilized. They give an example:

• Old Model - If you had set the SharePoint Online Site Collection quota at 100 GB, but was only using 20 GB, SharePoint Online would still calculate as if you are using 100 GB from the pooled storage.
• New Model – If you set a SharePoint Online Site Collection quota to 100, but are only using 20 GB in that specific site collection, SharePoint Online will only count the 20 GB that is actually being used. Meaning you still have 80 GB available in pooled storage.

This is great thing given that direction of:

1. 1 TB site collection.
2. No cap on the amount of additional storage you can purchase for SharePoint Online pooled storage.

Remember nothing has changed from how your pooled storage is calculated. It is 10 GB + 500 MB per user. You can purchase as much storage beyond that as you want. OneDrive for Business storage is in no way associated to the calculation for SharePoint Online Pooled storage. Details are located here - http://office.microsoft.com/en-us/office365-sharepoint-online-enterprise-help/sharepoint-online-software-boundaries-and-limits-HA102694293.aspx.

Auto versus Manual

Along with the changes above there is a change in how Site Collections sizes are managed.

• Old Model – In SharePoint Online, every site collection requires a quota to be set for the Site Collection.
• New Model – There is a new setting in SharePoint Online called Site Collection Storage Management. This setting can be set to Auto or Manual.
• New Model Manual - If set to Manual, the SharePoint Online administrator will continue to set quotas to each site collectoin. When set to Manual, the site collection will use the new storage utilization model described above. If the site collection reaches its quota, the administrator will have to allocate more quota. If the site collection does not utilize all the quota but all the SharePoint Online pooled storage, the administrator will either need to free up space or purchase more pool storage on a per GB per month basis.
• New Model Auto - If set to Auto, none of the Site Collections have a quota set. The site collections will grow to the current system maximum which is 1 TB per site collection. Do not worry, the site collections can only use the SharePoint Online pooled storage that is available to them. An administrator will have to purchase more SharePoint Online pooled storage when it gets maxed out. The SharePoint Online Admin Center has several new reports that will show how much storage you still have available.

I can see some pros/cons for both Auto and Manual. For Auto, the big benefit is if you want to keep it simple, Auto is the perfect choice. There are still some reason why you may want to use Manual over Auto; especially if you are a large organization with a lot of new content being created or acquired. What if there are specific site collections that are consuming large amounts of pooled storage? With Manual you will be able to continue to stop the growth by using quotas. For instance:

• Check if there are automation processes building up too much data, should something be changed?
• Determine if the retention policies for content need to be changed.
• Determine if there is a reason to logically partition data into a new site collections because you see even more long-term growth.
• Etc.

I still see good reasons why Manual would be needed.

Site Collection Limits

To go along with this, the number of Site Collections is now being increased from 10,000 to 500,000 Site Collections. That is a lot. This was a needed change to support this. You now have the ability to spin up tons of site collections to support this new storage model.

SharePoint Online DLP

Introduction
I have to say I have been waiting awhile for the new SharePoint Online DLP solution to be released. It is super exciting because we now see this DLP capability being applied to both Exchange Online and SharePoint Online with a strategy to centrally manage both. Here is a brand new blog discussing the new SharePoint Online DLP capability - http://blogs.office.com/2014/08/27/search-sensitive-content-sharepoint-onedrive-documents/. In this blog I will talk about the solution, important facts, plus talk about the future statements made in the Microsoft blog.

Status
As of this posting, according to the Office 365 Public Roadmap, the feature is in Rolling Out phase - http://office.microsoft.com/en-us/products/office-365-roadmap-FX104343353.aspx. If you are an Office 365 for Enterprises customer, you have signed up for the First Release program, you will have the ability to get early access.

The Solution
The new solution provides the ability to discover sensitive data that may be in SharePoint Online and OneDrive for Business. The solution looks very similar to the eDiscovery Center. What this solution is initially providing the ability to use 51 built-in sensitive security types. For instance SSN, Passport number, credit card numbers, etc. You will be able to identify the SharePoint Online and OneDrive for Business locations to search. When items are found you can:

• Preview the results.
• Refine with complex query.
• Export the results with a report and then remediate. Remediation would be anything from deleting the data, changing permissions around the data, check for false positives, etc.
• Ability to save and re-execute the query.

The Details for Implementing the DLP Policy
Additionally here is a TechNet Article that explains the actual steps to use the new SharePoint Online DLP capability - http://technet.microsoft.com/library/dn798914.aspx. As you see, the steps are same for using the eDiscovery Center. The great thing about this is the compliance officers do not need to be trained on a new solution, they will use what they already know to perform DLP operation.

Important Facts
Here are some important facts found out while reading this blog:

• Information types being used for SharePoint Online are the same as the one for Exchange Online. This link was referenced - http://technet.microsoft.com/library/jj150541(v=exchg.150).aspx. I like the consistency. These are basically complex regular expressions that are defined for you, and you have the ability to modify them.
• Data file types that can be detected is listed here - http://technet.microsoft.com/en-us/library/jj219530(v=office.15).aspx

The Future
In the blog, the referred to the future. The important fact is this is not the only release. If you have been an Office 365 customer for a while now, you know that Microsoft is releasing new enterprise features and capabilities quickly. This is reflected in public roadmap. In regards to futures on SharePoint DLP:

• They mention that they are building a capability to create policies that can automatically detect sensitive data along with taking an action once discovered, like quarantining or delete the data. I absolutely love this. SharePoint Online DLP will then have the same concepts that are built into Exchange Online DLP as data is going through transport rules.
• They also mention they plan to create a unified policy experience for DLP across all of Office 365. This will allow a compliance officer to define a DLP policy that would apply to Exchange Online, SharePoint Online and OneDrive for Business.

System Center Office 365 Management Pack

There is a new feature which is on the Office 365 public roadmap called “System Center Office 365 Management Pack” (http://office.microsoft.com/en-us/products/office-365-roadmap-FX104343353.aspx) which I am excited to see.

This is something a lot Office 365 customers will be excited to see because they want to have Office 365 “monitored” holistically across their enterprise. Today organizations have to System Center Operations Manager (SCOM) to monitor their entire organization, however they have no way to get feeds from Office 365. This results in them having to go to the Office 365 Administration Center and rely on RSS feeds for notification and messages. Additionally if you are an organization that may have multiple Office 365 tenants you will be able to consume status on all of your tenants.

With the System Center Office 365 Management Pack this will change. With management pack organizations will be able to get the following:

• Subscription health
• Service status
• Active/resolved incidents
• Message center communications

All of this information is readily available through the Office 365 Administration Portal but will be able as well through a SCOM Pack. Now organizations can notified of health, status, incidents and change through SCOM and if the organization already has existing business process to handle these events, Office 365 can plugged into those existing business processes.

Here is a guide on the new Management Pack - http://www.microsoft.com/en-us/download/details.aspx?id=43708.

Outlook MAPI over HTTP

There was a recent announcement in May 2014 that probably went by with not as much fanfare as some of the recent OneDrive for Business announcements. The announcement I am referring to is changing Outlook Connectivity to be MAPI over HTTP - http://blogs.technet.com/b/exchange/archive/2014/05/09/outlook-connectivity-with-mapi-over-http.aspx.

If you check out the public roadmap site, as of my writing today (June 2014), this feature is currently under the Rolling Out phase - http://office.microsoft.com/en-us/products/office-365-roadmap-FX104343353.aspx.

This change is part of the Exchange 2013 SP1 and Outlook 2013 SP1 updates.

So why am I excited?

Up to now, Outlook has used RPC over HTTP (commonly referred to Outlook Anywhere). This change removes the many of the complexities associated to Outlook Anywhere dependency on legacy RPC. This changes the connection to be a true HTTP connection and no longer requires long-lived TCP connections. This ultimately will provide better performance which is explained in detailed in the reference above.

Additionally this change provides the first step for Outlook to natively support third-party multi-factor solutions (such as smartcards). Today third-party multi-factor solutions can be supported with OWA through ADFS. The goal is to support this not just through browser but rich clients such as well as described here - http://blogs.office.com/2014/02/10/multi-factor-authentication-for-office-365/.

Office 365 Public Roadmap

There was a major announcement today for Office 365 customers. Microsoft has now publishing the Office 365 roadmap publically here - http://office.microsoft.com/en-us/products/office-365-roadmap-FX104343353.aspx. This is huge and been long awaited. This solution will allow enterprise customers to review the roadmap and prepare for change. I say this make the Office 365 cloud “predictable”!!!

Additionally here is an announcement about the new Roadmap site and the First Release program - http://blogs.office.com/2014/06/19/improving-visibility-to-service-updates/?wt_mc_id=publicroadmapsocial. The First Release program was announced at the SharePoint Conference and it is a new solution that allows customers to opt into a program so they can receive new features and capabilities quicker. You do not have the option to select on a feature-by-feature basis; you are either in the program or are not.

Lync Online, Exchange Online and SharePoint Online Bandwidth Planning and Estimation

Bandwidth planning comes up a lot with customers who are coming to the cloud. Most customers are initially concerned with understanding Exchange Online bandwidth. There are definitely new email traffic patterns that must be considered (especially if you are retaining the MX record on-premise). This can be remediated pretty quickly.

However the one that requires even more planning and consideration is Lync Online.

Here is the reference for the Office 365 Internet Bandwidth guide - http://technet.microsoft.com/en-us/library/hh852542.aspx. In this article it has references calculators for both Exchange and Lync bandwidth calculators. The Exchange bandwidth tool is fairly straight forward.

The real reason why I am writing this blog is for Lync Online Bandwidth estimation. Today Microsoft does not have a Lync Online bandwidth tool; there is only the on-premises tool. However we have always said when using this on-premises tool if you:

1. Treat Lync Online as your Lync server deployment.

2. Then treat all end users as “external” users.

3. Then model out all the locations for where you have end users.

The first 30 minutes of the Lync Online How to Estimate Bandwidth presentation (http://channel9.msdn.com/Events/Lync-Conference/Lync-Conference-2014/ONLI301) at the recent Lync Conference 2014 has a great discussion of how you should look and understand the traffic. There are several considerations you need to account for when estimating this traffic. I highly recommend sitting down and watching this session.

If you feel I missed on mentioning SharePoint Online, as you may see here - http://technet.microsoft.com/en-us/library/hh852542.aspx - that there is a tool in the works. However if you are a SharePoint person, you can look at many of the existing SharePoint on-premises planning tools, plan out your content acquisition strategy and look at how much content you have today. It comes down to knowing how many upload / download transactions you expect to have at peak times and what is the average file sizes. There can be other considerations for geographically dispersed organizations that again are working with large files. Organizations can look at their current SharePoint on-premise logs to plan.

Office 365: How we run it

My job requires me to talk with a lot of prospective customers about Office 365. One of the biggest tasks I have is to explain to large enterprise customers how we actually do Exchange, SharePoint and Lync in the cloud. Many customers come to the table with a lot of know about Exchange, SharePoint and Lync. They have a lot of operational experience for supporting these products on-premises and they want to know how Microsoft does it? Not to say they do not believe us; but I get the question a lot “show me”. Personally I am not on the operations team however I have to take customers through a lot of conversations to demonstrate to them how we deliver on such a scale.
There are actually some really good presentations from the past Microsoft Exchange Conference (MEC) and SharePoint Conferences which explain how we do it. These discussions cover our operations support, incident response, security, networking, farm provisioning, etc.
I high recommend you take a look at the following sessions:

• Behind the curtain: How we run Exchange Online – MEC 2014 - http://channel9.msdn.com/Events/MEC/2014/ARC301
• Making changes to Exchange Online at high scale – MEC 2014 - http://channel9.msdn.com/Events/MEC/2014/DMI312
• Optimizing Exchange Online for efficiencies and snappy experiences – MEC 2014 - http://channel9.msdn.com/Events/MEC/2014/ARC312
• Secure, Reliable and Evergreen Service at Massive Scale: Lessons learned from Running SharePoint Online – SharePoint Conference 2014 - http://channel9.msdn.com/Events/SharePoint-Conference/2014/SPC279
• Office 365 service communications to customers – SharePoint Conference 2014 - http://channel9.msdn.com/Events/SharePoint-Conference/2014/SPC238
• Office 365 Change Management – SharePoint Conference 2014 - http://channel9.msdn.com/Events/SharePoint-Conference/2014/SPC289
• How We Do It: Operating SharePoint Online – SharePoint Conference 2012 – yes it is from two years ago but still awesome to watch - http://channel9.msdn.com/Events/SharePoint-Conference/2012/SPC122
• How We Do It: Building and Managing SharePoint Online – SharePoint Conference 2012 – yes it is from two years ago but still awesome to watch - http://channel9.msdn.com/events/SharePoint-Conference/2012/SPC121