SharePoint 2013 Technical Diagrams Notes

Introduction

SharePoint 2013 Preview Technical Diagrams are now available here - http://technet.microsoft.com/en-us/library/cc263199(v=office.15).aspx

Ever since SharePoint 2007 started publishing these technical diagrams, I have recommended that architects become very familiar with them. I always start here when trying to understand a new major product release for SharePoint. If you search my blog, you will see that I have directly referenced these diagrams when building SharePoint strategies for customers.

The following is a high-level review of the new architecture changes available with SharePoint 2013.

Corporate Portal Diagrams

I reviewed the two new Corporate Portal Diagrams for SharePoint 2013. From a logical architecture perspective, these diagrams do not have any major changes from the SharePoint 2010 versions. These diagrams accurately capture how organizations should build web applications, site collections, application pools, SharePoint services, etc. to support major business initiatives. The diagrams are still a must read for people who are new or who need a refresher to understand how they should be segregating content and business functions across SharePoint.

Extranet Diagram

The new diagram for SharePoint 2013 extranet architecture closely resembles the corporate portal diagrams, however it is not very revealing on the type of information organizations need when making a decision on how to deploy an extranet. Looking back at the SharePoint 2010 Extranet Topologies diagrams (http://technet.microsoft.com/en-us/library/cc263199.aspx), I find that diagram to be much more helpful and the information contained here still holds true with SharePoint 2013. I would recommend reviewing both of them together.

Services in SharePoint 2013 Diagram

I admit this has always been one of my more favorite diagrams. When this was released in SharePoint 2010, it captured a fundamental change in how SharePoint services are configured and delivered. This new architecture was created to support Microsoft’s ability to deliver SharePoint Online as a SaaS solution.

I reviewed this diagram and nothing has significantly changed in regards to sharing services across farms, the logical architecture of services, service groups and service deployment.

In the services table there are a few new services that have been added.

• Access Services – Do not be confused by this. Yes there was Access Services in SharePoint 2010. At this early point, I know that that Access Services for SharePoint 2013 have been changed to be more focused on utilize the new App Architecture. As such, Access Services for SharePoint 2013 is pretty different. Access Services solutions created in SharePoint 2010 will still be supported moving forward, however they will run in a different service.
• App Management Services - This is a new service that will be used specifically for supporting the new internal catalog or the public SharePoint store. Remember that in SharePoint 2013, everything is an app; EVERYTHING. Even everyday SharePoint lists are now called an app. Once you get over the name change, you will find out it makes complete sense and Microsoft has just aligned what is does with how business users talk about technology.
• Machine Translation Service – This is a new one and as of right now, I do not have much information on the purpose of this service other than the description which says it performs automated machine translation.
• Work Management Services – This service provides task aggregation across management systems including SharePoint, Exchange and Project Server. This is huge from a user perspective. One single place to see all of your tasks. No more building content query web parts to find all tasks; this effectively does this plus goes outside the SharePoint boundary to find more tasks. This is a very exciting service.
• Office Web App Services – Is called out in here as a service that is no longer running inside of SharePoint server. Why? Microsoft strategy is to provide Office Web App Service to other enterprise application than just SharePoint and it strategically made sense to move it out of SharePoint.

In the rest of this diagram there are architecture diagrams for how to architect service groups across farms, none of which have changed from SharePoint 2010. If you are not familiar with this stuff, this is a must read and I recommend reading my old posting on it here.

Mobile Architecture Diagram

There is a brand new mobile architecture diagram provided and obviously this is drive by Microsoft’s focus on being a “services and devices” company. This is a pretty simple architecture that basically describes some things you need to think about if you are going to support mobile to your users and discusses some of the mobile capabilities. This can serve as a launch point for you to begin to dive deeper into how you will support mobile for your organization. The following are some high-level observations I had when reading this the first time:

Extranet – If you are not thinking extranet, you need to so you mobile users can access content when they are on a mobile device. They have some diagrams which will get your started thinking about it and additionally how you can use Unified Access Gateway (UAG) as a reverse proxy to help with that.

Mobile Device Management (MDM) – One interesting thing brought up in this diagram is how do you manage mobile devices? If you need something simple, you can leverage Exchange ActiveSync for remote device wipe, password enforcement, etc. If you are looking for application level MDM there are additional solutions out in the marketplace today that provide even more capabilities.

Application Architecture – The new SharePoint 2013 mobile architecture is introduced. They break it down basically into two logical layers: mobile and SharePoint. Some key points are:

• Automatic Mobile Browser Redirection – Is a new capability that can be used to optimize the mobile experience based on the connecting device. This Feature must be active on the site and will be activated by default on numerous site templates. First there is the Classic View which is used to provide backwards capability to mobile devices and will have a SharePoint 2010 mobile browser experience. Then there is the Contemporary View which is geared to support HTML5 browsers. The Contemporary View is several enhanced features for navigation of SharePoint sites. Additionally, Full Site View is available so the SharePoint site page can be viewed as if it were on a desktop browser or a tablet device.
• Office Hub for Windows Phone – Is an application for Windows phone devices that provides enhanced capabilities to access SharePoint content from multiple places in one spot. It also leverages mobile Office.
• Location – There is a new geo-location field type that is available in a SharePoint List. This can make a list location aware to capture latitude and longitude which can be used with map applications. For instance, if a user enters in data on their mobile device, it will capture where it was done from and then can be displayed on a map. Here is some more information about this - http://technet.microsoft.com/en-us/library/fp161355(v=office.15).aspx
• Push Notifications – There is a new capability to allow notifications to be sent from a SharePoint site to registered applications running on a mobile device. The nice thing about this is that Windows Phone Apps can receive notifications without having to poll. Here is some additional reading on the topic - http://msdn.microsoft.com/en-us/library/jj163784(v=office.15).aspx
• Device Channels – This is a really important new capability as device channels allow you to deliver a publishing site geared specifically to support different types of remote devices. Basically the site can mapped to multiple master pages and style sheets and even control what content you want to make available to specific devices. Here is an overview on the new device channels - http://technet.microsoft.com/en-us/library/fp161351(v=office.15).aspx
• Office Web Apps – As mentioned earlier in this posting, Office Web Apps is now a separate standalone server which does not run inside of the SharePoint boundary. Office Web Apps has been improved a lot to support mobile devices. There are Word, Excel and PowerPoint Mobile Viewers.

SharePoint 2013 Upgrade Process Diagrams

There are two upgrade diagrams that have been provided. Here are some high points I walked away with:

• Must be on SharePoint 2010 – To upgrade, you must be on SharePoint 2010 technologies. This means if you are on SharePoint 2003, 2007 or 2010, you will need to upgrade to the appropriate version to get to SharePoint 2013. There are Microsoft migration partners that have solutions to assist with this. I saw many times, this is the big value proposition for using SharePoint Online as this is handled for customers.
• Database Attach Upgrade – Is the only supported method for upgrading. There is no more “in-place” upgrade option. Frankly that is fine because most customers always went down a database-attach upgrade.
• Preparation – much of the preparation activities that we have discussed in the past with SharePoint 2010 hold true with SharePoint 2013. There is a bunch of information you are responsible for gathering.
• Manual Configuration Settings – In the preparation phase is recommended to get a understanding of all the custom configurations that you may have done because not all of them are going to be migrated. This is because not all databases are upgraded. So many custom configurations in central admin such as alternate access methods, time job tweaks, managed paths, incoming/outgoing email settings, certificates, etc. will need to be documented and reconfigured in the new farm.
• Databases That Can Be Upgraded – There is a set of databases that can be upgraded. They are Content, BDC, Managed Metadata, PerformancePoint, Secure Store, Search and User Profile databases.
• Customizations – This is an important task that needs to be completed. I have seen many cases where good software organizations have not implemented a strong configuration management process and the result is an organization may not know about all the customize code that may be implemented. There are numerous ways to find all of them by running PowerShell commands, doing system directory diffs, checking web.config, etc.
• Upgrade Health Checks – There are some new features that are available to site collection administrators that will show you a health check of a site collection before actually upgrading the site collection.
• Evaluation Site Collection – Site Collection Administrators also have the ability to request the site collection be copied into a new site collection to evaluate how the upgrade will affect any customizations they may have. This is helpful so you can remediate issues before you actually perform the upgrade. This is also nice because your site collection will run in a SharePoint 2010 mode until you are ready to actually upgrade it.
• Testing – Just like for SharePoint 2010, the best way to prepare for a migration is to build up your new SP 2013 farm and then multiple practices runs of that upgrade into the new production environment. An entire process is defined in one of the diagrams and is a great place to start.

SharePoint 2013 Search Diagrams

If you are a reader of my blog, I wrote some long postings about the search architecture for both SharePoint 2010 Search (here) and FAST for SharePoint 2010 (here). I am not going to do a deep dive into all these search components and roles because they are basically covered. As many people now know, the FAST search engine is now the core search engine for SharePoint. It will just be referred to as SharePoint Search. Now you will be able to leverage a very powerful search engine out of the box. However many of the advanced enterprise features of search will only be available in the SharePoint Enterprise addition. I am also really excited about this for SharePoint Online because it can leverage FAST too. SharePoint Online will not be able to do Enterprise Search of line of business systems but a Search Farm (which is FAST underneath the hood) can be configured on premise and SharePoint Online can invoke that search and provide the search results in the cloud; pretty exciting.

I highly recommend taking the time to review both of these diagrams. It explains how each of the components interacts with each other. Additionally there is a diagram the goes into how to scale the server farms for the amount of content you will need to index. There is a great, new table in there that shows you how scaling will work. To be honest, the folks who are really serious about search will say it is an art and a table does not always communicate how you will do it. It always comes down to how many items, the types of data sources, custom transformations, query latency, index latency, etc.

SharePoint 2013 App Overview Diagram

This is an area I plan to do a lot more exploration of this coming year on and writing on this blog. Why? This is something we have been waiting for a long time with SharePoint development. There are several ways to look and this. SharePoint Features which we have been writing for years are Apps. This is name change to better communicate our technology to the end users who have to use SharePoint. However the new SharePoint App architecture is way more than that.

I have seen so many things over the years.

• I think one of the biggest challenges people would run into is developing great SharePoint solutions only to find out they incorporated some dependency they should not have, they wrote some high-end code that should not be running in the SharePoint layer, they cannot leverage their solution outside the SharePoint boundary, etc. We want to resolve those problems by helping developers to deploy solutions in way that will keep their SharePoint environment nimble.
• Plus we want to provide third-party vendors quicker access to customers. We want to help customers to quickly acquire third-party solutions.
• Additionally we want to allow customers to leverage commodity based SharePoint Online. As you may know SharePoint Online has restrictions on high-end custom development and if that code where to run in another location, while be highly integrated with SharePoint Online, that is a huge win.

I will thing of many more reason this year on why this is so great J

Now we can achieve this through the new SharePoint App architecture. The old SharePoint Solution architecture where you create a WSP is still around. Nothing has changed there. This is used to create deployment packages and in many cases is used to deploy code that requires full trust. SharePoint Solution packages will continue to be used by third-party vendors or developed internally with such tools as Visual Studio. You can still create Sandbox Solutions which run in a more secure runtime and can be deployed in SharePoint Online.

Now the new Apps framework for SharePoint 2013 is a packaged up in a file called .APP. It is composed of many of the same types of files, AppManifest, embedded Solution.wsp, etc. Once an app is loaded into SharePoint, it is accessible through the App Catalog in SharePoint. This App Catalog can be controlled at an organizational level.

Remember the big point with Apps is, that the custom solution you are writing may or may not actually run in SharePoint. Full trust code is not supported. Your custom solution code itself may run in a different SharePoint farm, on an IIS server as ASP.net pages, ASP.net pages running in Windows Azure, etc. So how does SharePoint access these solutions running outside the SharePoint context? In simple terms we have an IFrame (with some extensions) that external solution is available through. OAuth provides the secure connection for access SharePoint objects from a remote location. We will additionally use a new extended and robust event model and remote client SharePoint library to write integrated, remote code.

Why is this so great? We are going to ensure that custom applications and solutions that are being developed with SharePoint are isolated. No more writing a bunch code and services that should not be running in SharePoint servers. It is great that you can do whatever you want with SharePoint, however this will drive solution management.

So you may be asking where does this get deployed? There are many different options for hosting.

• SharePoint Hosted – This means the app and all the resources run in SharePoint. Remember you server side is not supported however you can write applications with SharePoint’s JavaScript libraries and such.
• Windows Azure Autohosted – This is a model that is only supported in SharePoint Online. In This case you can write an App package that will have code for Azure and SQL Azure embedded into it. When the application is deployed, the azure solution will be automatically deployed for you. You do not have to go to Azure and set anything up at all; it is all handled for you behind the scenes.
• Provider-Hosted – This is the third model where custom code and solutions are hosted in a separate server in your organization, hosted in Azure, hosted in different SharePoint servers, etc.

Once an App package is installed, it can be managed and monitored through the catalog. End users have the ability to select an app to run in their sites (much in the same way as turning on a Feature). If and when an app is updated, the user can decide how they want to upgrade to the new app.

Again I really plan to go much deeper on this in my blog but for right now, these are just some introduction notes and ramblings on how excited I am about this new capability J

Back Up and Recovery Diagram

There is a new diagram that goes into the details of doing your own back-up and recovery for SharePoint 2013. I know many people have become accustomed to using third-party vendors for supporting these operations and I still believe these vendors will continue to provide features above and beyond what is out of the box. However, if you are a do it yourself sort of person, this is a great diagram to review.

Not much has changed in regards to developing back-up procedures for both the SQL Servers and the SharePoint Servers. There are tons of scenarios covered in here, and I recommend reading this if it is important to you.

SharePoint 2013 Database Diagram

Finally the database server diagram has been updated. This is a really really really important diagram to review if you are managing on-premise servers. It goes over all the SharePoint databases, plus provide sizing and scaling guidance. Great information.

Architecture Considerations for Moving SharePoint to Office 365 and SharePoint Online

Introduction
Why are customers looking to come to the Office 365 cloud? Customers want an environment that can scale on-demand. Customers do not want to be in the business of managing and patching software . Customers want a solution that give better business continuity and service level agreements to their users. Customers want solutions that are better governed and that will force them to do better governance. Customers want to be in the business of building solutions (i.e. an airplane company should focus on building planes; not writing enterprise applications from the ground up).
When I talk with customers about SharePoint Online, this is where they want to move to.
In this blog I am going to talk about how an organization should be looking at moving to SharePoint Online.

1. In first half we are going to analyze SharePoint Architecture and Governance and review what potential blockers people see when moving to the cloud.
2. In the second half I am going to have a more detailed discussion around how to architecturally plan to move to SharePoint Online.

Why the decision is not always straight forward?
Coming to SharePoint Online in Office 365 may not sometimes be the most straight forward decision for organizations that have made a large investment in SharePoint. Going to SharePoint Online is really easy for organizations that are in it for document management, intranets, OOB features, SharePoint Designer, etc. It gets harder for organizations that have had poor governance of their on-premise environments, made signification investments with custom code, utilize features not available in cloud, have a heavy reliance on third party solutions, etc.
There is a solution to these issues but it requires an organization to take a step back and look at what they have.
Poor governance is probably one of the largest changes organizations have faced with SharePoint. These challenges could have been avoided thought with good forethought and planning. There is not always a one size fit all solution either; SharePoint is powerful platform that can be used to be used to implement a very broad set of business requirements.
SharePoint Architecture Foundations
The following a Venn diagram that I always draw on whiteboard with customers.

I typically say that that:

• Information Architecture – Is content types, taxonomy, topology, site collections, sites, libraries, lists, and solutions that drive the management of this content. The information architecture should be driven by business requirements.
• Logical Architecture – Is the architecture of SharePoint services, web apps, databases, etc. to support the information architecture.
• Physical Architecture – Is the configuration, deployment, farms, etc. that actually host the SharePoint logical architecture.

I admit this is open to interpretation but let’s use this as a foundation for discussion.
Here are some of the biggest problems organizations that have come up when I discuss the diagram above:

• Started with Physical Architecture – This is the first mistake that many seem to make. Smart IT are concerned about how many servers to buy and configure they forget to actually create an environment needed to support their business requirements. What usually happens is the environment is not scaled or organized to support the actual utilization; this is where the trouble starts. This pretty much goes away if you go to the cloud.
• No business requirements and software design best practices – One of the biggest, consistent challenges I see many organizations have with SharePoint is no business requirements. I typically just see organizations create very light documents (a bullet list) of requirements and just start building. I very rarely see organizations actually do software design best practices. There are no use cases, data models, ERDs, UI wireframes, data dictionaries, architecture documents discussing what SharePoint elements they plan to use, etc. Why; because SharePoint is such an easy platform to start building with. I personally believe SharePoint is one of the best solution platforms on the market but sometimes organizations need architect a solution fist. This does not go away with SharePoint Online.
• No Governance – Organizations usually forget to put Governance plans to manage content and solutions running in SharePoint. A solution can be anything from a group of team sites or a complex .NET application in SharePoint. I have even seen organizations create Governance plans but then not manage nor adhere to them. An analogy would be a organizing a ton of boxes in their basement really well (with labels too). However as time goes on, they just keep on stuffing and stuffing the room with more boxes so that entry to the room is blocked. A good SharePoint environment needs care and feeding. The information, logical and physical architectures need to have policies and procedures to manage it. Policies, procedures, service level agreements, etc. must be put into place. With SharePoint Online several of these Governance woes are removed because the Office 365 cloud manages the environment. Still Governance needs to be but in place to manage content and solutions deployed in Office 365.
• Over Regulation – I have also seen the term “SharePoint Governance” used as a crutch. I have seen organizations lock down SharePoint too much. My response to that is was there a requirement to lock it down? If so, the right thing was done like a publishing intranet or Internet site – no user should have rights to do what they want. There should be a locked controlled publishing and branding process. However it is perfectly acceptable to have areas of SharePoint that are pure collaboration. Your information architecture and governance plans will drive business users to put content and solutions in the appropriate areas.

There are more things to think about but I am getting a little preachy J
A Not So Unusual Situation
Here is a very common scenario that backs up what I just described. It is very common for an organization to start a SharePoint environment like the following. As you can see they start with a top level SharePoint site. They create some department level sites and some sub sites for their intranet. All in all this is a pretty good start; right?

However within a few months (after some very heavy business and user adoption) we have the following.

We see such things as:

• Team sites starting to sprawl underneath some of the sub-sites within a department. The challenge is being able to support collaboration in what was meant to initially be a publishing site.
• Navigation, presentation, branding and user experience is not consistent.
• We see custom applications either built from scratch or third party solutions purchased and embedded in sub-sites of a department. The challenge is should these custom solutions be hooked directly here?
• Department level project sites are created. The challenge is that as project sites grow and take on new responsibilities for the business, they need to be elevated and made accessible like other sites.

There tons more. And I have had customers tell me this is not an issue with “SharePoint”. They currently have these same exact challenges with their legacy intranet, portals, etc. So why do organizations keep on having these challenges? I usually point to a lack of attention to Governance and Information Architecture.
So How Should You Be Thinking About This?
What if content and solutions that are to be deployed are delivered within a framework? Not a “novel” idea either. Hopefully we can make it as simple as we can. Instead of just adding more and more sites and applications; have rules that drive where we put things.

For example:

• The Intranet site should remain a dedicated publishing site collection. This would be geared towards business users having read-only access with only a small set of content owners. Content is delivered in a consistent and clean fashion so that every site is the same giving the user a unified user experience.
• Create a separate site collection for team sites with appropriate service level agreements. In the team site collection, sites should be can be dynamic generated from pre-define site templates. Users are given the ability to do pure collaboration, standing up lists, libraries, etc. They should be able to share information to complete everyday business tasks. There would also be an expectation that this area is not uniform and that users can do what they want with these sites. Retention polices can be put in place to discard these sites if they have not been used for a pre-defined period of time.
• Create another site collection for business project sites. These have different support SLAs and Governance. These sites are a little bit more formal in nature with only a subset of people that would have access to manage them. Maybe there are designers from the IT department who have responsibility for building and supporting them. There could be policies and automated procedures to move content out of them to other site collections. Retention policies on these sites would be completely different than team sites and these sites may only be deleted when the project is over (or never deleted).
• For custom applications, instead of embedding the application into a sub-site, place them into their dedicated site collections. Navigation links can be made to the application from other site collections. This would give significantly more flexibility to move it or provide it more resources.
• Instead of creating one massive site collection where all scanned documents or records create multiple site collections and then route content to them based on business rules.

This list I provided is not meant to be an all-encompassing list of possibilities and/or solutions. Hopefully you will see that you need to start vertically and horizontally partitioning solutions and data based on such simple things as business rules, security access, data characteristics, etc. Knowing this will help you identify the types of site collections, sites, features, content types, managed metadata, site templates, permission levels, etc. that you need to configure SharePoint with.
The net result is you would take the previous diagram and start creating management boundaries based on the characteristics of the solutions and data managed in SharePoint. This is why Information Architecture is so critical for Enterprise Content Management system like SharePoint.
Putting everything into a single site collection really puts an organization into a tough spot to scale with the business. Microsoft has been publishing great technical diagrams for SharePoint 2010 (http://technet.microsoft.com/en-us/library/cc263199.aspx) and SharePoint 2007 (http://technet.microsoft.com/en-us/library/cc263199(office.12).aspx). Please review and get to know these diagrams well because they accurately tell everything I am talking about here.
Now some people counter this whole thing with why does SharePoint not give this to me? Why does SharePoint not auto-govern itself? SharePoint absolutely comes with a ton of features and capabilities that support a good governance model like Features, site templates, sandbox, managed metadata service, permission levels, and the list goes on. All of the configuration settings are there and organizations just need to set them as appropriate for their business.
I promise; we are coming back to the SharePoint Online cloud but we need to finish setting the stage.
How You Should Change Your Thinking
The following is a diagram a colleague (http://blogs.msdn.com/b/edhild/) and I commonly discuss with customers. I am shamelessly using it because this simple diagram really helps customers with a basic understanding of Information Architecture with SharePoint. Plus it supports everything that I just discussed. This is what we call the “Arch of SharePoint Data”. This is not all encompassing list, however there are different extremes.

There is publishing which are sites that are managed by a small group of users and read by a large community of read-only users. While on the other side is something like MySites which is my personal area to manage information and data. In between these two extremes are tons of different types of sites. Department sites, project sites, organization sites, custom application sites, team sites, etc. There are too many to even try to draw. Each one of these types of sites has different security, retention, data usage, transaction management, business rules, automation, etc.
I tell customers is that is perfectly ok to have a team sites area that are for pure collaboration where business users can spin-up a site, do some work on it for two months and then move on. Some people call this the “wild wild west” and that is ok. Just do not allow pure collaboration in you publishing area which is probably one of the most common mistakes J.
Coming full circle, you can see this is all about solution and data management. An Information Architecture is going to tell you the data characteristics. This will drive you to put content in one area versus another.
This is the beauty of SharePoint. It allows you to create and manage business workloads based on your specific business and mission. Once you have the Information Architecture nailed down, you can determine the Logical Architecture (services) you need and then the Physical Architecture needed to support this. By doing this, you know that both your Logical and Physical Architectures will be driven off real business requirements.
Where Does this All Fit with SharePoint Online?
Once you build an Information Architecture you will see that many of these partitions can be moved to the cloud. Intranets, team sites, project sites, my sites, light weight custom solutions, etc. can all be moved to the cloud. If you look at the “Arch of SharePoint Data” diagram, depending on your scenarios, there is a good chance that 80% to 90% of your solutions can be moved to the SharePoint Online cloud. For some organizations, they will be able to move everything up to the cloud. For some organizations they will have a hybrid.
Regardless the re-architecture should be a primary task in your SharePoint 2007 migrations to SharePoint 2010 anyways. Going through this exercise will put you in a position to move pieces to the cloud and with time, more and more.
So what are the Gaps with SharePoint Online?
If you want to know the exact answer it is clearly spelled out and completely available for you. Read either the Multi-tenant SharePoint Online Service Description (http://www.microsoft.com/download/en/details.aspx?id=13602) or the Dedicated SharePoint Service Description (http://www.microsoft.com/download/en/details.aspx?id=18128).
However really understanding these gaps depends on the perspective and approach you are taking with SharePoint. As I say a lot, “SharePoint means a lot of different things to different people”. I have seen customers extremely happy with using SharePoint with out of the box features and SharePoint Designer. While I have seen other customers use SharePoint as a full application development platform writing thousands lines of code. Knowing what SharePoint means to you will dictate your approach to the SharePoint Online cloud. The approach that I outlined in the first part of this bog will really help you with that decision process for moving to the cloud.
When you read the SharePoint Online service descriptions for both the multitenant and dedicated you will see that the gaps are really small. However there are some ones you must be aware of as they will affect your decisions on how to move to the cloud. The big ones that most people bring up are full trust code, business intelligence, FAST search, PowerShell, and Central Administration. This list will quickly change and become outdated because more and more features will be released with time. However let me address each one as it stands today:

• Full trust code is usually the first one the first challenges. Right now, the only way to deploy custom code to the SharePoint Online Multitenant cloud is using a Sandbox solution. For the SharePoint Online Dedicated cloud, full trust code is supported but it must adhere to a strict set of rules (which are publically available) and the code will only be deployed within set windows. Why such restrictions and strong governance? Well, for all the obvious reasons that would come up if you were tasked with having to run an extremely large SharePoint environment on-premise. What has been one of the biggest issues with SharePoint 2007 Governance? It was developers writing complex code on SharePoint and disregarding the fact that an error they write may take down other sites in the farm (like the content query web part that retrieved too much data on the home page of the intranet <g>). We need to ensure that there is strong security and good performing code and that there is no possibility the Company A can take down Company B. The only way to achieve that service level agreement is to have an environment for running governed code. The net effect is that there will be limitations but you will have that guaranteed uptime. My golden rule is that all SharePoint development (on-premise and cloud) should begin with Sandbox solutions and only when the customizations cannot run in the Sandbox then build as full trust solutions. Doing this will ensure you have agility to move to the SharePoint Online cloud when you are ready. If you really need to do complex operations and manage data structures consider using services Windows Azure integrated through Business Connectivity Services (BCS), Silverlight, etc. But still that may not always suffice and that is why SharePoint Hybrid implementations will be commonplace for organizations with mature SharePoint deployments. I will cover this in more detail shortly.
• Business Intelligence today in Office 365 is Excel and Visio Services. Other SharePoint business intelligence services like PerformancePoint, SSRS, PowerPivot and Chart Parts are not available right now. Another limitation today is Excel and Visio services only utilize data that is within the SharePoint Online context; it is not able to reach outside (i.e. to back-end databases). More and more capability will be released through the Office 365 cloud; just for the time being Silverlight and Windows Azure can be used to satisfy these requirements.
• FAST Search is not available in the SharePoint Online cloud right now. It is possible to integrate a local FAST farm with SharePoint Online Dedicated; but not Multitenant. Still take comfort in the fact that SharePoint 2010 search made significant jumps forward from the SharePoint 2007 offering and provides a very strong search experience. With time more and more advanced search capabilities will be released in the cloud.
• PowerShell currently is not fully available with SharePoint Online. There are a lot of PowerShell commands available for Exchange Online and user subscription management; however the full set of SharePoint PowerShell commands is not available today.
• Central Administration Site is not available and this should be expected by anyone who understands what cloud architecture. There are administration screens available to some operation that you would normally perform in Central Admin however it is limited and does not give you the granular control. Why? Well this is the cloud. Customers want to come to the environment so they do not need to be in the business of managing every little configuration of SharePoint.

And the reality is with more and more releases of SharePoint Online this gap is going to continue to close. Still, even if it were to completely close, there will be perfectly valid reasons why some SharePoint may remain on-premise thus, creating a SharePoint Hybrid environment.
What is the SharePoint Hybrid Architecture?
SharePoint Hybrid is as simple as it sounds; it is some SharePoint delivered through the cloud and some SharePoint delivered through on-premise. What will drive you to have SharePoint on-premise? All of that has pretty much been covered to this point. Anything from a specific feature to a business policy may keep some SharePoint on-premise. However using the approach I laid out, you will be able to significantly reduce your footprint of SharePoint on-premise and gain the advantages of the cloud.
The great thing about using SharePoint on-premise is that it is the same software being run in the cloud. This means it is very easy to deliver a consistent user experience between on-premise and the cloud. Branding, navigation, security groups and single sign-on can be configured in such a way that the user can go between these two environments and not know it.
What Type of Cloud is SharePoint Online?
One other thing I want to discuss is what type of cloud is SharePoint Online. I really like this picture it really spells it out for people whom are not fully aware of the multiple delivery models for cloud computing.

Starting on the left side, you see on-premises and this is how most organizations run SharePoint. You must own the entire solution; all the way up the computing stack. Next is Infrastructure as a Service (IaaS) which is the cloud environment that is managed all the way up to the virtualization layer. The company is responsible for everything else; including the management of the operating systems. From a SharePoint perspective that means all the software installation, configuration, management, patching, adding new servers to meet demand, load balancing, etc. needs to be managed by the you. IaaS gets you of the business of hosting virtual SharePoint servers.
Platform as a Service (PaaS) is running the environment all the way up to the application and data tiers. Windows Azure is a PaaS cloud. In this cloud you build custom applications, data models and run them through a highly available environment.
Software as a Service (SaaS) is the entire stack delivered in the cloud and this is the delivery model for SharePoint Online. You do not have to install software, manage patches, availability, etc. This environment gets you out of the business of managing software and into the business of building solutions. SaaS does not provide the granular level of control of the SharePoint environment (which we have discussed).
The reality is that organizations and companies need to save costs and SaaS is the cloud delivery framework they want for the long-run. IaaS can run SharePoint 2010 and can be leveraged as a replacement for on-premise complex SharePoint computing. Still it is well recognized by industry that companies want more SaaS solutions.
Supplement with SharePoint Online with Windows Azure
Windows Azure (PaaS) and SharePoint Online (SaaS) can be used together to deliver end-to-end cloud solutions. Companies that have mature SharePoint deployments commonly have:

• Code that runs in full-trust
• Are managing complex data
• Require the ability to do back-end systems integration.

One thing I have been talking with customers about is offloading that code out of SharePoint and into to the Windows Azure (PaaS) cloud. The Windows Azure cloud allows you to develop and deliver custom code, services, complex data (SQL Azure) and connected back-end integration (AppFabric). These complex operations can be connected through Business Connectivity Services (BCS) in SharePoint Online.
This makes a lot of sense too when you take a step back. I have already said that you should develop SharePoint code to the Sandbox first and when there is good reason to deliver outside the Sandbox. Here is a similar question. At what point do you know you should be developing in SharePoint? I fully recognize there is a gray area here.
I say good software development patterns and practices should drive that decision. This is why Windows Azure is so interesting with Office 365 because we can move complex code that cannot run in the SharePoint Sandbox to the Windows Azure cloud. I recognize this is not a perfect rule because some code needs to run in SharePoint as full trust. However this should be part of your design analysis to reduce to your SharePoint on-premise footprint.
There are lots of different ways Windows Azure can be utilized with SharePoint Online. There may be situations where you need:

• To work with data in SharePoint but you have complex relationships in the data model that are not right for SharePoint lists. Use SQL Azure to manage that data and build services and connect via BCS or Silverlight.
• To integrate with line of business applications on premise use custom services deployed in Windows Azure or AppFrabic. Again BCS or Silverlight can connect Windows Azure which is conduit to line of business applications.
• To use custom web services to perform complex computations and logic. Again offload that to Windows Azure.
• As well, reverse the direction. There may be solutions and applications delivered in Windows Azure can utilize SharePoint Online services. There are SharePoint web services, REST services for data, SharePoint Client API, Javascript APIs and Silverlight APIs that can be used as integration points. A simple scenario could be a web page deployed in Windows Azure that needs to manage documents. Instead of building that up from scratch in Windows Azure just connect through SharePoint Online APIs and deliver SharePoint Online services through that custom web page.

At the end of the day both SharePoint Online and Windows Azure can be used to complement each other in the delivery enterprise business solutions through the cloud.

Conclusions
Why did I go through all of this? The answer is simple; to give SharePoint Architects ideas on how to move forward with SharePoint Online. There really should be no blockers as long as you take a realist look at your Information Architecture and assess how you use SharePoint. There will be lots of stuff which can clearly be moved to the cloud, there will be some stuff where it is not appropriate (hybrid) and then there is that gray area. However I really hope that approach I put forth will help you with your thinking into how to significantly reduce, if not completely remove, your SharePoint on-premise footprint.
Thanks
Special thanks to Chris Geier and Stephen Cawood for providing me feedback and advice as I wrote this.

What is SharePoint Governance?

1 Introduction

One of the most challenging things with SharePoint is not user adoption; far from it. From what we have seen with SharePoint is that enjoys extremely high adoption however with most implementations of SharePoint tend to grow at an uncontrolled pace. From an IT perspective, it is reminiscent of file share. Tons of sites, more sub sites, upon sub sites and upon sub sites of content. Clients come back every time saying, we love SharePoint but we have issues. In most cases it is not having governance and program management to support SharePoint. SharePoint is not like other server products out there that IT departments purchase or support. It is not like PeopleSoft, SAP, SQL Server, whatever else that is out there. SharePoint allows everyday users to freely create and store content. If not managed or configured correctly such that SharePoint does not align itself with the goals and objectives of the organization it will grow in an uncontrolled rate.

The following are reasons why governance should be established:

• Improve information reliability, availability, and security.
• Address how information is shared, used and analyzed internally and externally.

2 What is Governance?

Governance is the combination of people, policies and processes that an organization leverages to achieve a desired outcome. There needs to be a measurable outcome that the organization expects to achieve through governance to obtain a desired result. To accomplish this there must be adoption by both stakeholders and users. To have high adoption the right people must be selected to craft policies and then SharePoint stakeholders must given a clear understanding of how to meet these policies.

To create these policies executive, financial, IT, department leads, compliance, development and information workers need to be selected. It is extremely important to pick people who share the common vision and work together to achieve a common goal. Once policies have been created, heavy handed enforcements are not needed. Instead policies can be enforced with education, training and communications plans. Training needs to be geared specifically to targeted audiences and cannot be in the form of a onetime push of information. Still even with training a minimal level of enforcement will be required and processes will have to be created to enforce policies. Processes can be executed through system automation or manually. Finally a governing body is needed to measure performance and continually update policy based on the evolving workplace. What should be put in place is a governance process that is scalable and flexible to meeting the demands of the business.

3 SharePoint Governance

The following is Microsoft's definition of how governance should be implemented with SharePoint (http://technet.microsoft.com/en-us/library/cc263356.aspx). Every enterprise is unique and should determine the best way to implement its own governance plan. The following are suggested stages of a governance implementation:

• Determine initial principles and goals: The governance body should initially develop a governance vision, policies, and standards that can be measured to track compliance and to quantify the benefit to the enterprise. For example, at this stage, the initial corporate metadata taxonomy could be determined along with the initial IT service offerings. The initial principles, goals, and standards should be published and publicized.
• Develop an education strategy: The governance policies that you determine must be publicized to your enterprise, and you should have ongoing education and training plans. Note that this includes training in the use of Office SharePoint Server and training in the governance standards and practices. For example, your IT department could maintain a frequently asked questions (FAQ) page on its Web site to respond to questions about its Office SharePoint Server service offerings. Your business division could provide online training that describes the implementation and use of the document management system in the enterprise.
• Develop an ongoing plan: Because successful governance should be ongoing, the governance body should meet regularly. Ongoing activities include incorporating new requirements in the governance plan or reevaluating and adjusting governing principles or standards. Conflicts may need to be resolved as competing needs arise, such as between your IT department and one or more business divisions. Your governance body should report regularly to its executive sponsors to promote accountability and to help enforce compliance across the enterprise. Keep in mind that, although this sounds laborious, the goal is to increase the return on your investment in Office SharePoint Server, maximize the usefulness of your Office SharePoint Server solution, and increase the productivity of your enterprise.

Highlighting the last sentence of this is extremely important. What should remember is that this is not easy and that organizations should plan on understanding SharePoint governance before they start.

4 What does a SharePoint Environment without Governance?

Here are just a few things I have seen:

• Site administrator and contributors are not sufficiently trained nor did many have strong information architecture backgrounds.
• No governing body that understands how to use SharePoint to solve business problems.
• No consistency in the way content is presented across SharePoint.
• Roles and responsibilities have not been officially defined.
• Management and staff do not always understand the level of effort involved in building and managing a SharePoint site.
• Site topology is not actively managed.
• There is clear line between what is considered intranet, extranet, and public content.
• Islands of information are created in the form of SharePoint lists and not managed as enterprise business data. This analogous to having Excel and Access manage enterprise business data.
• Custom solution development, deployment and maintenance standards were never created.
• Content databases growth is uncontrolled.
• No content development and integration.
• No document discovery and impossible to find a document. Search is not always the solution.
• Myriad of web documents stored all over the place.
• No document retention and retirement.
• Content security is uncontrolled and impossible to manage accounts.

The challenge that we have seen with the purchase of SharePoint is that sometimes it is positioned as an out of the box silver bullet solution when in fact SharePoint is a platform for creating solutions. Much of the out of the box functionality of SharePoint can be used to solve many business needs. SharePoint empowers every day business users with the ability to create web based solutions that can be highly integrated with Microsoft Office. This is why we have seen such high adoption of SharePoint and why Microsoft is making significant investments into the technology. However SharePoint commonly runs into problems where business users are not properly trained to create solutions nor given parameters in which they should work within. This is why governance has become a predominant issue with many organizations that have implemented SharePoint.

5 SharePoint with Governance

Creating a governance structure could possibly bring:

• Reliable and available content.
• Business processes that are built around the web content management.
• Information management focusing on identifying owners of high-valued data.
• Secure information.
• Meets business objectives.
• Complies with policies and regulations.
• Carbon footprint reduced.
• Useful taxonomy and metadata management.
• Site topology matches the organization.
• Managed expectations (audience, staff, and executives).
• Roles and responsibilities identified.
• Site continuity.
• Processes to review site statistics to improve user experience and make most popular content readily available.
• Continuous improvement of Search by reviewing search statistics.
• Codified policies and procedures (management, planning, design).
• Process within the PMO to identify opportunities where SharePoint should be used in the enterprise architecture.
• Configuration Management policies for SharePoint.
• Change management business processes.

As you can see, SharePoint governance is not just about managing SharePoint for a system standpoint, it also provide guidance the users of SharePoint.

All of these things can be prioritized and driven by the SharePoint Governance team. To support this team you need:

• Business users and power users who are supportive of SharePoint.
• Systems analysts who are devoted to supporting SharePoint.
• Developers who create custom functionality.
• Systems administrators who manage the SharePoint farms.
• Technical leadership who understand how to translate business requests into SharePoint solutions.
• Business directors who can position SharePoint.

Once governance team is put together, they can continually manage and improve SharePoint as it evolves with the organization.

6 References

• Implementing SOA Governance - http://www.infoq.com/articles/implementing-soa-governance
• Governance Resource Center for SharePoint Server 2007 - http://technet.microsoft.com/en-us/office/sharepointserver/bb507202.aspx
• What is governance? - http://technet.microsoft.com/en-us/library/cc263356.aspx