Office 365 Preview Service Descriptions

Not sure if anyone has noticed but the Office 365 Preview Service Descriptions are published here - http://technet.microsoft.com/en-us/library/jj819284.aspx. There is really good information in here to help you evaluate all the features and capabilities that are available in each type of Office 365 plan. Remember these are Preview and subject to change; however this will really help you with your planning!!!

If you read my blog at all, you will know that I constantly talk about Service Description updates and such. These are really the most important documents customers should be reviewing as part of their decision to the cloud.

SharePoint Conference, SharePoint Online Operations Team Presentations

1.0 Introduction
I recently went to the SharePoint Conference and attended several great sessions. I am going to put up some of my notes from the conference as there was a ton of great information for the 10,000 people whom attended. I openly admit that I am a little focused on SharePoint Online because I am constantly talking with customers about all our solutions available in Office 365. Also excuse the grammar – I am just trying to get content up as quick as I can…

1.1 Cloud First and Aligned Management
From a SharePoint perspective, Office 365 has really driven Microsoft towards building solutions that are more scalable and manageable. For instance in SharePoint 2010 we were given this “new SharePoint Federate Services model” that really not many organizations stood up on-premise however it is heavily used in SharePoint Online. Now with SharePoint 2013 we can really see that everything is being built for a cloud environment. For instance SharePoint 2013 upgrades, wow. If you look at the way we will be doing upgrades from SharePoint 2010 to 2013 you are going say “this so much more well thought out now”. First, we have lessons learned since the product has been around since 2001 however the cloud has really driven Microsoft to deliver better solutions. Why? Because Office 365 delivers a finically backed SLA with a promise to keep customers moving forward on latest and greatest solutions in the cloud. We will not get stuck in an old version and this has forced Microsoft deliver even better upgrade capabilities.
In this blog I am going to be capturing some information from two specific sessions I attended on “How We Do It” for SharePoint Online. Throughout these sessions it really resonated to me how we were changing the architecture of the SharePoint product to be cloud first. The Microsoft SharePoint Product Group is the same group of people supporting SharePoint Online. I will talk about this later in the blog but this is a big deal as it really demonstrates Microsoft’s commitment to have people, process and technology strategically aligned.

1.2 Sessions on How We Do It at SharePoint Online
There were two amazing sessions at the SharePoint Conference. One was called Operating SharePoint Online and the other was called Building and Managing SharePoint Online. If you are a developer of SharePoint you may not have attended these two sessions but they will blow your mind. Specifically if you are a person that has ever managed a production SharePoint farm, you will really appreciate what they have done; not just from a physical and logical architecture perspective as there is program management and governance built into Office 365 that frankly organizations have a very tough time in building and delivering on-premise. This is driven by the 99.9% finically backed SLA that Microsoft delivers with Office 365.

1.3 Session on Operating SharePoint Online
Here are some of my notes (formalized a little) from the presentation.

Some Current Stats - Microsoft has made over a $3.28 billion dollar investment in data centers that are supporting Office 365. This really demonstrates Microsoft’s commitment to the cloud. At the time of this presentation, in support of SharePoint Online, there are currently more than 13,000 servers with over 37,000 SQL servers in the cloud data center. They indicated they are currently bringing on 30,000 companies a week! They have a 24/7 development staff. Plus they have actually maintained 99.95% YTD deliver of service for SharePoint Online – basically beating their stated SLA. This is absolutely amazing when you hear stats like this, and you have done SharePoint administration, it really makes understand the value that Microsoft is delivering to your organization. Stop and think a second for all the people, business processes, governance, management, etc. that is needed to deliver this.

Goals – The SharePoint Online Operations team discussed their focus on several things such as zero downtime, zero loss of data, always up to date and security/compliance. These are all things which organizations whom deploy SharePoint Online try to adhere to which are very challenging to implement because they can require serious investment in people, process and technology which may not simply deploying some SharePoint servers.

Zero Downtime – During the session they spent a lot of time discussing this.

• In order to support this, the SharePoint Online team is constantly monitoring from multiple different angles. SCOM is highly utilized in support of this activity. They specifically implement scenario based monitoring so it is just not checking server machine status. As part of this they do a lot of live traffic monitoring and watch for patterns. They also implement rather comprehensive scenario based monitoring on the SharePoint Online environment.
• They stated that one of the biggest reasons for their success thus far is their alignment to the SharePoint Product Group. They have direct integration with the people whom write the actual code for SharePoint and these same people have direct responsibilities support SharePoint Online. As I mentioned earlier, it is this sort of alignment which drives great delivery as there are is direct access to people who wrote SharePoint to support SharePoint Online.
• They also stated that even though they have access to the people who built SharePoint Online, from a support perspective they have a goal to “automate everything”. Really this is the only way they can ever scale and they have demonstrated that with the level they are currently delivering at.
• They said stated they are doing close to 172 million probes per month to make there are no issues. They stated that this can result in roughly 600,000 anomalies a month that SCOM may identify. Through correlations systems, they are able to identity roughly 200 escalations a month they may need to deal with. This is pretty amazing when you look at the number of probes and the amount of automation they put in place to discover and automatically resolve. Plus they continually find ways to reduce this.
• When issues are discovered they have an entire automated system will communicate to engineers, manage workflows and tasks, and proactively initiates meetings between responsible engineers. The system even provides a full report and list of past resolutions on how to immediately resolve it if is something that has been encountered before.
• Additionally they talked a little about this internal solution they created with Microsoft Research that can parse ULS logs. I know when I have had to debug SharePoint on-premise production issues in the past I had to work with ULS logs which can never be a fun task. However this tool provides a dashboard, drill down capabilities and pattern analysis across every ULS log across the entire SharePoint Online cloud. It is impressive.

Zero Data Loss – The SharePoint Online Operations team spent some time talking about zero data loss. If you have ever read the SharePoint Online Service Descriptions this directly correlates to RTO and RPO. RTO is Recovery Time Objective which is the target time to between when there is a disaster and when the service is running again. RPO is Recovery Point Objective which is the time associated to the possible data loss that could occur during an unexpected event. The key word here is “disaster recovery”. The definition is nice but how is this actually achieved with SharePoint Online and they explained how they do it. If you are a SharePoint architect you know that this would be driven by SQL Server configuration. The SharePoint Online team stated that this is what they do today. When a document hits SharePoint Online:

• The document is first stored in the content database associated to the site, so that is on place it is stored.
• Second, all the SQL databases are using RAID 10, so there is an immediate duplication.
• Third, there is synchronous SQL Mirroring built up to a DR SQL server in the immediate data center, so that is 4 copies of the file.
• Fourth, there is asynchronous log shipping from the primary cloud data center to the secondary data center. So that is roughly 4 additional copies of the file into the secondary data center.
• Fifth, there are schedule backups at the primary data center and then asynchronous replication of those backups to the secondary data center.

As many of you may know, getting that sort of SQL Server redundancy built and managed can be challenging for many organizations to handle however it is required for Microsoft to meet the RTO/RPO.

On top of all this, remember users have the Recycle Bin to recover items that they may have deleted. Note I re-checked the service descriptions and they state the Recycle Bin will keep deleted items for 30 days and backups are stored for 14 days. Also note with SharePoint Online, the Recycle Bin is can also be used to recover objects such as site and even site collections (through tenant administration).
The Operations team stated that Disaster Recovery for them is a hot standby where data centers are always paired with each other. They adhere to an Active-Passive farm set-up with automated failure overusing DNS. They do tons of monitoring, testing, and production fail overs tests. They states specifically that each data center is production and there is no such thing as on primary data center is taking all the traffic while secondary is just sitting there waiting. They ensure that all data centers are performing primary workloads and if there ever is a disaster, they would just re-distribute that workload across the data centers. They stated that as part of this they demand resiliency at both hardware and software layers. They also indicated that for SharePoint Online that they have never really run into the whole situation where the data center has gone. More realistic scenarios they run into are connectivity issues or something to that effect where they will do a DNS flip and keep operations going.

Always Up to Date – Again one of the biggest reasons why customers want to move to SharePoint Online is to ensure they are always up to date with latest SharePoint software, but additionally all the security and feature patching that is provided to ensure that best secure user experience is being delivered. The SharePoint Online Operations team discussed some of their change management and governance they implement to support this for their customers. They need to make sure that security patches, platform upgrades, escalation responses and latest/greatest features are deployed.
Doing this across a large cloud environment requires a significant amount of automation and they built internal tools that will orchestrate these changes. There is a Change Manager application that manages all the physical and virtual machines. The manager knows the state of every machine, the patch level, how it is being utilized and has deep logic for it to know how to apply patches based on scenarios. Plus VMs (where SharePoint servers roles running for SharePoint Farm) are not all located in the same physical servers. VMs are deployed across multiple physical machines and “availability groups” are created so that when a patch is run, it is executed by availability group to ensure there are no performance issues during patching. The Manager will handle lock management across VMs and SharePoint farms and they state they do patching roughly every two weeks worldwide but this could be more dependent on the need.

The Operations team also noted that changes are not rolled out whenever they feel like it J There is a phased roll out process including change approve board which analyzes every proposed change. They have an automated, multi-step process of numerous environments they will test these changes out before ever going into production. The SharePoint Online Operations team even said “we eat our own dog food” by pushing all completely vetted patches into the Microsoft Corporate’s SharePoint Online production tenant before it goes to customer. SharePoint Online is highly utilized by Microsoft employees.

Secure and Compliant – The final goal they discussed was security and compliance.

• First there was a good discussion on how they are fully patched 100% of the time. They have a team of security specialists (they joked hackers) whose job it to continually search and test for vulnerabilities.
• Security by Design was something the team stressed. Role based access is required at all times, regardless of the task or operation at hand. If there is an operation that must be done by a human, then there are secure consoles that are provided based on your role. Plus permissions are managed using an on-demand access model. The said almost no operations required admin access levels. They also stated the operations people do not need access to customer data to perform the tasks they need to complete. They need to work with system logs and such. If support needs to work with customer data, that would be done as part of a customer request. The goal is to be extremely respectful to customer data. They even discussed that for the US Government cloud personnel must be US Citizens.
• They discussed, which I talk a lot about is, Office 365 support for compliance through audits. ISO 27001, EU Model, HIPPA, FISMA, etc. This is the only way to scale and Microsoft has demonstrated they adhere to the most of them.
• One last thing they discussed is they take the approach that they always assume there could be a breach. This is basically to ensure that they are always proactive, checking, monitoring and improving. To assist them with this they actually have a Big Data solution (which is compliant with all our standards and scrubs out PII) that consume log data for them to proactive searching and security analysis. For instance they said SharePoint Online today generates roughly 2TB of ULS logs per day (that is amazing). They scrub and then push this data in the system and they check for instance SharePoint correlation logs in less than a second going back three or more years.

All in all, this was a very impressive session to sit in where the SharePoint Online Operations team shared with customer what they do to ensure how they meet their SLAs.

1.4 Session on Building and Managing SharePoint Online
The second session I sat in and took notes was on was a session on how SharePoint Online is built and managed. In this session they discussed at length how SharePoint Farms are provisioned.

Layers of Office 365 – They had a good discussion on how they logically break out the layers of Office 365.

• Office 365 Portals – This was the sign up experience and tenant administration services that allow customers to manger purchased services.
• Office 365 Platform Services – This is made up of Commerce / Billing, Identity Platform, authentication, and DNS.
• Office 365 Services – These are the services that you know and purchase today – SharePoint, Lync, Exchange and Office Web Apps.

The SharePoint Online team then discussed some of the components of Office 365. They noted that the SharePoint 2013 bits are the same bits that customer purchase and install on premise. The Service Fabric is made up all the components that are needed to run the service. For instance this is this is made up of several things such as deployment / environments, authentication, tenant administration, upgrade, high availability and production support management.

Layers of SharePoint Online – They then broke out the layers of SharePoint Online as being three core layers:

• Physical – this is all the data centers, machines and physical networks that are used to support SharePoint Online.
• Virtual Machines – they then discussed how Hyper-V was central to their delivery strategy. They also discussed how the break out units of scale by “networks”. Now the term network does not really mean what you normally think. Let’s come back to that a little later.
• Services – they noted that every service that runs in SharePoint Online has a 1+ redundancy strategy. There are thousands of services that are running and everything must be integrated.

Topology – Next the SharePoint Online service team showed a topology of SharePoint in Office 365.

• First they have a network. On that network they have a lot of common services that are available. For instance such services AD synchronization, provisioning services, SCOM, DNS, administration, back-up, etc.
• Then within each network they create what the call a stamp. A Stamp is a set of SharePoint Farms that a customers are brought into. First within the stamp the have a SharePoint Federated Services farm. This was introduce in SharePoint 2010 as a way to create scaled our services for such things a search, metadata managed service, etc. The second farm in the stamp is the SharePoint farm itself including all the WFEs, crawl WFEs, app servers, timer jobs, sandboxes, etc. They said this usually will be around 10 or more SharePoint servers. The third farm is a SQL Server farm. Finally there is a local Active Directory with accounts for the customers who have been provisioned to that stamp. Remember this could be a mixture of cloud based IDs or federated IDs from on premise. Once a stamp is built, there will be a second identical stamp set up on a network. They stated that each one of these stamps could support roughly 100,000 users.
• Third they discussed this component of Office 365 called the Grid Manager. This is the component of SharePoint Online that is responsible for basically running, coordinating and automating almost everything. Then there are other services such as the Global directory, tenant administration, commerce backend, DNS, authentication, incident management, Azure service and CDN services.

Grid Manager – They then proceed to discuss the Grid Manager at more length. Basically the Grid manager is a solution that is in constant communication with all the stamps and networks of SharePoint Online. It does this communication through APIs, web services and powershell scripts. It does a significant amount of remote orchestration through scripts to really support this goal of complete automation. The Grid Manager stores the state information for all managed objects in all of SharePoint Online. It has hundreds of automated jobs to strategically manage all these objects.

Provisioning Process – The operations team then discussed at a high-level how the Grid Manager would provision a new stamp. Many of the operations SharePoint Administrators do but this is completely automated. For instance they have stamps such as bring in the standard VMs, deploy the local AD and SQL farms, create the federated services farm, then the content management farm, then post deployment patching of VMs and SharePoint, etc.

Provisioning New Customers – The operations team then had another interesting discussion on how they provision customers based on the layered architecture they described earlier. They also gave some interesting stats that they on board roughly 30K new tenants a week with roughly 4K new tenants a day. They then discussed some of the rules that would determine when network and stamp that a customer is provisioned to. The Grid Manager basically has tons of factors that it evaluates as part of that such as geography, capacity of existing farms, operation activities currently occurring within a stamp, tenant vision (is it primarily a SP 2010 of 2013 farm), and dependency of services (for instance a government customer will go into a government network and stamps). Once that is done there is a whole another set of provisioning services that are responsible for setting up the initial site collections for the customer, creating DNS entries, creating user groups, etc. They even discussed how they have become pretty smart with doing pre-provisioning of tenants in advance and then can just adjust them as customers come into the service to be even more efficient with delivery.

Upgrades – The operations team had a very interesting discussion on this but my next blog will be focused on that with notes I captured from another session. Will post a link here once I have that done.

1.5 Conclusions
You can draw a ton of conclusions from this. The point everyone should be taking away is building this on your own, even if it is nowhere as near as automated as SharePoint Online is a major task for many organizations to take on. Why? Organizations are in the business of providing goods and services. Even though organizations create IT groups in support of their mission, it is really hard to justify this sort of level of automation and management for an organization that may have a just a 12 server farm on-premise. The value of SharePoint Online is your business can focus IT resources at building solutions versus running them.

1.6 Additional References
There is more information associated if you read the service descriptions. In this case read the SharePoint Online, Security and Continuity and Support Service Descriptions and you will see how all this information plays into supporting them.
http://www.microsoft.com/en-us/download/details.aspx?id=13602

Office 365 for Enterprise Service Description Updates

The Office 365 Services Descriptions have been updated in October 2012 - http://www.microsoft.com/en-us/download/details.aspx?id=13602
If you want to get further details of the changes since you last read them – please read this - http://community.office365.com/en-us/wikis/office_365_service_updates/974.aspx
Some ones you should be aware of are:

• New Office Web Apps features in SharePoint are being released!
• Dirsync Scoping and Filtering is now supported – this is great so you do have to sync up your entire directory.
• There is new Interop to work with Third Party Identity Solutions (or STS servers)
• New PDF Viewing through browser.

There are a few other things as well. I highly recommend adding an RSS feed to this wikipage.

Office 365 Dedicated Service Descriptions Updated - October 2012

The Office 365 Dedicated Service Descriptions have all been updated for October 2012 - http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=18128
The What’s New document nets out all of the latest changes. Some highlights are:

• Personal Archiving for email is now available in dedicated giving each end user 100 GB of storage on top of their primary mailbox.
• WAN accelerator support.
• Note – content migrations for SharePoint Online has changed.
• There are some additional new Lync Online capabilities around reporting, archiving, devices and voice.

SharePoint 2013 Training Videos

Are you a SharePoint professional trying to get a jump start on SharePoint 2013? If so, here are two links to a ton of videos on SharePoint 2013 to jumpstart you. I have been getting my through the material and is really good. It covers architecture changes, new features, new development practices, new infrastructure / deployment, etc.

• IT Pro: http://technet.microsoft.com/en-US/sharepoint/fp123606
• Dev: http://msdn.microsoft.com/en-US/office/apps/fp123626

Happy 5 Years

Holy smokes – I just noticed we need celebrate that I have been running this blog for 5 years now. I am ashamed to say that in July of this year, I actually missed a month. How did that happen? Well it is summer time and in July 2012 was one of the most busiest I have ever been with work. Well with all the new SharePoint products to be released in the next couple months, I will have more than enough stuff to do to make up for it. Happy trails….

SharePoint 2013 Technical Diagrams Notes

Introduction

SharePoint 2013 Preview Technical Diagrams are now available here - http://technet.microsoft.com/en-us/library/cc263199(v=office.15).aspx

Ever since SharePoint 2007 started publishing these technical diagrams, I have recommended that architects become very familiar with them. I always start here when trying to understand a new major product release for SharePoint. If you search my blog, you will see that I have directly referenced these diagrams when building SharePoint strategies for customers.

The following is a high-level review of the new architecture changes available with SharePoint 2013.

Corporate Portal Diagrams

I reviewed the two new Corporate Portal Diagrams for SharePoint 2013. From a logical architecture perspective, these diagrams do not have any major changes from the SharePoint 2010 versions. These diagrams accurately capture how organizations should build web applications, site collections, application pools, SharePoint services, etc. to support major business initiatives. The diagrams are still a must read for people who are new or who need a refresher to understand how they should be segregating content and business functions across SharePoint.

Extranet Diagram

The new diagram for SharePoint 2013 extranet architecture closely resembles the corporate portal diagrams, however it is not very revealing on the type of information organizations need when making a decision on how to deploy an extranet. Looking back at the SharePoint 2010 Extranet Topologies diagrams (http://technet.microsoft.com/en-us/library/cc263199.aspx), I find that diagram to be much more helpful and the information contained here still holds true with SharePoint 2013. I would recommend reviewing both of them together.

Services in SharePoint 2013 Diagram

I admit this has always been one of my more favorite diagrams. When this was released in SharePoint 2010, it captured a fundamental change in how SharePoint services are configured and delivered. This new architecture was created to support Microsoft’s ability to deliver SharePoint Online as a SaaS solution.

I reviewed this diagram and nothing has significantly changed in regards to sharing services across farms, the logical architecture of services, service groups and service deployment.

In the services table there are a few new services that have been added.

• Access Services – Do not be confused by this. Yes there was Access Services in SharePoint 2010. At this early point, I know that that Access Services for SharePoint 2013 have been changed to be more focused on utilize the new App Architecture. As such, Access Services for SharePoint 2013 is pretty different. Access Services solutions created in SharePoint 2010 will still be supported moving forward, however they will run in a different service.
• App Management Services - This is a new service that will be used specifically for supporting the new internal catalog or the public SharePoint store. Remember that in SharePoint 2013, everything is an app; EVERYTHING. Even everyday SharePoint lists are now called an app. Once you get over the name change, you will find out it makes complete sense and Microsoft has just aligned what is does with how business users talk about technology.
• Machine Translation Service – This is a new one and as of right now, I do not have much information on the purpose of this service other than the description which says it performs automated machine translation.
• Work Management Services – This service provides task aggregation across management systems including SharePoint, Exchange and Project Server. This is huge from a user perspective. One single place to see all of your tasks. No more building content query web parts to find all tasks; this effectively does this plus goes outside the SharePoint boundary to find more tasks. This is a very exciting service.
• Office Web App Services – Is called out in here as a service that is no longer running inside of SharePoint server. Why? Microsoft strategy is to provide Office Web App Service to other enterprise application than just SharePoint and it strategically made sense to move it out of SharePoint.

In the rest of this diagram there are architecture diagrams for how to architect service groups across farms, none of which have changed from SharePoint 2010. If you are not familiar with this stuff, this is a must read and I recommend reading my old posting on it here.

Mobile Architecture Diagram

There is a brand new mobile architecture diagram provided and obviously this is drive by Microsoft’s focus on being a “services and devices” company. This is a pretty simple architecture that basically describes some things you need to think about if you are going to support mobile to your users and discusses some of the mobile capabilities. This can serve as a launch point for you to begin to dive deeper into how you will support mobile for your organization. The following are some high-level observations I had when reading this the first time:

Extranet – If you are not thinking extranet, you need to so you mobile users can access content when they are on a mobile device. They have some diagrams which will get your started thinking about it and additionally how you can use Unified Access Gateway (UAG) as a reverse proxy to help with that.

Mobile Device Management (MDM) – One interesting thing brought up in this diagram is how do you manage mobile devices? If you need something simple, you can leverage Exchange ActiveSync for remote device wipe, password enforcement, etc. If you are looking for application level MDM there are additional solutions out in the marketplace today that provide even more capabilities.

Application Architecture – The new SharePoint 2013 mobile architecture is introduced. They break it down basically into two logical layers: mobile and SharePoint. Some key points are:

• Automatic Mobile Browser Redirection – Is a new capability that can be used to optimize the mobile experience based on the connecting device. This Feature must be active on the site and will be activated by default on numerous site templates. First there is the Classic View which is used to provide backwards capability to mobile devices and will have a SharePoint 2010 mobile browser experience. Then there is the Contemporary View which is geared to support HTML5 browsers. The Contemporary View is several enhanced features for navigation of SharePoint sites. Additionally, Full Site View is available so the SharePoint site page can be viewed as if it were on a desktop browser or a tablet device.
• Office Hub for Windows Phone – Is an application for Windows phone devices that provides enhanced capabilities to access SharePoint content from multiple places in one spot. It also leverages mobile Office.
• Location – There is a new geo-location field type that is available in a SharePoint List. This can make a list location aware to capture latitude and longitude which can be used with map applications. For instance, if a user enters in data on their mobile device, it will capture where it was done from and then can be displayed on a map. Here is some more information about this - http://technet.microsoft.com/en-us/library/fp161355(v=office.15).aspx
• Push Notifications – There is a new capability to allow notifications to be sent from a SharePoint site to registered applications running on a mobile device. The nice thing about this is that Windows Phone Apps can receive notifications without having to poll. Here is some additional reading on the topic - http://msdn.microsoft.com/en-us/library/jj163784(v=office.15).aspx
• Device Channels – This is a really important new capability as device channels allow you to deliver a publishing site geared specifically to support different types of remote devices. Basically the site can mapped to multiple master pages and style sheets and even control what content you want to make available to specific devices. Here is an overview on the new device channels - http://technet.microsoft.com/en-us/library/fp161351(v=office.15).aspx
• Office Web Apps – As mentioned earlier in this posting, Office Web Apps is now a separate standalone server which does not run inside of the SharePoint boundary. Office Web Apps has been improved a lot to support mobile devices. There are Word, Excel and PowerPoint Mobile Viewers.

SharePoint 2013 Upgrade Process Diagrams

There are two upgrade diagrams that have been provided. Here are some high points I walked away with:

• Must be on SharePoint 2010 – To upgrade, you must be on SharePoint 2010 technologies. This means if you are on SharePoint 2003, 2007 or 2010, you will need to upgrade to the appropriate version to get to SharePoint 2013. There are Microsoft migration partners that have solutions to assist with this. I saw many times, this is the big value proposition for using SharePoint Online as this is handled for customers.
• Database Attach Upgrade – Is the only supported method for upgrading. There is no more “in-place” upgrade option. Frankly that is fine because most customers always went down a database-attach upgrade.
• Preparation – much of the preparation activities that we have discussed in the past with SharePoint 2010 hold true with SharePoint 2013. There is a bunch of information you are responsible for gathering.
• Manual Configuration Settings – In the preparation phase is recommended to get a understanding of all the custom configurations that you may have done because not all of them are going to be migrated. This is because not all databases are upgraded. So many custom configurations in central admin such as alternate access methods, time job tweaks, managed paths, incoming/outgoing email settings, certificates, etc. will need to be documented and reconfigured in the new farm.
• Databases That Can Be Upgraded – There is a set of databases that can be upgraded. They are Content, BDC, Managed Metadata, PerformancePoint, Secure Store, Search and User Profile databases.
• Customizations – This is an important task that needs to be completed. I have seen many cases where good software organizations have not implemented a strong configuration management process and the result is an organization may not know about all the customize code that may be implemented. There are numerous ways to find all of them by running PowerShell commands, doing system directory diffs, checking web.config, etc.
• Upgrade Health Checks – There are some new features that are available to site collection administrators that will show you a health check of a site collection before actually upgrading the site collection.
• Evaluation Site Collection – Site Collection Administrators also have the ability to request the site collection be copied into a new site collection to evaluate how the upgrade will affect any customizations they may have. This is helpful so you can remediate issues before you actually perform the upgrade. This is also nice because your site collection will run in a SharePoint 2010 mode until you are ready to actually upgrade it.
• Testing – Just like for SharePoint 2010, the best way to prepare for a migration is to build up your new SP 2013 farm and then multiple practices runs of that upgrade into the new production environment. An entire process is defined in one of the diagrams and is a great place to start.

SharePoint 2013 Search Diagrams

If you are a reader of my blog, I wrote some long postings about the search architecture for both SharePoint 2010 Search (here) and FAST for SharePoint 2010 (here). I am not going to do a deep dive into all these search components and roles because they are basically covered. As many people now know, the FAST search engine is now the core search engine for SharePoint. It will just be referred to as SharePoint Search. Now you will be able to leverage a very powerful search engine out of the box. However many of the advanced enterprise features of search will only be available in the SharePoint Enterprise addition. I am also really excited about this for SharePoint Online because it can leverage FAST too. SharePoint Online will not be able to do Enterprise Search of line of business systems but a Search Farm (which is FAST underneath the hood) can be configured on premise and SharePoint Online can invoke that search and provide the search results in the cloud; pretty exciting.

I highly recommend taking the time to review both of these diagrams. It explains how each of the components interacts with each other. Additionally there is a diagram the goes into how to scale the server farms for the amount of content you will need to index. There is a great, new table in there that shows you how scaling will work. To be honest, the folks who are really serious about search will say it is an art and a table does not always communicate how you will do it. It always comes down to how many items, the types of data sources, custom transformations, query latency, index latency, etc.

SharePoint 2013 App Overview Diagram

This is an area I plan to do a lot more exploration of this coming year on and writing on this blog. Why? This is something we have been waiting for a long time with SharePoint development. There are several ways to look and this. SharePoint Features which we have been writing for years are Apps. This is name change to better communicate our technology to the end users who have to use SharePoint. However the new SharePoint App architecture is way more than that.

I have seen so many things over the years.

• I think one of the biggest challenges people would run into is developing great SharePoint solutions only to find out they incorporated some dependency they should not have, they wrote some high-end code that should not be running in the SharePoint layer, they cannot leverage their solution outside the SharePoint boundary, etc. We want to resolve those problems by helping developers to deploy solutions in way that will keep their SharePoint environment nimble.
• Plus we want to provide third-party vendors quicker access to customers. We want to help customers to quickly acquire third-party solutions.
• Additionally we want to allow customers to leverage commodity based SharePoint Online. As you may know SharePoint Online has restrictions on high-end custom development and if that code where to run in another location, while be highly integrated with SharePoint Online, that is a huge win.

I will thing of many more reason this year on why this is so great J

Now we can achieve this through the new SharePoint App architecture. The old SharePoint Solution architecture where you create a WSP is still around. Nothing has changed there. This is used to create deployment packages and in many cases is used to deploy code that requires full trust. SharePoint Solution packages will continue to be used by third-party vendors or developed internally with such tools as Visual Studio. You can still create Sandbox Solutions which run in a more secure runtime and can be deployed in SharePoint Online.

Now the new Apps framework for SharePoint 2013 is a packaged up in a file called .APP. It is composed of many of the same types of files, AppManifest, embedded Solution.wsp, etc. Once an app is loaded into SharePoint, it is accessible through the App Catalog in SharePoint. This App Catalog can be controlled at an organizational level.

Remember the big point with Apps is, that the custom solution you are writing may or may not actually run in SharePoint. Full trust code is not supported. Your custom solution code itself may run in a different SharePoint farm, on an IIS server as ASP.net pages, ASP.net pages running in Windows Azure, etc. So how does SharePoint access these solutions running outside the SharePoint context? In simple terms we have an IFrame (with some extensions) that external solution is available through. OAuth provides the secure connection for access SharePoint objects from a remote location. We will additionally use a new extended and robust event model and remote client SharePoint library to write integrated, remote code.

Why is this so great? We are going to ensure that custom applications and solutions that are being developed with SharePoint are isolated. No more writing a bunch code and services that should not be running in SharePoint servers. It is great that you can do whatever you want with SharePoint, however this will drive solution management.

So you may be asking where does this get deployed? There are many different options for hosting.

• SharePoint Hosted – This means the app and all the resources run in SharePoint. Remember you server side is not supported however you can write applications with SharePoint’s JavaScript libraries and such.
• Windows Azure Autohosted – This is a model that is only supported in SharePoint Online. In This case you can write an App package that will have code for Azure and SQL Azure embedded into it. When the application is deployed, the azure solution will be automatically deployed for you. You do not have to go to Azure and set anything up at all; it is all handled for you behind the scenes.
• Provider-Hosted – This is the third model where custom code and solutions are hosted in a separate server in your organization, hosted in Azure, hosted in different SharePoint servers, etc.

Once an App package is installed, it can be managed and monitored through the catalog. End users have the ability to select an app to run in their sites (much in the same way as turning on a Feature). If and when an app is updated, the user can decide how they want to upgrade to the new app.

Again I really plan to go much deeper on this in my blog but for right now, these are just some introduction notes and ramblings on how excited I am about this new capability J

Back Up and Recovery Diagram

There is a new diagram that goes into the details of doing your own back-up and recovery for SharePoint 2013. I know many people have become accustomed to using third-party vendors for supporting these operations and I still believe these vendors will continue to provide features above and beyond what is out of the box. However, if you are a do it yourself sort of person, this is a great diagram to review.

Not much has changed in regards to developing back-up procedures for both the SQL Servers and the SharePoint Servers. There are tons of scenarios covered in here, and I recommend reading this if it is important to you.

SharePoint 2013 Database Diagram

Finally the database server diagram has been updated. This is a really really really important diagram to review if you are managing on-premise servers. It goes over all the SharePoint databases, plus provide sizing and scaling guidance. Great information.