SharePoint and Office Apps

Introduction

There truly is some new exciting releases that were part of 2013 release that were targeted for developers. In regards to development for SharePoint Online and Office 365 there are a bunch of new advancements that will allow for more development.
Back in 2004 I started to build Office and SharePoint solutions. I built a solution on InfoPath 2003, SharePoint 2003, BizTalk 2004 and K2.net 2003 that would integrate in a VAX mainframe with Office (solution was finalist at MS Worldwide Partner Conference). At the time I saw a vision how Microsoft was bringing together their enterprise solutions and I loved it. I stuck with it and saw the waves of SharePoint 2007 and 2010 change the industry.
Now with SharePoint 2013 and Office 2013, I have seen tremendous advancement in giving developers platforms to build solutions and supporting more flexibility to rapidly design, build and publish apps in a way I have never seen Microsoft do. I see significant strides in enabling developers to do SharePoint and Office development is rapid fashion. For me personally, the vision has completely come together.

• There are multiple options for developing solutions (Visual Studio, Napa, SharePoint Designer, SharePoint UI) providing me choice in how I can architect my solutions.
• SharePoint / Office have APIs and services that provide common interfaces making it extremely easy to integrate and to have consistency in design.
• The cloud is providing us flexibility in the delivery of software.

In the past I used spend a lot of time writing detailed blogs to explain solution architecture, design considerations, assumptions, etc. This time around Microsoft has actually written a series that has done a fantastic job at explaining this - http://msdn.microsoft.com/en-us/library/office/apps/jj220030.aspx. This is required reading as far as I am concerned. Solution architects and developers can really use this information them make decisions on how to design and develop their applications. I highly recommend reading every article in the series.
In this blog will capture some of my notes and salient points you should know.
I will be focusing on SharePoint Online and Office 365 development.

SharePoint Apps

It is still possible to do SharePoint development the way many of you know how to do it today on-premise. You can do full trust and farm based development. There is definitely a need to continue to support these sort of high-end, deeply integrated solutions. However full-trust does have limitations:

• I personally feel that sometimes developers jump to0 quickly to full-trust based solutions versus looking at the longer-term solution.
• Cloud based SharePoint Online in Office 365 does not support this.

Initially Sandboxed Solutions, Silverlight and REST services were available to facilitate development in SharePoint Online. Sandboxed Solutions supported a limited set of server-side SharePoint code. What many people found out is that yes it was possible to create SharePoint solutions but there were gaps in development options.
With the new release of SharePoint 2013 in SharePoint Online several new solutions have been provided for us other than just Sandboxed Solutions / Silverlight.

• Client Side Object Libraries - From an App Dev perspective one of the biggest improvements you will see is around Client Side Object Libraries. Specifically .NET and JavaScript libraries have been introduced and can be utilized with SharePoint Online. Client libraries are very robust and provide APIs to work with BCS, lists, libraries, workflow, etc. This also includes new remove eventing models, new mobile development, etc. Here is a reference to those libraries - http://msdn.microsoft.com/en-us/library/office/apps/jj220040.aspx.
• REST Services - As well there is expanded support for REST services with more API support and new OData support. As you will see, this investment is the foundation for the new SharePoint Apps.
• Hybrid – It is not possible to create hybrid solutions with BCS, Search and REST services that unify data across both environments.
• Extending to Office – We will talk more about this later. There were many reasons why the SharePoint product was created by Microsoft. One was to provide a server in which Office could be delivered from. It initially started as a way to deliver and manage Office documents. With Office 2007, 2010 and 2013 clients we have seen direct integration with document library features. This vision was immensely expanded with Office Web Apps. Now with Office Apps, SharePoint and Office can be unified from a custom application perspective. Very exciting as I really think this is uncharted territory.
• The New SharePoint Apps Model – I cannot emphasize how important this is. If you want an overview of it please review this - http://msdn.microsoft.com/en-us/library/office/apps/fp179930.aspx. Apps can he hosted in SharePoint Online, hosted in Azure (auto hosted apps) or hosted on other platforms (Provider Hosted apps). SharePoint Online hosted apps are limited using HTML and SharePoint JavaScript which is very powerful. I would really challenge a developer to start building your SharePoint Online custom solutions there. Auto Hosted Apps is as it sounds, server side code you write is provisioned and executed out of Azure while still utilizing SharePoint to deliver pieces of the solution that the developer is building. Provider Hosted apps can be hosted in on-premise SharePoint, custom .NET applications, ASP.net, Java, PHP, LAMP, etc. It really does not matter where the app lives as long as it can use the API and REST services to integrate. With these new client libraries and OData support we are empowered to do this.

I would like to second here and really focus in on the New SharePoint Apps Model. I think this is great way to move forward. When I was consultant building customer applications on SharePoint 2003, 2007, and 2010 I would find that way too often well intentioned developers would quickly jump to building code for complex business operations and running them on SharePoint directly. Yes you have the power to do this because SharePoint is a platform. However in many cases good software design would have moved those processes off to another resource to run that code. Moving to the new SharePoint Apps model is really going to force developers to decouple their code and create the application layers they should have been doing from the beginning.
This is really where the industry is moving. Creating hosted applications and services that companies can quickly purchase without having own a complex on-premise deployment is what companies seek. This also goes for homegrown custom solutions created internally, build up you custom applications and then manage / deliver them to end users through SharePoint. Yes I know there are exceptions and it totally depends on the context of the solution you are creating. However I see the new SharePoint Apps model as a landmark change for SharePoint. With SharePoint Apps you can build applications on other platforms and then use all the services and API to integrate them. I see environment where companies procure or build solutions which just “hook into” SharePoint Online. SharePoint basically because the corporate portal from providing applications and these applications can leverage a customer’s existing investment in SharePoint. It really is truly exciting. Embrace it. Change for the right reason is good J
To help get your head wrapped around this, I highly recommend you read the following three articles:

• What you can do in an app for SharePoint - http://msdn.microsoft.com/en-us/library/office/apps/fp179899.aspx
• Three ways to think about design options for apps for SharePoint - http://msdn.microsoft.com/en-us/library/office/apps/fp179889.aspx
• Hosting options for apps for SharePoint - http://msdn.microsoft.com/en-us/library/office/apps/fp179887.aspx

Here are some other good reads as you start investigating SharePoint Apps:

• New permission considerations - http://msdn.microsoft.com/en-us/library/office/apps/fp179922.aspx
• Managing Apps - http://msdn.microsoft.com/en-us/library/office/apps/fp179922.aspx
• Host Webs and App Webs - http://msdn.microsoft.com/en-us/library/office/apps/fp179925.aspx
• Chrome control - http://msdn.microsoft.com/en-us/library/office/apps/fp179934.aspx
• UI Guidelines - http://msdn.microsoft.com/en-us/library/office/apps/jj220046.aspx
• New UI Callouts - http://msdn.microsoft.com/en-us/library/office/apps/dn135236.aspx
• Development Get Started - http://msdn.microsoft.com/en-us/library/office/apps/jj163230.aspx
• Deployment Options - http://msdn.microsoft.com/en-us/library/office/apps/fp179933.aspx

Office Apps

From an Office perspective, I personally believe this is one of the areas where I think even more attention needs to be given. Building applications for Office has complete changed in Office 2013; the game has changed. If you recall the days of doing VSTO (Visual Studio Tools for Office) and all the hard work you had to do, those days are gone. The new Office Apps model takes on the same approach as SharePoint Apps.
How has the game changed? Now in you build applications in Office using HTML5, JavaScript, CSS, Rest Services and XML. Yes that is a total change. Now your developers are empowered to using development technologies that many organization have in house. The development is light weight and if you have any sort of web development experience, you will have a starting point for building Office Apps. It is basically a webpage that is hosted inside of an Office client application and it supports the ability for that web application and Office to communicate with each other.
Below probably one of the simplest high-level diagram that explains how Office Web Apps work. Basically a manifest is deployed into Office which knows about the externally hosted application and then provides the ability for the two solutions to integrate with each other. Office Apps can work with ASP.net, PHP, Java, etc. using these client based APIs. You have the flexibility to build Office Apps in the new Napa and Visual Studio.

You can build Office Apps in Word 2013, Excel 2013, Outlook 2013, Outlook Web App (OWA) 2013, PowerPoint 2013 and Project 2013. This really opens the realm of possibilities for apps that organizations of typically avoided because of the older complexities of doing Office development. Remember – it is very common in a business work place that Information Workers user Office at some point in their day. It is even arguable that many people spend several hours a week in Office. In many cases business processes are initiated or people need to go do “something” somewhere else.
For example:

• A user may receive and email, they need initiate a business process. How about providing an automated step to use the data in the email to initiate that business process.
• A user is building a data model in Excel and they need to pull in data using a set of control parameter. How about provide an app that can go retrieve that data and then move it into Excel.
• A user is building a Word document which needs data from various locations. How about an app that review that data and move it into the word document.
• A user is building a project plan and needs rates for users based on their classifications but that sort of information is managed in other financial systems. How about an app that can retrieve that data.
• A user needs to create a Word document and when finished that document needs to be submitted to another system. How about create a simple solution to move that document to that other system.

Hopefully you are seeing the potential. Since users do spend a significant amount of time in Office, why not use Office as a launch point enhance the business productivity experience.
If you are interested in getting started and learning about Office Apps, there are a ton of really good resources that are available to you. Please review the following:

• Must read staring from root http://msdn.microsoft.com/en-us/library/office/apps/jj220060.aspx
• What is Office Development - http://msdn.microsoft.com/en-us/library/office/apps/jj220082.aspx
• What can an App for Office do - http://msdn.microsoft.com/en-us/library/office/apps/jj220082.aspx
• Runtime - http://msdn.microsoft.com/en-us/library/office/apps/jj220082.aspx
• Development basics - http://msdn.microsoft.com/en-us/library/office/apps/jj220082.aspx
• Development Details - http://msdn.microsoft.com/en-us/library/office/apps/jj220077.aspx
• Scenarios: http://msdn.microsoft.com/en-us/library/office/apps/jj220082.aspx and http://msdn.microsoft.com/en-us/library/office/apps/fp161012.aspx
• Device requirements - http://msdn.microsoft.com/en-us/library/office/apps/jj220082.aspx
• Creating your first Office Apps - http://msdn.microsoft.com/en-us/library/office/apps/jj220065.aspx and http://msdn.microsoft.com/en-us/library/office/apps/jj220075.aspx
• Common tasks - http://msdn.microsoft.com/en-us/library/office/apps/fp142243.aspx
• Publishing - http://msdn.microsoft.com/en-us/library/office/apps/fp123515.aspx

SharePoint 2013 Technical Diagrams Notes

Introduction

SharePoint 2013 Preview Technical Diagrams are now available here - http://technet.microsoft.com/en-us/library/cc263199(v=office.15).aspx

Ever since SharePoint 2007 started publishing these technical diagrams, I have recommended that architects become very familiar with them. I always start here when trying to understand a new major product release for SharePoint. If you search my blog, you will see that I have directly referenced these diagrams when building SharePoint strategies for customers.

The following is a high-level review of the new architecture changes available with SharePoint 2013.

Corporate Portal Diagrams

I reviewed the two new Corporate Portal Diagrams for SharePoint 2013. From a logical architecture perspective, these diagrams do not have any major changes from the SharePoint 2010 versions. These diagrams accurately capture how organizations should build web applications, site collections, application pools, SharePoint services, etc. to support major business initiatives. The diagrams are still a must read for people who are new or who need a refresher to understand how they should be segregating content and business functions across SharePoint.

Extranet Diagram

The new diagram for SharePoint 2013 extranet architecture closely resembles the corporate portal diagrams, however it is not very revealing on the type of information organizations need when making a decision on how to deploy an extranet. Looking back at the SharePoint 2010 Extranet Topologies diagrams (http://technet.microsoft.com/en-us/library/cc263199.aspx), I find that diagram to be much more helpful and the information contained here still holds true with SharePoint 2013. I would recommend reviewing both of them together.

Services in SharePoint 2013 Diagram

I admit this has always been one of my more favorite diagrams. When this was released in SharePoint 2010, it captured a fundamental change in how SharePoint services are configured and delivered. This new architecture was created to support Microsoft’s ability to deliver SharePoint Online as a SaaS solution.

I reviewed this diagram and nothing has significantly changed in regards to sharing services across farms, the logical architecture of services, service groups and service deployment.

In the services table there are a few new services that have been added.

• Access Services – Do not be confused by this. Yes there was Access Services in SharePoint 2010. At this early point, I know that that Access Services for SharePoint 2013 have been changed to be more focused on utilize the new App Architecture. As such, Access Services for SharePoint 2013 is pretty different. Access Services solutions created in SharePoint 2010 will still be supported moving forward, however they will run in a different service.
• App Management Services - This is a new service that will be used specifically for supporting the new internal catalog or the public SharePoint store. Remember that in SharePoint 2013, everything is an app; EVERYTHING. Even everyday SharePoint lists are now called an app. Once you get over the name change, you will find out it makes complete sense and Microsoft has just aligned what is does with how business users talk about technology.
• Machine Translation Service – This is a new one and as of right now, I do not have much information on the purpose of this service other than the description which says it performs automated machine translation.
• Work Management Services – This service provides task aggregation across management systems including SharePoint, Exchange and Project Server. This is huge from a user perspective. One single place to see all of your tasks. No more building content query web parts to find all tasks; this effectively does this plus goes outside the SharePoint boundary to find more tasks. This is a very exciting service.
• Office Web App Services – Is called out in here as a service that is no longer running inside of SharePoint server. Why? Microsoft strategy is to provide Office Web App Service to other enterprise application than just SharePoint and it strategically made sense to move it out of SharePoint.

In the rest of this diagram there are architecture diagrams for how to architect service groups across farms, none of which have changed from SharePoint 2010. If you are not familiar with this stuff, this is a must read and I recommend reading my old posting on it here.

Mobile Architecture Diagram

There is a brand new mobile architecture diagram provided and obviously this is drive by Microsoft’s focus on being a “services and devices” company. This is a pretty simple architecture that basically describes some things you need to think about if you are going to support mobile to your users and discusses some of the mobile capabilities. This can serve as a launch point for you to begin to dive deeper into how you will support mobile for your organization. The following are some high-level observations I had when reading this the first time:

Extranet – If you are not thinking extranet, you need to so you mobile users can access content when they are on a mobile device. They have some diagrams which will get your started thinking about it and additionally how you can use Unified Access Gateway (UAG) as a reverse proxy to help with that.

Mobile Device Management (MDM) – One interesting thing brought up in this diagram is how do you manage mobile devices? If you need something simple, you can leverage Exchange ActiveSync for remote device wipe, password enforcement, etc. If you are looking for application level MDM there are additional solutions out in the marketplace today that provide even more capabilities.

Application Architecture – The new SharePoint 2013 mobile architecture is introduced. They break it down basically into two logical layers: mobile and SharePoint. Some key points are:

• Automatic Mobile Browser Redirection – Is a new capability that can be used to optimize the mobile experience based on the connecting device. This Feature must be active on the site and will be activated by default on numerous site templates. First there is the Classic View which is used to provide backwards capability to mobile devices and will have a SharePoint 2010 mobile browser experience. Then there is the Contemporary View which is geared to support HTML5 browsers. The Contemporary View is several enhanced features for navigation of SharePoint sites. Additionally, Full Site View is available so the SharePoint site page can be viewed as if it were on a desktop browser or a tablet device.
• Office Hub for Windows Phone – Is an application for Windows phone devices that provides enhanced capabilities to access SharePoint content from multiple places in one spot. It also leverages mobile Office.
• Location – There is a new geo-location field type that is available in a SharePoint List. This can make a list location aware to capture latitude and longitude which can be used with map applications. For instance, if a user enters in data on their mobile device, it will capture where it was done from and then can be displayed on a map. Here is some more information about this - http://technet.microsoft.com/en-us/library/fp161355(v=office.15).aspx
• Push Notifications – There is a new capability to allow notifications to be sent from a SharePoint site to registered applications running on a mobile device. The nice thing about this is that Windows Phone Apps can receive notifications without having to poll. Here is some additional reading on the topic - http://msdn.microsoft.com/en-us/library/jj163784(v=office.15).aspx
• Device Channels – This is a really important new capability as device channels allow you to deliver a publishing site geared specifically to support different types of remote devices. Basically the site can mapped to multiple master pages and style sheets and even control what content you want to make available to specific devices. Here is an overview on the new device channels - http://technet.microsoft.com/en-us/library/fp161351(v=office.15).aspx
• Office Web Apps – As mentioned earlier in this posting, Office Web Apps is now a separate standalone server which does not run inside of the SharePoint boundary. Office Web Apps has been improved a lot to support mobile devices. There are Word, Excel and PowerPoint Mobile Viewers.

SharePoint 2013 Upgrade Process Diagrams

There are two upgrade diagrams that have been provided. Here are some high points I walked away with:

• Must be on SharePoint 2010 – To upgrade, you must be on SharePoint 2010 technologies. This means if you are on SharePoint 2003, 2007 or 2010, you will need to upgrade to the appropriate version to get to SharePoint 2013. There are Microsoft migration partners that have solutions to assist with this. I saw many times, this is the big value proposition for using SharePoint Online as this is handled for customers.
• Database Attach Upgrade – Is the only supported method for upgrading. There is no more “in-place” upgrade option. Frankly that is fine because most customers always went down a database-attach upgrade.
• Preparation – much of the preparation activities that we have discussed in the past with SharePoint 2010 hold true with SharePoint 2013. There is a bunch of information you are responsible for gathering.
• Manual Configuration Settings – In the preparation phase is recommended to get a understanding of all the custom configurations that you may have done because not all of them are going to be migrated. This is because not all databases are upgraded. So many custom configurations in central admin such as alternate access methods, time job tweaks, managed paths, incoming/outgoing email settings, certificates, etc. will need to be documented and reconfigured in the new farm.
• Databases That Can Be Upgraded – There is a set of databases that can be upgraded. They are Content, BDC, Managed Metadata, PerformancePoint, Secure Store, Search and User Profile databases.
• Customizations – This is an important task that needs to be completed. I have seen many cases where good software organizations have not implemented a strong configuration management process and the result is an organization may not know about all the customize code that may be implemented. There are numerous ways to find all of them by running PowerShell commands, doing system directory diffs, checking web.config, etc.
• Upgrade Health Checks – There are some new features that are available to site collection administrators that will show you a health check of a site collection before actually upgrading the site collection.
• Evaluation Site Collection – Site Collection Administrators also have the ability to request the site collection be copied into a new site collection to evaluate how the upgrade will affect any customizations they may have. This is helpful so you can remediate issues before you actually perform the upgrade. This is also nice because your site collection will run in a SharePoint 2010 mode until you are ready to actually upgrade it.
• Testing – Just like for SharePoint 2010, the best way to prepare for a migration is to build up your new SP 2013 farm and then multiple practices runs of that upgrade into the new production environment. An entire process is defined in one of the diagrams and is a great place to start.

SharePoint 2013 Search Diagrams

If you are a reader of my blog, I wrote some long postings about the search architecture for both SharePoint 2010 Search (here) and FAST for SharePoint 2010 (here). I am not going to do a deep dive into all these search components and roles because they are basically covered. As many people now know, the FAST search engine is now the core search engine for SharePoint. It will just be referred to as SharePoint Search. Now you will be able to leverage a very powerful search engine out of the box. However many of the advanced enterprise features of search will only be available in the SharePoint Enterprise addition. I am also really excited about this for SharePoint Online because it can leverage FAST too. SharePoint Online will not be able to do Enterprise Search of line of business systems but a Search Farm (which is FAST underneath the hood) can be configured on premise and SharePoint Online can invoke that search and provide the search results in the cloud; pretty exciting.

I highly recommend taking the time to review both of these diagrams. It explains how each of the components interacts with each other. Additionally there is a diagram the goes into how to scale the server farms for the amount of content you will need to index. There is a great, new table in there that shows you how scaling will work. To be honest, the folks who are really serious about search will say it is an art and a table does not always communicate how you will do it. It always comes down to how many items, the types of data sources, custom transformations, query latency, index latency, etc.

SharePoint 2013 App Overview Diagram

This is an area I plan to do a lot more exploration of this coming year on and writing on this blog. Why? This is something we have been waiting for a long time with SharePoint development. There are several ways to look and this. SharePoint Features which we have been writing for years are Apps. This is name change to better communicate our technology to the end users who have to use SharePoint. However the new SharePoint App architecture is way more than that.

I have seen so many things over the years.

• I think one of the biggest challenges people would run into is developing great SharePoint solutions only to find out they incorporated some dependency they should not have, they wrote some high-end code that should not be running in the SharePoint layer, they cannot leverage their solution outside the SharePoint boundary, etc. We want to resolve those problems by helping developers to deploy solutions in way that will keep their SharePoint environment nimble.
• Plus we want to provide third-party vendors quicker access to customers. We want to help customers to quickly acquire third-party solutions.
• Additionally we want to allow customers to leverage commodity based SharePoint Online. As you may know SharePoint Online has restrictions on high-end custom development and if that code where to run in another location, while be highly integrated with SharePoint Online, that is a huge win.

I will thing of many more reason this year on why this is so great J

Now we can achieve this through the new SharePoint App architecture. The old SharePoint Solution architecture where you create a WSP is still around. Nothing has changed there. This is used to create deployment packages and in many cases is used to deploy code that requires full trust. SharePoint Solution packages will continue to be used by third-party vendors or developed internally with such tools as Visual Studio. You can still create Sandbox Solutions which run in a more secure runtime and can be deployed in SharePoint Online.

Now the new Apps framework for SharePoint 2013 is a packaged up in a file called .APP. It is composed of many of the same types of files, AppManifest, embedded Solution.wsp, etc. Once an app is loaded into SharePoint, it is accessible through the App Catalog in SharePoint. This App Catalog can be controlled at an organizational level.

Remember the big point with Apps is, that the custom solution you are writing may or may not actually run in SharePoint. Full trust code is not supported. Your custom solution code itself may run in a different SharePoint farm, on an IIS server as ASP.net pages, ASP.net pages running in Windows Azure, etc. So how does SharePoint access these solutions running outside the SharePoint context? In simple terms we have an IFrame (with some extensions) that external solution is available through. OAuth provides the secure connection for access SharePoint objects from a remote location. We will additionally use a new extended and robust event model and remote client SharePoint library to write integrated, remote code.

Why is this so great? We are going to ensure that custom applications and solutions that are being developed with SharePoint are isolated. No more writing a bunch code and services that should not be running in SharePoint servers. It is great that you can do whatever you want with SharePoint, however this will drive solution management.

So you may be asking where does this get deployed? There are many different options for hosting.

• SharePoint Hosted – This means the app and all the resources run in SharePoint. Remember you server side is not supported however you can write applications with SharePoint’s JavaScript libraries and such.
• Windows Azure Autohosted – This is a model that is only supported in SharePoint Online. In This case you can write an App package that will have code for Azure and SQL Azure embedded into it. When the application is deployed, the azure solution will be automatically deployed for you. You do not have to go to Azure and set anything up at all; it is all handled for you behind the scenes.
• Provider-Hosted – This is the third model where custom code and solutions are hosted in a separate server in your organization, hosted in Azure, hosted in different SharePoint servers, etc.

Once an App package is installed, it can be managed and monitored through the catalog. End users have the ability to select an app to run in their sites (much in the same way as turning on a Feature). If and when an app is updated, the user can decide how they want to upgrade to the new app.

Again I really plan to go much deeper on this in my blog but for right now, these are just some introduction notes and ramblings on how excited I am about this new capability J

Back Up and Recovery Diagram

There is a new diagram that goes into the details of doing your own back-up and recovery for SharePoint 2013. I know many people have become accustomed to using third-party vendors for supporting these operations and I still believe these vendors will continue to provide features above and beyond what is out of the box. However, if you are a do it yourself sort of person, this is a great diagram to review.

Not much has changed in regards to developing back-up procedures for both the SQL Servers and the SharePoint Servers. There are tons of scenarios covered in here, and I recommend reading this if it is important to you.

SharePoint 2013 Database Diagram

Finally the database server diagram has been updated. This is a really really really important diagram to review if you are managing on-premise servers. It goes over all the SharePoint databases, plus provide sizing and scaling guidance. Great information.