Saturday, May 15, 2010

New SharePoint 2010 Features Part 2 – Enterprise Data Access and Management Features

Enterprise Data Access and Management Features

In this section I will discuss many of the new features associated to working with enterprise data with SharePoint 2010. SharePoint 2010 is much more compelling with than SharePoint 2007 with more robust out of the box functionality that allows you to integrate with enterprise data.

Business Connectivity Services (BCS)

Business Connectivity Services (BCS) is probably one of the top five most important new releases for SharePoint 2010. BCS is the enhancement to SharePoint 2007's Business Data Catalog (BDC) which was used to provide read-only access to Line of Business (LOB) data across the enterprise into SharePoint 2007. Note if you see BDC mentioned in the context of SharePoint 2010, it now means Business Data Connectivity.

BCS for SharePoint 2010 supports the follow:

  • Supports full Create, Read, Update, and Delete (CRUD). SharePoint 2007's BDC just supported only read operations.
  • Supports integration with databases and web services and additionally WCF, DLLS and custom domain services.
  • Provides an extensible model provider which provides a pluggable framework for which developers can write to other data source types that are not provided out of the box with BCS.
  • BCS supports batch processing for read operations unlike its predecessor which could only support processing a single item at a time.
  • Can now work with BLOB data type (i.e. streaming BLOBs of data from external systems).
  • It can support read/write of complex data types and not just primitive types.
  • Rich API that can be used by custom applications to use data services provided by BCS.
  • Tools like SharePoint Designer 2010 can be used to build solutions using a declarative model which does not require a developer to write code.
  • Users can effectively consume and work with data from multiple external data sources in a user interface that they are familiar with such as SharePoint lists, web parts, Outlook contacts/tasks/appointments, MS Word documents, etc.
  • BCS has a robust cache mechanism for supporting users to work with data provide by BCS offline. Solutions written in correlation with MS Outlook 2010 and SharePoint Workspace 2010 and work with data while offline and then be synchronized when back online.
  • BCS can be integrated with SharePoint Enterprise Search to allow users to search external SharePoint data.

Business Connectivity Service Thoughts

I am really excited to see BCS leveraged as a solution to build composite applications that expose enterprise data instead of building expensive, custom applications. Typically the cost of building these custom composite applications come in the form of heavy design, development and maintenance. In many instances the demise of these custom applications is that a significant investment is made in engineering of the solution but the flexibility and scalability does not meet the future business demand. BCS removes some of these challenges in that it is a framework provided by SharePoint to expose enterprise data so organizations, so they can focus on solving existing and reacting to new business requirements instead of trying to build an enterprise data access framework.

On a side note - I also believe that SharePoint 2007's BDC did exactly what it was supposed to do. SharePoint 2007's BDC was originally intended to be an enterprise search solution that could expose data from web services or databases in the result search of a search. However this was confused when BDC web parts were created to expose enterprise data. When these web parts were utilized business users typically asked to perform full CRUD operations (FYI, it was actually possible to support CRUD with SharePoint 2007 BDC but it was not great solution). Microsoft stepped up to the plate and solved both problems. First they acquired FAST and integrated it in the SharePoint solution architecture and we now have a true best in class enterprise search solution to work with which SharePont 2007's BDC cannot even compete with. Second they created the Business Connectivity Services which can support full CRUD operations which is fully integrated with the SharePoint Foundation.

Enterprise Search (Reference)

SharePoint 2010 includes significant improvements in regards to enterprise search. The biggest is Microsoft's acquisition of FAST (one of the top three enterprise solutions in the marketplace) which has subsequently been integrated into SharePoint 2010. FAST is an extremely rich and deep application that should be dived into separately. Basically you now have two options SharePoint 2010 Search and FAST for SharePoint 2010.

Let's first understand at a glance what in general the improvements to search for SharePoint 2010 are:

  • Better Keyword Query Syntax has been incorporated into SharePoint 2010 Search. Such new features as Boolean, proximity, synonym, wildcard operators, prefix matching for free-text, property restriction queries and property operators for non-text fields have been incorporated. What this basically means is creating complex queries to run against the search service is much easier in SharePoint 2010. Note that the new Keyword Query Syntax is supported for both SharePoint 2010 Search and FAST for SharePoint 2010.
  • The search web parts are unified so they support both SharePoint 2010 Search and FAST for SharePoint 2010. This will ensure there is some consistency in configuration, management and user experience regardless of which search engine you use. Note that the search web parts are built upon the federated search object model.
  • The search results page new includes a new refinement panel that gives the user a summary about the search results and provides them the ability to quickly filter results based on the taxonomy of the data.
  • New connector frameworks for both SharePoint 2010 Search and FAST for SharePoint 2010 have been incorporated to support indexing data external to SharePoint. SharePoint 2010 Search utilizes the new Business Connectivity Services (BCS) while FAST has its own highly scalable connector framework. Both of the connector frameworks are significantly better than the BDC used for SharePoint 2007.
  • Relevance has been enhanced for SharePoint 2010 Search while FAST provides an extremely rich and advanced relevance model. For SharePoint 2010 Search, we now have the ability to create customized ranking models to improve result relevancy by collection, content source and data verticals.
  • Federated Search is again supported by both search options for SharePoint 2010. If you are not familiar with it, what Federated Search supports is the ability to display search results that were crawled by other search services such as Bing, Google, etc.

Enterprise Search Thoughts

I personally believe that FAST for SharePoint 2010 is a game changer and one of the compelling reasons why organizations should upgrade as soon as possible. The features and functionality of it are unparalleled and it is a mature product that has been injected into SharePoint. Content and data stored in or out of SharePoint will be easily accessible utilizing highly advanced relevance models. To learn more about Enterprise Search and FAST in general to understand the value proposition of it, please read this blog series.

List Enhancements (Reference)

There are several new features that have been introduced into SharePoint 2010 to support their initiative to better handler larger quantities of data:

  • List throttling – To ensure that open ended queries for list data to not effect performance of the site for other users. Thresholds can be set to ensure web application level.
  • Referential Integrity – In previous versions of SharePoint it was possible to use lookup fields or custom events to enforce referential integrity betweens items between lists. With SharePoint 2010, referential integrity can be set up directly between two lists and SharePoint will manage it from there forth.
  • Enforcing Uniqueness in Column Values – There is now the ability to select which column in a list will define uniqueness. This does not have to be the internal ID column of the list.

List Enhancements Thoughts

The ability to efficiently and effectively handle list with large amounts of data is a very important new feature to SharePoint 2010. In SharePoint 2007 it was possible to create efficient solutions that had SharePoint lists with more than 2,000 items, however complete custom code solutions had to be written. I still have some hesitancy to make sure that data management be well thought out before dumping extremely large amounts of data into a SharePoint list (my old blog entry). If data is highly relational or must be reported on, it is probably best to use SQL Server and then expose the data to SharePoint using the new Business Connectivity Services.

Still the new ability to manage very large quantities of data is not just for SharePoint Lists, it is also for managing large values of documents in a document library. It is very exciting to see this and will be very interesting to see how SharePoint lists evolve as a new enterprise resource for data storage.

Microsoft Sync Framework and SharePoint Workspace 2010 (Reference)

Microsoft Sync Framework is a unified synchronization architecture that allows for bidirectional communications between SharePoint and Line of Business systems. This framework as a specific focus on synchronizing SharePoint data stored in either lists or document libraries. This architecture is used as the backbone for SharePoint Workspace 2010 which is the replacement for Microsoft Groove 2007. What this tool supports is the ability to work with data stored within SharePoint in a disconnected environment. This includes lists, document libraries and even data within BCS.

Microsoft Sync Framework Thoughts

This again is another very exciting release for SharePoint 2010 and provides a tool that no CMS competitors are providing. Users now can work while travel, we can ensure they have the latest information locally, etc. Groove was an initial release given this is Microsoft's second release of this technology I have high expectations that it will be a very effective tool.

User Profiles and Social Data (Reference)

There are several aspects to the new features that have been integrated into SharePoint 2010 to support social computing. There are new APIs discussed in this article about activity feeds, social data and user profiles. Activity Feeds API supports the sharing of data between users. Social Data is a new API that has been created to facilitate social tagging, rating and comments for content that is stored within SharePoint. The User Profiles API has been augmented to support both organization and user profiles.

The new social computing features of SharePoint 2010 support scenarios such as (this is not an entire list):

  • The ability to find commonalities between two user profiles.
  • Providing the ability for users modify profile data
  • Adding links to your profile.
  • Adding colleagues to your profile.
  • Social Data Statistics Web Part that displays social statistics such as each URL that has been tagged and their terms, the number of times that term has been used in a tag, each user who has added a social tag.
  • The ability to approve who is marking you as a colleague.
  • Ability to send content to colleagues.
  • Ability to pull in external social computing data like from LinkedIn.
  • Ability to create organizational charts based on enterprise data.
  • Ability to rank, tag and comment content within SharePoint and use that information to the most relevant information to users based on the data's actual usefulness.

User Profiles and Social Data Thoughts

At first glance this does not really seem like much however this new foundation provides a foundation for building social computing business solutions that was significantly lacking in SharePoint 2007. I cannot tell you how many clients wanted to incorporate social computing into their Internet, Extranets and Intranets. Common use cases were who is the SME for what I need and who do I contact them? What content is considered to have the best information? And so on. To support this in SharePoint 2010 typically highly customize solutions had to be created.

I know when I first started doing SharePoint 2007 projects I was little skeptical of social computing and how it applied to the business world. However client after client I saw that this was significantly important to companies and I even saw how it would be important to me at my company. Today I rely on the fact that I have been at a company for several years, I know everyone, I have seen people come and go, I have seen lots of projects and I know where to go find something or who I need to call to find it. But if I am new, how do I find what I need? We have talked about search, but that is only one piece of the puzzle. There is still an element of finding information which cannot be gleaned by a search engine. The data indexed is only as good as the metadata that supports it, and if there is no metadata, it is not relevant.

Another scenario is very geographically dispersed or remote environment can highly benefit from the adoption of business social computing. We are so used be able to crowd around the water-cooler, well we can now do that with SharePoint. With the new Social computing features we can find out who is doing what and when and if you should chime in with some information that stop that person from potentially making a costly business decision.

Now we are empowered with SharePoint 2010 to come up with these new business social computing solutions and I am truly excited to see how this is used over the few months.

No comments: