InfoPath Forms for SharePoint Server 2016

If you have read my blog over the years, you will know that I was heavy into InfoPath. There was a small update to this InfoPath announcement made last year - http://blogs.office.com/2014/01/31/update-on-infopath-and-sharepoint-forms/. SharePoint Server 2016 will include InfoPath Forms Server 2013 on-premises. This is just a reminder that if you have InfoPath dependencies, and you want to upgrade to the next version of SharePoint on-premises, InfoPath is supported on SharePoint Server 2016. Additional there is continued support of InfoPath on Office 365. The lifecycle support for InfoPath has not changed. Please review the referenced blog for all the details.

InfoPath Being Retired

Announcement: Well there was a big announcement in the SharePoint world that hit me personally – InfoPath is being retired - http://blogs.office.com/2014/01/31/update-on-infopath-and-sharepoint-forms/

Reflection: I had been waiting for public announcement and it was pretty obvious it was coming down when we saw that no new additions or investments (of any importance) we made into InfoPath 2013 and InfoPath Form Services.

Now InfoPath has a special place in my heart and I will say that my love for InfoPath put me where I am today. Really. When I first started at RDA Corp back in 2005 I was just a .NET engineer. My first project I was thrown into an engagement where I had to learn InfoPath 2003, K2.net 2003, BizTalk 2004 and SharePoint 2003 in a week and make them work. That project change my life. I built this solution (with some smart people) which eventually was the finalist for solution of the year at the Microsoft World Wide Partner conference. It had tons of InfoPath components which were hooked on top of a VAX mainframe. I subsequently got hooked into the developer community in ways I had not done before. I starting doing more and more InfoPath and K2 projects and then I got really hooked into SharePoint; the rest is history. I was able to get a job as a SharePoint TSP at Microsoft which has transcended into a career doing Office 365. I can say, it was InfoPath that sparked my love for the Microsoft productivity stack to build business solutions. I started saying “why do I need to build solutions from scratch when I can take all the Microsoft solutions together and build something”.

Over the years on this blog I had wrote extensively on how to do development, best practices and build solutions for InfoPath - http://www.astaticstate.com/search/label/InfoPath. My first blogs in September 2007 were focused on InfoPath. I thought it was a great solution with SharePoint to build business forms and automate them using simple or complex business processes. Reflecting back, I found that InfoPath did a lot of great things for standard to moderately complex forms but when you brush up against the edges (which is what I had to do a lot as a consultant) you found tough things you would be able to resolve with a custom web form. I would always be put into these debates with colleagues on this topic.

Where Do We Go From Here: I believe Microsoft product team is making the right decision. InfoPath had its day, it is a great concept behind it to create forms quickly, extract the XML, shred it and sent it to a database. However today there are so many development frameworks for rapid development and Microsoft is going to continue to create new solutions that will help customers build forms and process automated solutions. Support for InfoPath is not going away anytime soon, they are keeping support through April 2023 (not a typo) and I still say InfoPath is a great solution, just nothing new is being introduced. They noted they plan to create some migration scenarios in Q4 of CY2014, so I will be keeping an eye out for that.

InfoPath 2010 and Digital Signatures

I have had several questions about InfoPath 2010 and support for Digital Signatures. The answer is yes, InfoPath 2010 supports them and there is even more improved capability. Here are some good references.

• This provides good general information about InfoPath Digital Signatures - http://office.microsoft.com/en-us/infopath-help/digital-signatures-in-infopath-2010-HA010381868.aspx
• This provides information describing in more detail the improvements with SharePoint and InfoPath 2010 - http://blogs.msdn.com/b/infopath/archive/2010/02/18/digital-signature-support-in-infopath-2010.aspx
• This is a good article about the event model around digital signatures and how you create customizations around those event - http://msdn.microsoft.com/en-us/library/aa948780.aspx

Create InfoPath Form Instance through Workflow

Issue

Have you ever wanted to have an InfoPath form generated by a workflow? Well I thought it would be a simple task but I had to do some extra things to make it happen correctly.

I recently wanted to be able to generate an InfoPath form based on an event on a SharePoint Designer workflow. So I published my InfoPath form to a Form Library, added an action to create a list item and then ran the workflow. I believed this would work because I have created Word templates, associated them to content types and the word file would be created just fine with the appropriate template. The net result in this situation does not work. Instead the InfoPath form template (.xsn file) is created as a file instance. The InfoPath xml that I had expected was not there.

Solution

I found this article (http://www.bizsupportonline.net/blog/2009/06/create-infopath-forms-sharepoint-designer-workflow/) which I reviewed. My solution is a little bit different because I was building this solution in SharePoint Online in Office365 and I was not able to implement this exact solution. However I was given some clues and came up with the following.

My solution was to:

• Publish the InfoPath form the same way I did before.
• Then I created an instance of the InfoPath form xml and saved it to my desktop.
• Then went to the form library configuration, advanced settings and turned on “Allow management of content types”.
• Then clicked on the default Form Content Type.
• Then clicked on Advanced Settings for the Form Content.
• Then I selected Upload a new document template and selected xml template I saved locally.

The following is what I had.

The InfoPath form template (.xsn file) is still there behind the scenes and the form will render correctly through InfoPath form services.

There are a few limitations of this solution:

• The user will not be able to create a new InfoPath form instance through the browser using the New Document button in the ribbon. What will happen is the InfoPath client will launched. This is not such a big deal because in my situation InfoPath forms will always be created through the workflow.
• If the InfoPath form changes, you will have to publish the InfoPath form template (.xsn file) and then go through these steps again.

SharePoint Designer 2010 Workflow and InfoPath 2010 Notes

I was recently building up some SharePoint 2010 functionality and ran into a couple small things I would like to share.

Document Sets and InfoPath Forms

There is a new cool feature of SharePoint 2010 that allows you to modify the Update form of a SharePoint list with an InfoPath form. This is really powerful because you can now push in custom logic around the data in your lists. As well, there are Document Sets which are basically folders that can be treated like documents or list items. In SharePoint 2007 a folder was a folder. But now with SharePoint 2010 Document Sets you can create content types of this type, add on metadata, associate workflow, add security, etc. However there is a limitation of both:

· Document Sets only work in Document Libraries

· InfoPath List forms only work in SharePoint Lists.

So if you ever want to do a cross-section of both features it is currently not possible.

SharePoint Designer 2010 Multiple Person/Group Picker In InfoPath Task Form

I created a workflow in SharePoint Designer 2010. In one of the task forms I wanted to allow the user to enter in a list of people who will perform the next action in the workflow. I did not want the data to be part of the list item because I wanted the manager to determine who would these tasks and not the person who is entering the original list item. What I found out was the InfoPath form would allow me to select multiple users to perform the next step but all of the values would be lost except for the first one once I hit submit on the form. Here is a reference.

http://social.technet.microsoft.com/Forums/en/sharepoint2010customization/thread/e720a3e2-e33b-42d4-ad3b-40696172fcd1

This was brought up but apparently not fixed before RTM.

Workflow Coercion Exception

You may see the following exception in your SharePoint Designer 2010 workflows:

SharePoint Exception: Coercion Failed: Input cannot be null for this coercion

This exception is not that helpful but this is typically associated to when you try to set a value to an item field or internal variable of your workflow. I have seen the issue on choice fields, people pickers, etc. Instead of selecting Return field as “XXX, Comma Delimited” select “As String”. This will resolve the issue.

Visio Must be installed to Publish Workflow with Visualization

If you are creating a SharePoint Designer 2010 workflow and the checkbox to enable Visualization is disabled you need to have Visio 2010 installed on the machine where you are doing the workflow publication. This is because the Visio visualization is built when the process is published, and it NOT based on the export .vwi file. This is why the workflow visualization looks different between the Visio diagram and the work you see on the workflow status page in SharePoint for the running workflow instance.

SharePoint 2010 Capacity Planning

• SharePoint 2010 Service Architecture
• SharePoint 2010 Logical Architecture
• SharePoint 2010 Architecture and Design Models
• SharePoint 2010 Performance and Capacity Case Studies
• SharePoint 2010 Capacity Planning
• SharePoint 2010 Cache Strategies
• SharePoint 2010 Databases
• SharePoint 2010 Physical Topology
• Simple Real World Examples

Introduction

One of the most important things for planning, managing and governing a SharePoint 2010 environment is to understand the boundaries and thresholds that affect performance and maintenance. Capacity Planning is extremely important when architecting a new SharePoint 2010 environment. A lot of times people designing SharePoint environments or creating SharePoint solutions completely forget about how Capacity Planning is directly tied to use cases and business scenarios. Knowing where your limits are and how you plan to grow will affect both your physical topology and logical topology.

For SharePoint 2007 we had a Capacity Planning tool (old blog I wrote). The old tool did its job at the time but there is no new one as of yet. However Microsoft has actually put out a ton of information that you can use for SharePoint 2010 Capacity Planning.

In this blog I will touch on:

• Web Site and Site Collection Capacity Planning
  
• Content Database Capacity Planning
  
• List and Library Capacity Planning
  
• Search Capacity Planning
  
• Security Capacity Planning
  
• Social Networking Capacity Planning
  
• User Profile Service and Social Networking Capacity Planning
  
• Web Analytics Capacity Planning
  
• Workflow Capacity Planning
  
• Excel Services Capacity Planning
  
• PerformancePoint Service Capacity Planning
  
• InfoPath Services Capacity Planning
  
• Visio Services Capacity Planning
  
• Word Automation Services Capacity Planning
  
• Office Web Application Services Capacity Planning
  
• Access Services Capacity Planning
  

Resources

The following are the resources that I used for this analysis

• SharePoint 2010 Capacity Planning (Boundaries and Limits) Whitepaper (View or Download). This is a really good whitepaper which should be used as a baseline for architecting your SharePoint 2010 environments. I know there will be creative solutions over the long run that will allow you to exceed these however this is the best place to start. I actually enjoyed reading through this.
  
• SharePoint Server 2010 performance and capacity test results and recommendations (View or Download). This is a set of several detailed whitepapers that go into the nuts and bolts of how SharePoint 2010 was tested and provides tons of very detailed recommendations.
  

All this information in this blog comes from these whitepapers and these are basically my notes.

Since these are just notes – I have not spent any time on grammar J

Web Application Limits

• Try to limit to 300 content databases per web application. This is an important one because a well managed SharePoint environment will not have one big content database; but instead have many to facilitate back-up and recovery procedures. If you plan to have lots of content databases, the recommendation is to use Powershell to manage the web application instead of using the management interface.
  
• Zones have a hard limit of five (default, intranet, extranet, internet and custom); nothing new there.
  
• It is recommended that you do not exceed 10 managed paths per web application without testing.
  

Thoughts: Not much new here other than the most important one everyone should know – do not have one big massive content database for your web application.

Web Server and Application Pools

• It is recommended to have no more than 10 Application Pools per web server. This is mostly driven by amount of RAM and usage by the users.
  

Content Database Limits

• Content databases should not exceed 200GB of data. Again SharePoint 2010 does scale up to 1 TB but that is only recommended for large, single site repositories.
  
• Remote BLOB Storage (RBS) cannot exceed 20 milliseconds to access the first byte of data.
  

Thoughts: I can say this over and over but you need to plan to keep your content database sizes down and do not let them exceed recommended limits. RBS is good for performance but you still think about not letting your content databases store TBs of data. That is a business requirements issue.

Site Collection Limits

• It is recommended to not exceed 250,000 sites per Site Collection. Basically the deal is that there is performance overhead when both creating and deleting sites. If you have a highly dynamic SharePoint web site, where this many sites are created and deleted on a regular basis, you will experience performance issues across the board.
  
• Site Collections should not exceed more than 100 GB. Again the issue is to make backup and restore procedures run quickly.
  

Thoughts: If this becomes an issue you probably have too many people with rights to create sites. Another way I can see this happening is if you create some sort of automated site provisioning process that just creates too many sites. Your SharePoint Topology should plan for not exceeding these limits.

SQL Service Column Limits

• As we mentioned before, we need to limit the number of row wrapping to 6 rows. According to SQL Server the threshold is 64 columns per row, so you should not exceed more than 384 columns in your SharePoint list or library definition.
  
• As mentioned before, the amount data stored SharePoint item per row cannot be more that 8,000 bytes.
  
• This whitepaper goes into specifics the size limits by SharePoint column type.
  

Thoughts: I had never really thought about this in the past. It will become a more important factor in your SharePoint design given the amount that can now be stored in SharePoint.

Web Page Limits

• It is not recommended to exceed more than 25 web parts per SharePoint web page.
  

Thoughts: If you have than many on your screen you are trying to display too much or your have made your web parts way too granular.

List and Library Limits

Understanding SharePoint list and library capacity is extremely important, especially with the new improvements to SharePoint 2010.

Here are some specifics:

• For each row in a SharePoint List or Document Library List Item you have a hard maximum of 8,000 bytes. What this basically means is the amount of data stored in the columns of a list item cannot total more than 8,000 bytes.
  
• The maximum File Size is 2GB. The default is 50 MB.
  
• It is recommended to not exceed 30,000,000 documents per document library.
  
• It is recommended to not exceed 30,000,000 items per SharePoint list.
  
• Should not exceed more than 6 table rows in the Content database to store a list or document library item. What this basically means is that if your list or library definition has lots of columns, SharePoint will break that storage of that data across rows. There is fixed amount of columns in a SQL table and if there more list columns that SQL table columns the data for the list item will be stored across rows (referred to as row wrapping). So this will only become an issue for you if you have an extreme amount of columns in your SharePoint list definitions.
  
• The SharePoint list user interface only allows 100 items to be processed in bulk at any one time.
  
• SharePoint List queries have a maximum of 8 joins allowed per query. If there are more than 8 joins in the query, it will be blocked.
  
• List view threshold by default will show 5,000 items. Going beyond this you will experience poor performing queries and overall throughput will decrease. List view threshold for auditors/administrators is 20,000 items.
  
• It is not recommended to exceed 2,000 subsites for each website. Performance will degrade on several of the out of the box pages, controls and web parts when 2,000 subsites are exceeded in a single website.
  
• It is recommended to not exceed 10 users simultaneously editing the same document. If there are more than 10, you will receive performance issues when committing the document to SharePoint. The hard boundary is 99 users.
  
• When extracting datasheet view into Excel, there is a hard limit of 50,000 items that can be exported.
  
• A SharePoint Workspace has a hard limit of not containing more than 30,000 items for synchronization.
  

Now are some other recommendations that you should be made aware of:

• A basic way for calculating the potential size of the list is the following. First estimate the average size of a document to be stored. Multiply that against the number of versions of that document. They multiple that by 20% for the data stored associated to the content database. It is also recommended to add in an appropriate buffer.
  
• When designing how content will be stored in lists, you need to consider if storing data in a single list, across multiple lists or even across lists in different site collections. There is a detailed discussion of this in "Single list, multiple lists, or multiple site collections" section in DesigningLargeListsMaximizingListPerformance.docx.
  
• Even through list and libraries can store large amount of content, we still need to get it back out. Microsoft recommends that when retrieving data from large lists using lists views or CAML queries, they be partitioned across folders, indexes or both. If not, the best and most efficient way to retrieve items is through search.
  
• Lots of permissions applied to a library will degrade performance. It is worse if you are doing item level permissions on large amounts of documents. There is a configuration will limit 50,000 unique permissions to a list. However it is recommended to lower this to 5,000.
  
• I mentioned row wrapping earlier and why it is used. It is recommended on for lists with large amount of data, to not exceed 1 or 2 additional rows of wrapping.
  
• Lookup columns can again be another source for performance issues. Lookup columns in a list view will result in a join being applied to the query which increases the complexity. By default each list view will support up to 8 lookup columns. It is strongly recommended to not exceed this number as it will cause significant throughput decrease for queries in a view and use lots of SQL resources.
  
• List indexes have little effect to list performance but can insert and update operations if there are lots of them. Up to 20 indexes can be applied per list. It is recommended to only use indexes when they are needed.
  
• There are three basic ways to access list data: list views, content query web parts and search. List views access data directly off SQL Server which incur slower query performance and higher load on the SQL Server. They also generate the most HTML which can slow down page rendering. However list views provide the best user experience when it comes to managing data in a list. Content query web parts display a configured view of data that is cached in the portal site map provider. It will generate the least amount of HTML and is cached, so overall performance will be better. Search web parts offer the best performance as it is optimized for queries large amounts of data. However the data returned in search is only as recent as the last crawl.
  
• Microsoft has several types of lists they have identified and when designing your SharePoint site, it may be good to classify in this manner so you can plan and govern better of the long run.
  
• There are unstructured document libraries which are basically team libraries with tens to hundreds of documents. They are highly utilized with expensive operations but the volume is low. It is important to make sure they libraries stay under 5,000 items.
  
• There are collaborative large lists which can have thousands of documents that can be edited, like in a knowledge management solution. Typically these lists grow significantly faster than anticipated and requiring lots of administration.
  
• There are structured large repositories which may have thousands to hundreds of thousands of documents. Typically these are departmental archives with automated processes to publish the documentation in a controlled manner. Almost all of the interactions with the list are read only.
  
• Finally there are large scale archives which contain millions of documents spread across multiple lists. Typically there are is a low amount of reads and updates and the purpose is for long term storage for compliance requirements.
  
• Content Organizers are a new feature that will route content to configured libraries, folders or sites. Users can submit data and not be concerned about the rules for storing content. It can support evenly storing data within a library to better manage performance.
  
• Metadata Navigation is a new feature that allows users to search for content in a tree based on the data associated to the document. This feature will use the most efficient query possible to return items instead of allowing the user to freely search and filter for content.
  
• Throttling of SharePoint list is also new. This can be configured at the web application so that inefficient operations by a user will not affect performance of the entire farm.
  
• Another new feature is compound indexes so indexes can be built across multiple columns instead of just one.
  
• The developer dashboard is another new feature that can help when understanding potential performance issues with a page. When it is turned on, tons of statistics are available, including database queries and load times.
  
• Allow object model is a new configuration that allows for object model to override the list view thresholds. This allows for developers to create web parts that query for content that do not adhere to list view thresholds, which is a good thing.
  
• Daily time window is another new configuration that when turned on, allows queries to be run that do not adhere to list view thresholds.
  
• You need to be aware that if the list will grow very large and exceed thresholds, several operations will be blocked, prevented or limited. For the details read the "Differences between large lists and regular lists" section in DesigningLargeListsMaximizingListPerformance.docx.
  
• Microsoft testing of extremely large document libraries (millions of documents) generally concluded that most of the bottlenecks would occur at the SQL Server level. To get around this bottleneck you should split content across multiple instances of SQL Server.
  

Thoughts: I am a broken record on this topic. Yes SharePoint 2010 resolved the challenges of storing mass amounts of data in SharePoint lists. However I still believe quantities of list data, especially "highly relational data", should be stored in a relational database and then exposed to SharePoint through the appropriate means.

Security Limits

• It is recommended to not exceed having a user belong to more than 5,000 SharePoint groups. There are no severe performance penalties if you go over, just a recommendation. If you do go over the user's token is bigger, it takes longer to cache access control list (ACLs) for search, and increases the amount of time check security.
  
• It is recommended to not exceed 2,000,000 users per site collection. Again you can exceed this number however there are long term management issues with that many users and it is recommended that you use PowerShell.
  
• It is recommended to not exceed 5,000 users or external groups (AD group) per SharePoint Group. The more users, the longer it takes to validate permissions or render a screen.
  
• It is not recommended to exceed more than 10,000 SharePoint Groups per Site Collection.
  
• It is not recommended to exceed 5,000 permission items in an access control list (ACL).
  

Thoughts: It is important to keep in mind for very large organizations and when planning to pull in data from external directory services (which may not always be Active Directory).

SharePoint Search Limits (not FAST)

When it comes to search, this is just such a big topic. I highly recommend reading all of "Estimate performance and capacity requirements for SharePoint Server 2010 Search" (SearchforSPServer2010CapacityPlanningDoc.docx). Understanding the search architecture is critical for sizing and capacity management. I am going to go over some high level concepts to get you started:

• First when planning you need to understand what data needs to be searchable and how much of it is there. You need to know how available the data must be. You need to know how fresh the data must be kept and many people will be searching for data simultaneously. This actually goes a lot deeper and I have blogged about scaling FAST 5.3 here. The concepts are the same.
  
• For SharePoint 2010 there is a search life-cycle that you need to understand. There is Index Acquisition which is where full crawls for data are performed which is dependent on the size and access to the content is being crawled. Next there is Index Maintenance which is incremental crawls of all content. Finally there is Index Cleanup is basically when content sources change or removed from the crawler.
  
• Next there is the query service which is responsible for querying for data. It is important to know both the query latency and query throughput. You want to decrease latency which is the amount of time it takes for data to become searchable. You want to increase throughput which is the amount of data which can be searched.
  
• There are really just so many strategies for configuration your SharePoint logical and physical topology based on your requirements. Here are some simple things to think about. First never run both the query and search services on the same machine. Second, if you have the hardware, it is also recommended to run your query service on a separate machine which will decrease latency and increase throughput. Third it is good to have multiple search servers which build the indexes so there is redundancy when there is a failure or if you need to dedicate resources to search specific content sources.
  
• There some calculations which you can do to estimate the amount of space you need to indexes that will be built. Again, read this whitepaper for further details.
  

Some general thresholds and boundaries are:

• It is recommended to not have more than 20 SharePoint search service applications deployed to a single farm.
  
• For each SharePoint search service, do not exceed 10 crawl databases which store crawl data. The optimal configuration is 4 crawl databases per Search service. Each crawl databases should not exceed 25 million items.
  
• It is recommended to not exceed 16 total crawl components per service application.
  
• It is recommended to not exceed 20 index partitions for a search service. The limit is 128 index partitions. Partitioning an index allows for smaller indexes to created which can be searched faster, however having too many can have opposite effects.
  
• It is recommended to not exceed 10 million items per search index. The overall recommended limit for all indexes in a search service is 100 million items.
  
• It is recommended to not exceed 100 million crawl log items per search service.
  
• For each search service, it is not recommended to exceed 10 property databases. The property database contains metadata for items in each index partition. There is a hard limit of 128 property databases.
  
• There is a hard limit of 128 query components per search application.
  
• For search scopes, you should not exceed 100 scope rules per scope. As well you should not exceed 600 scope rules per search service. Exceeding these thresholds will affect performance. As well, you should not exceed more than 200 scopes per web site, which again can affect performance.
  
• It is recommended to not exceed 25 display groups per site. Display groups are used for grouped display of scopes through the user interface. Exceeding this number will affect performance.
  
• For alerts, there is a limit of 1M per search application.
  
• For search content sources it is not recommended to exceed 50 data sources but the top threshold is 500 data sources.
  
• There is a threshold for running 20 concurrent crawls within the same search service; exceeding this will cause performance issues.
  
• During crawling, each search application will support up to 500,000 properties.
  
• Recommended not to exceed 100 crawl impact rules per farm.
  
• Recommended not to exceed 100 crawl rules per search service. This is because the administrative screen performance will degrade.
  
• There is a threshold of 100,000 managed properties per search services. Managed properties are used in search queries. Basically the crawled properties are mapped to the managed properties.
  
• It is not recommended to exceed 100 mappings per managed properties. Exceeding this will degrade crawl speed and query performance.
  
• Each search service will support 100 URL removals per operations.
  
• It is recommended to have only 1 top level authoritative page and minimize second and third level pages.
  
• It is recommended not to exceed 200 keywords per site collection but the maximum is 5,000 per site collection. The only real effect of exceeding the recommendation is performance of the administration page.
  
• There is a limit of 10,000 metadata properties that can be crawled per item crawled.
  

Thoughts: Microsoft has put a lot of emphasis into this. Even though FAST is a very compelling, the out of the box SharePoint search has been dramatically improved.

User Profile Service and Social Networking limits

• The user profiles service supports up to 2 million user profiles with full social networking features. This basically means this is the limit of profiles that can be imported into the people profiles repository.
  
• There is a limit of 500 million total social tags, notes and ratings in the social database. Exceeding this runs the risk of significant performance issues.
  
• Microsoft's testing found that there was linear increase in throughput up to 8 WFEs. Past that there was no improvement. Scaling can be further achieved when separating the content and services database onto two separate database services.
  
• Most of the bottlenecks that will be experienced are typically due to WFEs. They found no bottlenecks associated to any of the application servers when evaluating social networking features.
  
• It is expected that separate web site with dedicated resources is created for MySites and the new social networking features. When planning to support performance this is not just maintaining a user's profile information, connections, newsfeeds, etc. It is also storage of documentation, collaboration, etc.
  
• User Profile service does have caching. The user profile service job is to take data from whatever location where you are loading from, and it will save that data in SharePoint profile database. Once the user profile cache is loaded, WFEs request for profile data is returned from the application server without requiring a call to the SQL database. I guess Microsoft's strategy is that profile information is not really dynamic data and they want to keep transactions down.
  
• I am not going to go in the details of the Outlook Social Connector however there is very detailed information about the performance of this in the Social Computing Capacity Planning whitepaper.
  

Thoughts: Much of this is associated to the new social networking features of SharePoint 2010.

Business Connectivity Services limits

• There is a hard limit of 5,000 external content type definitions that can be loaded into memory at any given time.
  
• There is a limit of 500 external system connections that can either be active or open at any given time. The default is set to 200 and it does not matter what type of connection is being used either.
  
• The default is 2,000 database items that can be returned in a request when using the database connector. The boundary is 1 million.
  
• There are three variables that you must always consider: the number of items in the external list, the number of columns per item and the size of each item.
  
• Profile pages display data from external content type data. The performance of these pages is driven by the complexity of the associations to external systems.
  
• The internal process for bringing external data into SharePoint via BCS is pretty simple. There is load (queries external source and loads into SharePoint), process (applies sort, filter, group processing) and render (display data onto page). BCS does not have in-memory cache for external items. Data has to go through load, process and render each time an external list is refreshed. Knowing this you need to make sure you control the amount of data that is processed at any given time.
  
• Microsoft's recommendation is to keep the number of items to be processed as possible by reducing the amount of items returned from external systems. It is recommended to keep the number of returned between 100 to 500 rows and it is recommended to not exceed 2.000 rows. It is recommended to use filters to ensure you work within these guidelines. More can be returned if needed but this needs to be done by an administrator.
  
• Rendering external list can be intensive for both the WFE and the application server. It is recommended to keep the number of items being displayed at any time to around 30. Note that the number of items that are rendered is not the same amount of items that were processed. The number of items rendered is controlled by the external list view that is on a SharePoint page.
  
• It is recommended to reduce the number of columns from external list. Obviously a large number of columns will affect performance.
  
• When rendering it is recommended not to use large-sized columns in list views. Columns larger that 1KB should not be utilized in a view. However performance is more affected by the number of items and not the size. So it is recommended always try to keep the number of items lower for better results.
  
• When designing an external list which is using BCS, make sure the default view will be the view the user needs to see the most. If the user needs to sort or filter the view, that will require data go back through the load, process and render process.
  
• For a profile page, the number of associations is the key for good performance. It is recommended to not exceed two associations. There will be lower performance of both throughput and latency when large number of items in an association.
  
• Diagnostic logging of BCS can become a factor for performance and it is recommended to lower when not doing testing.
  
• Performance of the external system will have performance implications to BCS and you need to sure those systems perform well.
  

Thoughts: I personally think is an area that has not been explored enough. Understand that BCS is another layer that the data must pass through. In Microsoft's detailed testing results for BCS they only tested WCF Web Services using .NET data types and SQL Server 2008 databases. As we know in an enterprise architecture not everything will be SQL Server or web service based. Data resides in formats all over the organization. Performance will be driven by factors out of scope of this blog.

Web Analytics limits

• There are three basic categories of analytics – traffic, search and inventory. Reports can be aggregated at the site, site collection, and web application level. The web analytics service mostly utilizes the application servers and SQL server, so that is where most of the capacity planning needs to be.
  
• At a high-level this is how the data is gathered. On each web server usage data is gathered and there is a Usage time job then calls the Logging Web Service to submit the data. The data is stored in a staging database for seven days. The web analytics components clean and process the submitted data and then every 24 hours the data is aggregated from the staging database and written to a reporting database. The aggregated data is stored for 25 months (default).
  
• Performance of the logging web service is directly related to the number of application servers on the farm. However the performance of the web analytics components is dependent on the performance of the analytics and reporting databases.
  
• Other areas of performance challenges are number of queries run each day, number of unique users each day, total number of unique assets clicked each day and existing data size in the reporting database. Basically the point is, the more data and more interactive your SharePoint farm is, the more performance overhead there will be to process that data for reporting purposes.
  

Workflow limits

• Workflow throughput can be affected by numerous factors such as the number of users, the type of workflow, the complexity of the workflow, dependencies on external calls and frequency on the number of users accessing the workflow. Note that workflows that use data from the content database or are registered for lots of events will run slower. So if you reference a large SharePoint list or you call out to an external database obviously that will take a little more time.
  
• Microsoft's testing found that throughput for workflows topped out at three to four WFEs added.
  
• There are some administrative settings that can be set which will affect the performance of workflows.
  
• Postpone Threshold is the number of maximum workflows that can run concurrently against a single content database. Subsequent ones will be placed into a queue. The default max is 15 workflows. It is important to understand this is the number of workflows that can be in progress at one time and it not the total number workflows that can be in progress.
  
• Batch Size is an exception to the Postpone Threshold limit and can pick up larger amounts of workflow items from the queue. The default batch size is 100 and this can be set per service instance.
  
• Another configuration is the Timer Job Frequency which is the frequency that the workflow time service runs. This service is responsible for picking up items off the workflow queue based on the interval. The default is every 5 minutes.
  
• I found this interesting but you should consider how all three of these configurations are used together to affect workflow performance. Workflow throughput is impacted based on how quickly operations get out of the queue and then processed. In the example I read, if there are 1,000 work items in the queue, it will take the time job 10 runs to execute all of them (Batch Size), which would take close to 50 minutes to execute (Timer Job Frequency). Modifying the Batch Size would increase that at the expense of taking up lots of processing resources.
  
• Another thing to make sure you do not do is use the same list for workflow history and task lists. Lots of items will be written to these lists as time goes on. Allowing these lists to get very large will affect performance.
  
• To keep the task lists from not getting too large there is a workflow job (autoclean) that will delete workflow instances and associated tasks that have been in a completed state for greater than 60 days. It is recommended that if historical information is needed past that, to write the data to other locations as part of the workflow. Workflow history items are not part of this autoclean job and a script should be written to purge those items.
  
• Another maintenance trick I learned which can affect performance is removing workflow columns. If you have a list with like 50,000 items in it, and you remove the workflow status column will cause database operation proportional to each item in the list, which can flood your SharePoint environment with transactions. It is recommended to just turn off the workflow so no new instances can be created and may do the operation during off hours.
  
• Another performance consideration when building workflows is make sure you are not going to violate SharePoint's database locking for modifying data at the same time. You need to make sure users and/or other workflows do not try to access the same thing at the same time.
  

Thoughts: I found the discussion on how to configure the workflow service very interesting because I had really never thought about that when I had built workflows in MOSS 2007. I can see how you may want to configure this very differently based on the site where the workflow is running. They had a creative example that was discussed where if you want to dedicate running workflows to a specific machine in the topology, you could try to lower the postpone threshold to force all workflows onto the queue so the batch job would pick them up on a specific machine. You would have to set the batch size to be high so that it picks up the work items before another machine. If you want to make sure the processing is shared across all the machines, you could try increasing the postpone threshold so items never sit around for a long time.

Excel Services limits

• Excel Services performance is directly correlated to the size and complexity workbooks that are hosted within SharePoint.
  
• Excel Services is stateful meaning that workbooks will remain in memory between user interactions.
  
• Excel Services have most effect on both the application servers and the WFEs. Excel Services will not highly utilize SQL Server because the workbook is read as a binary blob and used within the Excel Service. Microsoft's finding was that bottlenecks only happened on either the application or WFE servers, which can be scaled out by upping CPU memory or adding new Excel Services machines to the topology.
  
• Within Central Admin there are several configurations which can be used to better manage performance.
  
• Maximum Private Bytes is a percentage value Excel Services can use of memory on the machine. If the machine is dedicated to Excel Services, you can up this percentage. This is one of the main bottlenecks identified by Microsoft.
  
• Memory Cache Threshold is another configuration available. Excel Services will cache unused objects in memory and by default it will use 90% of the Maximum Private Bytes. Increasing this number gives a better chance that the requested workbook would already be in memory when the user accesses it. However lowering this number will improve performance of other services if the application server is not just hosting Excel Services.
  
• Maximum Unused Object Age is a configuration that helps keep objects in memory as long as possible but you will want to again modify this if other services are hosted on the application server.
  
• There are also settings which are not global to Excel Services however they are specific to the trusted locations where Excel Services are being used. For instance there is max workbook size, max chart/image size, session timeout, volatile function cache lifetime and allow external data all of which can be used to better manager performance.
  

Thoughts: For SharePoint 2010 Excel Services I would say there is more anticipation for a more feature rich solution than what was available for SharePoint 2007. It is very hard in my opinion to come up with a one size fits all sort of assessment for performance when it comes to Excel Services. This is because each hosted workbook can be dramatically different than the other.

InfoPath Service limits

• There are numerous factors that affect the throughput of InfoPath Services, some of which are: number of users, type/complexity/frequency of user operations, number of postbacks per operation, and performance of data connections. When building and deploying forms, this should always be accounted for.
  
• In the design of the form, try to optimize the first request to the form template. Try to exclude custom onLoad events, queries or business logic that occur when opening the form. You should try to make these operations as user driven as possible. What we want to do is delay the creation of session state data in the database until an actual post occurs. If the form only have one post, meaning when it is submitted, no session state for the form will ever be created.
  
• If the InfoPath form needs to be saved to a document library, it is better to submit the form into the document library rather than saving it. The reason is a submit operation triggers only one post but a Save will trigger two posts.
  
• Document library forms can support more throughput that InfoPath list forms.
  
• Form complexity such as the number of controls, rules, encapsulated data, etc. dramatically affects performance. The more complex, the more pressure will be put on the WFEs to support greater throughput.
  
• It is recommended to minimize the amount of controls displayed at any given time. If you do have a lot of controls that have to be displayed, it is recommended to put them into a secondary view so that you reduce the first page loading we talked about earlier. This is not just to reduce the amount of HTML generated; there can be a lot of presentation rules which can slow performance down.
  
• Large amounts of complex XML can cause more performance issues. Be careful of using attachment controls as the binary of the attachment will actually be embedded into the XML of the InfoPath form, making it very large.
  
• Database locks prevent multiple users from making changes to the same set of data. Since an InfoPath form is more than a piece of data in SharePoint, a lock is placed on it while it is edited. So it will not support concurrent usage of the same form well.
  
• It is recommended to return filtered data directly to the InfoPath form instead of filtering it within SharePoint.
  
• When reviewing the results of Microsoft's test, there was always more performance overhead when the form has web service submits.
  

Thoughts: From a performance perspective, not much has changed from a performance perspective. Yes - InfoPath 2010 form services will be faster than SharePoint 2007 – but the design concepts to create forms that perform well has not really changed. Your design of the forms drives performance and you should not lean on adding more WFEs to solve your problems.

Additional References:

• http://blogs.msdn.com/b/infopath/archive/2009/10/27/infopath-2010-performance-highlights.aspx
  
• http://blogs.msdn.com/b/infopath/archive/2008/05/09/designing-browser-enabled-forms-for-performance-in-infopath-forms-services.aspx
  

Visio Services limits

• There are three basic factors that affect performance: drawing size, complexity and data connectivity.
  
• Typically Visio Services are tough on the WFEs which will require you to scale up or out.
  
• The initial file size of Visio web drawings is 50MB. This can be changed by the administrator however larger files will increase memory required for Visio services, increase in CPU usage, reduction in service requests per second, increased latency, and increased SharePoint farm network load.
  
• There is a configuration called Visio Web drawing recalculation timeout which changes the maximum amount of time it will spend recalculating a drawing after a data refresh. The default it 120 seconds. Increasing the timeouts will increase CPU and memory, reduce application requests per second and increase latency across all documents.
  
• For Visio services there is a Minimum cache age setting which is used for data connected diagrams. This setting is between 0 to 24 hours. Setting this to a lower value will reduce throughput and increase latency because recalculations have to occur more often. This subsequently reduces CPU and memory.
  
• As well there is a Maximum cache age for non-data connected diagrams that determines how long to keep the diagram in memory. Increasing this value will decrease latency for diagrams that are used a lot. However increasing the maximum cache age will increase latency and slow down throughput for all items that have not been cached because the cached items consume available memory.
  

Thoughts: I found this section interesting to read because it gave me insight into how Visio Services is actually working. I had heard the Visio services can be data driven. When reading this you need to focus on caching as part of your design and understand the types of documents that you plan to post.

PerformancePoint Service limits

• The factors that most affect throughput are number of users, type of user interaction, frequency of use, number of postbacks in an operation and data connection performance.
  
• When a scorecard uses an Excel data sources, there is a limit 1 million cells per query.
  
• In a dashboard that is using Excel as a data source, the max number of columns is 15 and 60,000 rows.
  
• It is recommended that if you do have business logic or complex controls, the demand on WFEs will increase and adding more WFEs will alleviate those issues.
  

Additional Resource

• http://technet.microsoft.com/en-us/library/bb660539(office.12).aspx
  

Word Automation Services limits

• The input file size cannot exceed 512MB.
  
• The frequency of the conversions can be configured by minutes. A lower number will allow the timer job to run faster. The default is every 15 minutes but the recommendation is every 1 minute.
  
• There is a threshold for the number of conversions to start per conversion process. Obviously if the value is high, this can cause intermittent failures.
  
• The conversion job size can support 100,000 items at one time. A large number of conversion items increase the amount of time it takes to execute.
  
• The total number of conversion processes is directly related to the number of core processors. You should have not more conversion processors running than you have of processors. The reason why you should know this is because word automation services can at times full leverage a processing core and cause issues if other services are hosted on the application server. There is a property of word automation services called Total Active Conversion Processes which can be used to "throttle down" word automation services such that it does not use too many processing cores per application server.
  
• The work automation service database size cannot exceed 2 million conversion items. Items are not deleted from the database automatically after processing; so administrators will have to write jobs to remove the history. Exceeding 2 million items can cause performance challenges the longer the word automation service runs.
  
• When scaling out word automation services it is recommended to add more application servers and potentially add a new SQL Server which has the work automated services database dedicated to it. Another scaling recommendation would be to create dedicated word automation application servers.
  
• There is a fundamental performance limitation with converting PDF and XPS file types. Adding servers will not always solve the challenge.
  

Office Web Application Services limits

• This includes the Word Web App, PowerPoint Web App and OneNote Web App. There are several different perspectives that you have to look at each of these. Specifically you should focus on viewing versus editing because the resources required are quite different.
  
• The drivers for performance are expected concurrent users and what type of operations are going to be done. Microsoft's initial recommendations are 100 daily users with average of 10 concurrent can be supported by 1 WFE and 1 App Server. Going to 1000 daily and 30 concurrent would require 2 WFE and 2 App servers. Going to 10,000 daily and 300 concurrent would require 4 WFE and 3 app servers.
  
• Heavy Word Web App viewing will typically require more CPUs to the application servers.
  
• Heavy Word Web App editing and OneNote Web App viewing/editing require more CPU to the WFEs.
  
• Heavy PowerPoint Web App viewing requires more CPUs to the app servers and more memory to the WFEs.
  
• Heavy PowerPoint Web App editing requires more memory to be added to the application services.
  
• If you did not know what this is this feature enables presenters to broadcast a slide show from Microsoft PowerPoint 2010 to remote viewers over the web through SharePoint. It is positioned as a low-infrastructure presentation capability. Heaving PowerPoint Broadcast will require the WFEs to have more CPU. It is recommended that if you have extremely PowerPoint Broadcast feature to create an environment to support that. The reason being is web browsers will hit the server every second to get the latest changes to the slidedeck.
  
• Much of the limits associated to OneNote Services are directly tied to the limits for list and libraries. This is because each section in OneNote is stored as folders and documents in a library.
  
• The maximum size for each section of an OneNote section is again driven by the file size limits for lists and libraries.
  
• If there are embedded images, files, etc. in OneNote, that are greater than 100KB, they will be split out into their own binary files within the SharePoint library.
  

Access Services limits

• The performance of Access Services is dependent on the other applications that are hosted within the service. It would be recommended that Access Services to be dedicated to a service if there is lots of data to be managed.
  
• The amount of data in the tables and the size of the queries will have the most impact to performance of Access Services. It is recommended to limit the size and complexity of queries and well as control the amount of data that flows through. This can be done in Central admin. First the query can be configured by controlling the max sources per query and max records per table. Second results can be configured by max columns per query, max rows per query and allowing out joints. Third processing complexity can be configured by max calculation columns per query and max order by clauses per query.
  
• There is another configuration called All Non-Remotable Queries. Way Access Services work under the hood is that is augments SharePoint query operations support Access Service features, like working with large amounts of data. If there are performance challenges, you can change this configuration so that Access Services just uses the SharePoint query operations – subsequently making data fetches less complex (at the same time not as robust).
  
• Access Services is stateful. It will maintain in memory cursors and record sets between user interactions. Microsoft's testing did not determine that this would be a bottleneck.
  
• There are no special hard requirements for WFE or application servers to support Access Services. It is recommended to increase capacity by scaling up your existing servers or scale out by adding more servers into the SharePoint topology.
  
• To support more users it is suggested to add more CPUs to the servers and more Access Services computers if needed. Microsoft's testing found out that if you add too many Access Service machines the WFEs can become a bottleneck to the Access Services. This tends to happen when the four Access Service machines have been added.
  
• Reporting services are utilized with Access Services. It is recommended to create a dedicated machine to support it if processing of reports is taking a long time.
  
• There are several performance counters that can be used to determine performance that are in the whitepaper.
  

Thoughts: Again reading over this gave me some great initial insight into the architecture of Access Services. I actually intrigued to see how this will be utilized with organizations.

New SharePoint 2010 Features Part 5 – Office and BI Features

• Introduction
  
• Enterprise Data Access and Management Features
  
• Development Features
  
• Enterprise Content Management Features
  
• SharePoint Office and BI Features
  
• Administration Features
  

SharePoint Office and BI Features

In this part of my new features of SharePoint 2010 blog series, I am going to focus on Office and Business Intelligence features of SharePoint 2010.

Excel Services (Reference and Reference)

Excel Services was introduced in SharePoint 2007 to much fanfare. It provided a great initial release however there were several limitations to it which hinder the highly advanced Excel power users when they tried to place their Excel Spreadsheets into Excel Services. However there have been several new improvements that will make Excel Services for SharePoint 2010 even more compelling. Two of the biggest changes are first we can now edit Excel workbooks in Excel Services programmatically. Second users can now edit workbooks within SharePoint and edit the data directly in the browser.

Excel Services Thoughts

It is great to see Excel Services mature as it is the second version of this solution offering by Microsoft. What we are going to start seeing is a shift of the standard Microsoft Office desktop being available over the web. It will never be as fully robust as running Excel locally as a client but what this does enable is better communication of business data. There are so many businesses that have mission critical data stored within Excel with no way to distribute or control it in an efficient manner. I have heard the condescending remarks before by software consultants who say "how can you be using Excel instead of a custom application to manage such mission critical data". My response is, live the day of life in the corporate world where IT resources may not always available. In these many cases, business users (Information Workers) are forced to come up with solutions with readily available software and one of them is Excel. I am really glad to see how this is being used.

One thing I think is needed to ensure success of Excel Services within an organization is an effective governance strategy for rolling it out across the enterprise. This would include, providing the proper infrastructure, correct SharePoint logical architecture, information worker training, rules for deployment, maintenance, etc. I think the biggest thing is lack of education of business users that they have such a powerful solution available for them to expose their data.

Access Services (Reference and Reference)

The goal of access services is basic – to provide the ability to make Access database available over the web. Access has been a tool that helps enable non-technical business users the ability to create business applications. Now users can continue to create these applications with Access 2010 on the client and then publish them to SharePoint 2010. The full functionality of Access 2010 on the client is not available over the web through SharePoint as there is only a limited subset of full client object functionality.

Access Services Thoughts

I am really excited about this new feature and I cannot believe I am saying that. I say that because from a professional services perspective we have had a lot business where have moved Access solutions to .NET/SQL Server based solutions because Access applications could not scale.

I think the vision of this is great but since it is a first release and there is only a limited subset of functionality initially available it will suffer from the same issues Excel Services in SharePoint 2007 suffered from. The biggest issue is that Access developers are very advanced and require full features which will not be available right now. That notwithstanding, I can name more than two past SharePoint 2007 clients of mine who could use this even though it is lightweight. Their business users know MS Access and they do not mind functionality limitations if they maximize the investments they have made in their current staff and they build simple applications.

I see down the road this as another option to allow business to create codeless applications that solve tactical business challenges that companies face every day.

InfoPath Form Services (Reference and Reference)

In SharePoint 2007 we were given the ability to use InfoPath within the browser and we subsequently saw a lot of adoption of InfoPath to create data capture forms for business processes. SharePoint 2010 has expanded the features of InfoPath to include the following:

• Enhance SharePoint List Forms – This new capability allows you to very easily replace create, edit and view list item forms in a SharePoint List with a completely custom InfoPath form. This effectively allows you to add additional rules, load in data from other locations, incorporate business automation into lists, etc.
  
• One Click Publish – You no longer have to click through the entire publishing wizard after you publish the form one time.
  
• SharePoint Workspace 2010 – As part of this series I introduced SharePoint Workspace 2010 which is the replacement for the MS Groove solution. SharePoint Workspace uses InfoPath 2010 as the editor for all SharePoint list and library data that entered while offline. When SharePoint Workspace is brought back online, the data in the forms are synchronized back to SharePoint.
  
• SharePoint Designer 2010 Integration – InfoPath 2010 is now integrated with SharePoint Designer 2010. We now have the ability to define and publish InfoPath forms along with a business process that is designed in SharePoint Designer 2010.
  
• Business Connectivity Services (BCS) Integration – InfoPath 2010 has direct integration with BCS to expose enterprise data through InfoPath allowing for read, insert, update and delete operations.
  
• InfoPath Web Part – There is a new InfoPath Web Part that allows you to embed and InfoPath form directly into a web part page. Previously we had to go to a special web page that could render an InfoPath form over the web or we had to write a bunch of custom code. Now the InfoPath form can be displayed along with web parts that may provide complete different functionality.
  
• Deploy .NET Managed Code to Sandboxed – Publishing InfoPath forms with .NET managed code no longer require an administrator to publish the form through Central Admin. You can now simply publish the form using as a Sandbox Solution which I introduced this earlier in this series.
  
• New Web Form Controls – With the previous version InfoPath, there are several controls which were not available when the InfoPath form was web enabled. With InfoPath 2010 we have nine new controls we can use.
  
• Better Support for Open Standards.
  
• Better support for digital signatures.
  
• Integration with SharePoint PowerShell cmlets
  
• Integration with SharePoint 2010 health and analysis tool called SharePoint Maintenance Manager which automatically checks for configuration, performance and usage issues. There are specific rules for SharePoint 2010 for monitoring.
  

InfoPath Form Services Thoughts

If you read my blog or talk with me you will find out I have been a huge proponent of InfoPath since Office 2003. Obviously I have some important rules that I follow, most notably if .NET managed code is needed stop using InfoPath move to another platform. However InfoPath is such a great tool because:

• It is XML based.
  
• Can be rendered over the web through SharePoint with no client desktop footprint.
  
• It is very easy to create simple to medium level data capture forms.
  
• Does not require a developer.
  
• Very cost effective.
  
• Very easy to hook into web services to persist data externally.
  

I really believe developers just get too wrapped up not have a full IDE to work with and just cannot architect a solution well.

I am really very very very excited to see these new features and expect great things from InfoPath for SharePoint 2010. I really think this is a game changer for organizations to build much more powerful applications and solutions within the SharePoint platform.

Visio Services (Reference)

For SharePoint 2010, there is a new service called Visio Services which provide the ability to load, display and interact programmatically with Visio 2010 documents published into SharePoint.

From a programmatic perspective you have two options. First you can create a data provider for Visio which can be used to load in external data in the Visio form. Second ECMAScript Object Model provides programmatic access to the objects and data within the Visio diagram allowing you to create mashups, create overlays on the diagram, respond to mouse clicks, allow for panning and zooming, etc.

Visio Services Thoughts

I will be very interested to see use cases on how Visio Services will be used with SharePoint 2010. I have had clients ask me in the past, how they can make their Visio diagrams available over SharePoint without having to deploy Visio to every desktop. Well that is now solved. However this release goes well beyond that in that Visio is being made into a data driven application that can graphically display data. I think this again just adds to the richness of the SharePoint platform as another technology we can use to solve business demand.

Word Automation Services (Reference)

Word Automation Services is a new application service to SharePoint 2010. What this service provides is unattended, server-side conversion of documents into formats that are supported by Microsoft Word on the desktop. What is basically does is replicated the Save As functionality that you are used to in Microsoft Word client.

Word Automation Services Thoughts

At first glance as a developer you may not see the immediate value of this however there are several scenarios (Reference) where this can be very helpful. For instance let's say every week or month a large set of invoices need to be generated from a custom database and saved out as Word documents on SharePoint, this service could be utilized to support that. Another scenario would be once a Word document has gone through approval process, when it is complete PDF of that Word document needs to be generated.

PerformancePoint Services (Reference)

When SharePoint 2007 first came out they provided some light weight but very effective tools for Business Intelligence (BI) with SharePoint. Microsoft acquired a company called ProClarity and rebranded their dishoarding BI solution as a product called PerformancePoint which was eventually made part of the SharePoint eCAL. This was done because everyone who wanted PerformancePoint wanted it integrated within SharePoint.

For SharePoint 2010, PerformancePoint has been completely integrated as an enterprise service. PerformancePoint services allow users to create interactive dashboards for displaying KPIs, scorecards, reports, filters, etc. As well there is a full API that developers can use to create highly customized solutions. The architecture of PerformancePoint (Reference) is pretty robust and like I mentioned completely integrated into SharePoint 2010.

PerformancePoint Services Thoughts

I openly admit this is one of the areas of SharePoint I have little knowledge of and feel bad about because I know PerformancePoint is a pretty good solution with lots of features for providing dashboarding solutions. I personally have not used much of PerformancePoint in SharePoint 2007 and do know that most of the challenges associated with getting it to work were around configuration. Our SharePoint team would get tons of questions in regards to Kerberos configuration and such. Now that PerformancePoint is integrated within SharePoint many of these challenges will go away.

Plus given this is the second generation of Microsoft's solution offering I highly anticipate that it will be a very strong offering. Microsoft has made a large investment into this and Dashboard solutions pretty much go hand-and-hand with SharePoint as this is where users want to access their data.

SharePoint 2007 SP2 Released

SharePoint 2007 SP2 has been officially release. It "includes all the fixes prior to SP2, and also several enhancements to improve server farm performance, availability and stability. Plus, a new stsadm operation has been added to help customers prepare for the upgrade to the next version of SharePoint."

You should start planning on how to deploy this into your QA, Staging and Production environments…

The things I was happy to see for MOSS where (KB953334) improvements with Microsoft Office Forms Server (InfoPath) given I use it a lot with K2 blackpearl:

• Improves performance of large browser forms on InfoPath Form Services, both for memory usage and for page load time.
  
• Improves reliability of Forms Server and of Microsoft Office SharePoint Server by addressing the previous behavior in which an upgrade of an administrator-approved form template triggers an IIS reset.
  
• Improves the Digital Signature functionality for InfoPath Forms Services.

Custom SharePoint Web Service – Upload InfoPath Attachment

1.0 Creating the Utility

1.1 Introduction

This blog will hopefully help when trying to get attachments out of InfoPath forms and into SharePoint document libraries. If you are doing standard MOSS and InfoPath development, you are going to want to eventually get the attachments out of the InfoPath form. For instance when you search for attached documents, if they are still in the InfoPath form, search cannot find them because they are a binary string inside the InfoPath instance.

For K2 developers many of you know of the storage ramifications if the InfoPath form stores the attachment binary. I wrote a blog back in the K2.net 2003 days where I wrote a custom web service that would using the K2 API to save the attachment off in into SharePoint 2003. Some others came up with other solutions like using K2 SmartObjects, or extracting the binary out of the XML before the process instance starts. I just figured that there must be a better way of doing this in a generic way.

1.2 Solution

The solution is to create a custom SharePoint web service that runs under the context of SharePoint. Why did I decide to ultimately use a custom SharePoint service to implement this solution?

• I wanted to use the WSS 3.0 SharePoint API to implement this. I wanted to make this reusable solution whether I am doing K2 development or not.
• I wanted to use the SharePoint context to upload the document. When a document is uploaded in the fashion, the user must have permission to the document library. Now I get security for free.
• SharePoint will upload the file under the current user account. If I had done this as an external web service, the username that would have uploaded the file would the system account that the web service is running under; instead of the actual user who uploaded the document.

I will provide a detailed solution on how to get InfoPath attachments out of the forms and into SharePoint document libraries. This solution is geared towards anyone doing MOSS and InfoPath development. The solution is straight forward:

• First, I will create a utility class that will handle the saving of a attachment binary into SharePoint.
• Second, I will create a custom SharePoint service to wrap these methods.
• Third, I will show how to deploy the custom service in a WSP solution.
• Fourth, I will show how to wire up the custom service in an InfoPath form.

2.0 Upload Utilities

Before going off and creating this web service, let's create the utilities that will upload the files.

• I will create a utility class that will be responsible for uploading the file.
• I will also create a different class to assist with the creation of folders for uploaded documents. The reason why I provided this is because when building a workflow for the InfoPath form, you will probably have a unique key that is for the process instance (WF, K2, etc). For attached items, you will want to store them by process instance so you do not have attachment names conflict.
• I will also write another class that will get the files so that you can display links to all the files in the InfoPath form.

The ultimate user experience is that they will not ever know that they are uploading the document to a SharePoint library and it is completely transparent.

I will create a C# class library called WSSDistillery.Utilities. The reason why I am doing this is because all of these methods can be completely reusable in a list event handler, in a WF workflow, web part, etc. All the custom service is going to do is wrap the method calls.

In this project add references to Microsoft.SharePoint and System.Web.

2.1 Upload Attachments

The trick associated to getting an attachment out of an InfoPath form is removing the header. What happens is when an attachment is pulled into an InfoPath form; InfoPath will add a header into the binary of the attachment that captures information like the name and size of the file. We need to rip that out and only send the binary of the attachment itself into a document library. Plus we want to make sure the name of the file set into SharePoint too.

The following class will show you how to do it. Now in my old blog I show how to do it. I wanted to rewrite it plus I wanted to do it in a better way. I found this blog which made it so much simpler. I have repurposed that code here.

public class InfoPathForm
{
    /// <summary>
    /// Uploads an InfoPath attachment to a list and folders.  Method
    /// only supports saving items up to two folders deep.
    /// </summary>
    /// <param name="siteURL"></param>
    /// <param name="webUrl"></param>
    /// <param name="docLibName"></param>
    /// <param name="parentFolderName">Pass NULL if there is no folder.</param>
    /// <param name="subFolderName">Pass NULL if there is no sub folder.</param>
    /// <param name="infopathAttachment"></param>
    /// <param name="overwrite"></param>
    public static void UploadAttachmentFile(string siteURL, string webUrl,
        string docLibName, string parentFolderName, string subFolderName,
        byte[] infopathAttachment, bool overwrite)
    {
        try
        {
            MemoryStream stream = new MemoryStream(infopathAttachment);
            BinaryReader reader = new BinaryReader(stream);

            //Read header
            reader.ReadBytes(16); // Skip the header data.
            int fileSize = (int)reader.ReadUInt32();
            int attachmentNameLength = (int)reader.ReadUInt32() * 2;
            byte[] fileNameBytes = reader.ReadBytes(attachmentNameLength);

            //Get the attachment name
            Encoding enc = Encoding.Unicode;
            string attachmentName = enc.GetString(fileNameBytes, 0, attachmentNameLength - 2);

            //Get the real attachment without InfoPath header
            byte[] attachment = reader.ReadBytes(fileSize);

            using (SPSite site = new SPSite(siteURL))
            {
                using (SPWeb web = site.OpenWeb(webUrl))
                {
                    SPList list = web.Lists[docLibName];
                    if (String.IsNullOrEmpty(parentFolderName))
                    {
                        //Save document directly into list.
                        list.RootFolder.Files.Add(attachmentName, attachment, overwrite);
                    }
                    else
                    {
                        if (string.IsNullOrEmpty(subFolderName))
                        {
                            //Save document in folder in root of list.
                            list.RootFolder.SubFolders[parentFolderName].Files.Add(
                                attachmentName, attachment, overwrite);
                        }
                        else
                        {
                            //Save document in a sub folder.
                            list.RootFolder.SubFolders[parentFolderName].SubFolders[subFolderName].Files.Add(
                                    attachmentName, attachment, overwrite);
                        }
                    }
                }
            }
        }
        catch (Exception ex)
        {
            throw new Exception("Error saving attachment >> " + ex.Message.ToString());
        }
    }
}

2.2 Folder Methods

The following are the folder methods that I will use to create a folder in the root of a SPList. I also have a method for creating a sub folder. I really did not try to make this handle creating folders deeper than two levels. I also know it possible to write something a little more generic however I just want to expose allowing external callers to create a parent and a sub directory only.

public class Folder
{
    /// <summary>
    /// Create a folder in the root of a list.
    /// </summary>
    /// <param name="siteURL"></param>
    /// <param name="webUrl"></param>
    /// <param name="docLibName"></param>
    /// <param name="folderName"></param>
    /// <returns></returns>
    public static SPListItem CreateTopFolder(string siteURL, string webUrl,
        string docLibName, string folderName)
    {
        SPList list = GetList(siteURL, webUrl, docLibName);

        return CreateFolder(list,
            folderName,
            list.RootFolder.ServerRelativeUrl,
            null);
    }

    /// <summary>
    /// Creates a sub folder using a reference to a parent folder.
    /// </summary>
    /// <param name="siteURL"></param>
    /// <param name="webUrl"></param>
    /// <param name="docLibName"></param>
    /// <param name="parentFolderName"></param>
    /// <param name="subFolderName"></param>
    /// <returns></returns>
    public static SPListItem CreateSubFolder(string siteURL, string webUrl,
        string docLibName, string parentFolderName, string subFolderName)
    {
        SPList list = GetList(siteURL, webUrl, docLibName);

        return CreateFolder(list,
            subFolderName,
            list.RootFolder.ServerRelativeUrl + "/" + parentFolderName,
            parentFolderName);
    }

    //Get the list
    private static SPList GetList(string siteURL, string webUrl,
        string docLibName)
    {
        using (SPSite site = new SPSite(siteURL))
        {
            using (SPWeb web = site.OpenWeb(webUrl))
            {
                return web.Lists[docLibName];
            }
        }
    }

    /// <summary>
    /// Creates a folder.
    /// </summary>
    /// <param name="list"></param>
    /// <param name="folderName"></param>
    /// <param name="folderUrl"></param>
    /// <param name="parentFolder"></param>
    /// <returns></returns>
    public static SPListItem CreateFolder(SPList list, string folderName,
        string folderUrl, string parentFolder)
    {
        SPListItem folder = null;
        foreach (SPListItem f in list.Folders)
        {
            if (f.Name == folderName)
            {
                folder = f;
                break;
            }
        }

        if (folder == null)
        {
            folder = list.Items.Add(folderUrl,
                SPFileSystemObjectType.Folder, parentFolder);

            if (folder != null)
            {
                folder["Name"] = folderName;
                folder["Title"] = folderName;
                folder.Update();
            }
        }

        return folder;
    }
}

2.3 Get Files
This method will return all of the files for a specified SPList or folder. Right now it is only geared towards going two layers deep. I could write something that is recursive where I can say get all files from a specified folder at a specific level however right now that is not needed.

public class Items
{
    /// <summary>
    /// Method will return all of the files in a specified folder.  It is
    /// limited from only getting two levels deep.
    /// </summary>
    /// <param name="siteURL"></param>
    /// <param name="webUrl"></param>
    /// <param name="docLibName"></param>
    /// <param name="parentFolderName">optional</param>
    /// <param name="subFolderName">optional</param>
    /// <returns></returns>
    public static SPListItemCollection GetItems(string siteURL, string webUrl,
        string docLibName, string parentFolderName, string subFolderName)
    {
        try
        {
            SPListItemCollection items = null;

            using (SPSite site = new SPSite(siteURL))
            {
                using (SPWeb web = site.OpenWeb(webUrl))
                {
                    SPList list = web.Lists[docLibName];
                    SPFolder folder = null;

                    if (String.IsNullOrEmpty(parentFolderName))
                    {
                        //Get from root
                        folder = list.RootFolder;
                    }
                    else
                    {
                        if (string.IsNullOrEmpty(subFolderName))
                        {
                            //Get from first level of folders
                            folder = GetFolder(list.RootFolder.SubFolders, parentFolderName);
                        }
                        else
                        {
                            //Get from second level of folders
                            folder = GetFolder(list.RootFolder.SubFolders[parentFolderName].SubFolders,
                                subFolderName);
                        }
                    }

                    //Get the items
                    if (folder != null)
                    {
                        SPQuery query = new SPQuery();
                        query.Folder = folder;

                        items = web.Lists[docLibName].GetItems(query);
                    }

                    return items;
                }
            }
        }
        catch (Exception ex)
        {
            throw new Exception("Error creating list of items >> " + ex.Message.ToString());
        }
    }

    private static SPFolder GetFolder(SPFolderCollection folders, string folderName)
    {
        foreach (SPFolder folder in folders)
        {
            if (folder.Name == folderName)
            {
                return folder;
            }
        }

        return null;
    }
}

3.0 Creating the Custom SharePoint Web Service

The next step is to create the custom SharePoint web service. There is a MSDN Article and SDK article but in this article I have streamlined it. Here is another blog I found but it has some issues associated to the deployment.

3.1 Create a web service project

Create a web service project called WSSDistillery.Utilities.Service.InfoPath with a web service called InfoPathAttachment.asmx.

3.2 Create Web Methods

I then added a reference to the WSSDistillery.Utilities project I created in the previous section. You will need to add reference to Microsoft.SharePoint. All my web service does is wrap and expose the methods from my utility class.

public class InfoPathAttachment : System.Web.Services.WebService
{

    [WebMethod]
    public void UploadInfoPathAttachment(string siteURL, string webUrl,
        string docLibName, string parentFolderName, string subFolderName,
        byte[] attachment, bool overwrite)
    {
        InfoPathForm.UploadAttachmentFile(siteURL, webUrl, docLibName,
            parentFolderName, subFolderName, attachment, overwrite);
    }

    [WebMethod]
    public void CreateFolder(string siteURL, string webUrl,
        string docLibName, string folderName)
    {
        Folder.CreateTopFolder(siteURL, webUrl, docLibName, folderName);
    }

    [WebMethod]
    public void CreateSubFolder(string siteURL, string webUrl,
        string docLibName, string parentFolderName, string subFolderName)
    {
        Folder.CreateSubFolder(siteURL, webUrl, docLibName, parentFolderName, subFolderName);
    }

    [WebMethod]
    public XmlDocument GetAttachments(string siteURL, string webUrl,
        string docLibName, string parentFolderName, string subFolderName)
    {
        StringBuilder xml = new StringBuilder();
        xml.Append("<items>");

        SPListItemCollection items = WSSDistillery.Utilities.Items.GetItems(
            siteURL, webUrl, docLibName, parentFolderName, subFolderName);

        foreach (SPListItem item in items)
        {
            //Do not include folders items
            if (item.Folder == null)
            {
                xml.AppendFormat("<item><name>{0}</name><url>{1}</url></item>",
                    item.Name, HttpUtility.UrlPathEncode(item.Web.Url + "/" + item.Url));
            }
        }

        xml.Append("</items>");

        //InfoPath can consume an XML Document easily
        XmlDocument xmlDoc = new XmlDocument();
        xmlDoc.LoadXml(xml.ToString());
        return xmlDoc;
    }
}

3.2.1 Remove Code Behind Reference

After creating the service, right click InfoPathAttachment.asmx, click on View Mark Up and remove the code behind attribute leaving just.

<%@ WebService Language="C#" Class="WSSDistillery.Utilities.Service.InfoPath.InfoPathAttachment" %>

3.3 Create the Discovery and WSDL files

Next we need create the discovery and WSDL files. Now these steps are a little strange and may not make complete sense at first. Basically we have to make the service discoverable inside of SharePoint.

First what I did was create a virtual directory in IIS that pointed to my web service project. So my project is located at C: \WSSDistillery\Utilities\WSSDistillery.Utilities.Service.InfoPathAttachment and I point the virtual directory to use those files. The name of virtual directory is TestInfoPathAttachment. Then I opened the .NET command line window and ran the following command.

disco http://localhost/TestInfoPathAttachment/InfoPathAttachment.asmx

Second, this will generate two files (InfoPathAttachment.wsdl and InfoPathAttachment.disco) that are located at C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin. Go get those files, copy them into your web service project directory and add them into your Visual Studio project by adding existing items. Rename the files to be the following:

• Change InfoPathAttachment.disco to InfoPathAttachmentdisco.aspx
• Change InfoPathAttachment.wsdl to InfoPathAttachmentwsdl.aspx

Third, in both files do the following. Remove the XML tag line:

<?xml version="1.0" encoding="utf-8"?>

And replace it with this:

<%@ Page Language="C#" Inherits="System.Web.UI.Page" %>
<%@ Assembly Name="Microsoft.SharePoint, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" %>
<%@ Import Namespace="Microsoft.SharePoint.Utilities" %>
<%@ Import Namespace="Microsoft.SharePoint" %>
<% Response.ContentType = "text/xml"; %>

Be really careful when doing this because Visual Studio will sometimes try to add in double quote marks. If that happens, get rid of them because InfoPath will break when making the web service call. Someone actually mentioned this in the MSDN Article and I would have been scratching my head of weeks if that was not there.

Fourth, go into the disco file replace the contractRef and both soap nodes with the following. Pay special attention to where you replace in the name of the service. I have bolded the things you must change based on the name of the web service – in this case InfoPathAttachment.

<contractRef ref=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request) + "?wsdl"),Response.Output); %>
docRef=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request)),Response.Output); %>
xmlns="http://schemas.xmlsoap.org/disco/scl/" />
<soap address=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request)),Response.Output); %>
xmlns:q1="http://tempuri.org/" binding="q1:InfoPathAttachmentSoap" xmlns="http://schemas.xmlsoap.org/disco/soap/" />
<soap address=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request)),Response.Output); %>
xmlns:q2="http://tempuri.org/" binding="q2:InfoPathAttachmentSoap12" xmlns="http://schemas.xmlsoap.org/disco/soap/" />

Fifth, in the WSDL file, go to the every end of the file and you will find the following soap nodes.

<soap:address location="http://localhost/foo/InfoPathAttachment.asmx" />

<soap12:address location="http://localhost/foo/InfoPathAttachment.asmx" />

Change them to be:

<soap:address location=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request)),Response.Output); %> />

<soap12:address location=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(SPWeb.OriginalBaseUrl(Request)),Response.Output); %> />

DONE - That is really as simple as it gets!

4.0 Deploying the Service

Next we need to deploy this web service. I was disappointed in the MSDN Article because instructions had you go directly into the files of the server and dropping files on manually. In this blog, the author had you putting DLLs in both the GAC and the web bin directory. Nuts!

As I have said before, if I cannot deploy using a SharePoint solution you should probably reconsider your deployment plan. Plus, I have to consider the fact that if I have a SharePoint farm, I will have to manually put the files up on each server which is a drag. In this section I will show you how to create a WSP solution that you can use to deploy your custom SharePoint web service. Note, was not simple and required me to refresh myself of CAS.

It is a well known best practice to deploy all of your DLLs to the web bin directory (i.e. C:\Inetpub\wwwroot\wss\VirtualDirectories\80\bin). The reason why is:

• Deploying to the GAC makes the DLL completely accessible. If you have a real SharePoint farm and you are trying to respect security some DLLs should not be accessible across all farms.
• As well, may people get lazy and say just drop the DLL in the GAC or worse, make the security level medium or full. If anyone tells you that – they are not thinking straight. Not production environment should run above minimum unless a CAS policy has been specifically applied to a DLL.

4.1 Sign both Projects

You will need to sign both the web service and the utilities libraries. Even though they will not be deployed to the GAC, to write a CAS Policy we will need them to be signed to ensure they are uniquely named.

4.2 APTC the WSSDistillery.Utilities Project

When I was going through the deployment steps farther down I started to get errors, specifically when making the web service call. The errors were saying that partially trusted callers are not permitted. What was basically occurring was the WSSDistillery.Utilities class does not fully trust the web service call. This would occur because the web service DLL is in the SharePoint web site bin directory and not in the GAC. Even if I put the WSSDistillery.Utilities in the GAC, I would still get this error. The solution is to add the following into the AssemblyInfo .cs file of the WSSDistillery.Utilities project.

[assembly: System.Security.AllowPartiallyTrustedCallers]

I did some reading and found out that Microsoft.SharePoint allow partially trusted callers too; then I did not feel too bad. The deal is that make sure you are not susceptible to injection if you are going to allow this.

4.3 Manifest File

Next I create the following manifest.xml file which will deploy the DLLs and the .asmx and .aspx files to the ISPI folder. You do not need to create any sort of SharePoint Feature because all we are doing is pushing files on to the web servers. I am going to provide you with two options and this will work in a production environment that is running under minimal trust.

Before going continuing it would be good to open a Visual Studio command prompt and run the following commands as you will need these values when setting up the CAS policies:

sn -Tp C:\WSSDistillery\Utilities\WSSDistillery.Utilities.Service.InfoPathAttachment\bin\WSSDistillery.Utilities.Service.InfoPath.dll

sn -Tp C:\WSSDistillery\Utilities\WSSDistillery.Utilities.Service.InfoPathAttachment\bin\WSSDistillery.Utilities.dll
4.3.1 Option 1

In this option I deploy on DLL to the GAC and one to the web bin. What we are doing is allowing the web service to run in a safe mode.

<Solution xmlns="http://schemas.microsoft.com/sharepoint/"
          SolutionId="2583E0DD-2B0E-41b4-BFF3-4D4100B3D11B">
    <Assemblies>
        <Assembly Location="WSSDistillery.Utilities.dll" DeploymentTarget="GlobalAssemblyCache" />
        <Assembly Location="WSSDistillery.Utilities.Service.InfoPath.dll" DeploymentTarget="WebApplication">
            <SafeControls>
                <SafeControl Assembly="WSSDistillery.Utilities.Service.InfoPath.InfoPathAttachment, WSSDistillery.Utilities.Service.InfoPath, Version=1.0.0.0, Culture=neutral, PublicKeyToken=7e600b50acc43694"
                             Namespace="WSSDistillery.Utilities.Service.InfoPath"
                             Safe="True"
                             TypeName="*"/>
            </SafeControls>
        </Assembly>
    </Assemblies>
    <RootFiles>
        <RootFile Location="ISAPI\InfoPathAttachment.asmx"/>
        <RootFile Location="ISAPI\InfoPathAttachmentdisco.aspx"/>
        <RootFile Location="ISAPI\InfoPathAttachmentwsdl.aspx"/>
    </RootFiles>
    <CodeAccessSecurity>
        <PolicyItem>
            <Assemblies>
                <Assembly PublicKeyBlob="[ADD VALUE]"/>
            </Assemblies>
            <PermissionSet class="NamedPermissionSet" Name="WSSDistillery.Utilities.Service.InfoPath" version="1" Description="Permission for WSSDistillery.Utilities.Service.InfoPath">
                <IPermission class="AspNetHostingPermission" version="1" Level="Minimal" />
                <IPermission class="SecurityPermission" version="1" Flags="Execution" />
                <IPermission class="Microsoft.SharePoint.Security.SharePointPermission, Microsoft.SharePoint.Security, version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" version="1" Unrestricted="True" />
            </PermissionSet>
        </PolicyItem>
    </CodeAccessSecurity>
</Solution>

4.3.2 Option 2

In this option I deploy both DLLs to the web site bin directory.

<Solution xmlns="http://schemas.microsoft.com/sharepoint/"
          SolutionId="2583E0DD-2B0E-41b4-BFF3-4D4100B3D11B">
    <Assemblies>
        <Assembly Location="WSSDistillery.Utilities.dll" DeploymentTarget="WebApplication">
            <SafeControls>
                <SafeControl Assembly="WSSDistillery.Utilities.Folder, WSSDistillery.Utilities, Version=1.0.0.0, Culture=neutral, PublicKeyToken=5695a828a53b1107"
                         Namespace="WSSDistillery.Utilities"
                         Safe="True"
                         TypeName="*"/>
                <SafeControl Assembly="WSSDistillery.Utilities.InfoPathForm, WSSDistillery.Utilities, Version=1.0.0.0, Culture=neutral, PublicKeyToken=5695a828a53b1107"
                         Namespace="WSSDistillery.Utilities"
                         Safe="True"
                         TypeName="*"/>
                <SafeControl Assembly="WSSDistillery.Utilities.Items, WSSDistillery.Utilities, Version=1.0.0.0, Culture=neutral, PublicKeyToken=5695a828a53b1107"
                         Namespace="WSSDistillery.Utilities"
                         Safe="True"
                         TypeName="*"/>
            </SafeControls>
        </Assembly>
        <Assembly Location="WSSDistillery.Utilities.Service.InfoPath.dll" DeploymentTarget="WebApplication">
            <SafeControls>
                <SafeControl Assembly="WSSDistillery.Utilities.Service.InfoPath.InfoPathAttachment, WSSDistillery.Utilities.Service.InfoPath, Version=1.0.0.0, Culture=neutral, PublicKeyToken=7e600b50acc43694"
                             Namespace="WSSDistillery.Utilities.Service.InfoPath"
                             Safe="True"
                             TypeName="*"/>
            </SafeControls>
        </Assembly>
    </Assemblies>
    <RootFiles>
        <RootFile Location="ISAPI\InfoPathAttachment.asmx"/>
        <RootFile Location="ISAPI\InfoPathAttachmentdisco.aspx"/>
        <RootFile Location="ISAPI\InfoPathAttachmentwsdl.aspx"/>
    </RootFiles>
    <CodeAccessSecurity>
        <PolicyItem>
            <Assemblies>
                <Assembly PublicKeyBlob="[ADD VALUE]"/>
            </Assemblies>
            <PermissionSet class="NamedPermissionSet" Name="WSSDistillery.Utilities" version="1" Description="Permission for WSSDistillery.Utilities.Service.InfoPath">
                <IPermission class="AspNetHostingPermission" version="1" Level="Minimal" />
                <IPermission class="SecurityPermission" version="1" Flags="Execution" />
                <IPermission class="Microsoft.SharePoint.Security.SharePointPermission, Microsoft.SharePoint.Security, version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" version="1" Unrestricted="True" />
                <IPermission class="System.Security.Permissions.FileIOPermission, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" version="1" Read="C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\TEMPLATE\LAYOUTS" />
            </PermissionSet>
        </PolicyItem>
        <PolicyItem>
            <Assemblies>
                <Assembly PublicKeyBlob="[ADD VALUE]"/>
            </Assemblies>
            <PermissionSet class="NamedPermissionSet" Name="WSSDistillery.Utilities.Service.InfoPath" version="1" Description="Permission for WSSDistillery.Utilities.Service.InfoPath">
                <IPermission class="AspNetHostingPermission" version="1" Level="Minimal" />
                <IPermission class="SecurityPermission" version="1" Flags="Execution" />
                <IPermission class="Microsoft.SharePoint.Security.SharePointPermission, Microsoft.SharePoint.Security, version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" version="1" Unrestricted="True" />
            </PermissionSet>
        </PolicyItem>
    </CodeAccessSecurity>
</Solution>

4.4 Create a ddf File

Second you will need to create .ddf file, with the following commands. I usually just call this WSP.ddf and I add it to my web service project. This will package up the two DLLs and the three web service files.

.OPTION Explicit
.Set CabinetNameTemplate="WSSDistillery.Utilities.Service.InfoPath.wsp"
.Set DiskDirectory1="C:\ "

manifest.xml
%outputDir%WSSDistillery.Utilities.dll
%outputDir%WSSDistillery.Utilities.Service.InfoPath.dll

.Set DestinationDir="ISAPI"
InfoPathAttachment.asmx
InfoPathAttachmentdisco.aspx
InfoPathAttachmentwsdl.aspx

; if we don't delete this variable, we get an error.
.Delete outputDir

4.5 Project Post Build Events

Then I add the following to the post build events of the Web Service project which will build the WSP every time I have successful build of the web service project.

cd $(ProjectDir)
MakeCAB /D outputDir=$(OutDir) /f "WSP.ddf"

4.6 Deployment Scripts

Now all you need to do is running the following STSADM command to push the solution out. Note you have to provide the URL of the web server because the DLLs are deployed to the web site bin and not the GAC.

stsadm.exe -o addsolution -filename C:\WSSDistillery.Utilities.Service.InfoPath.wsp
stsadm.exe -o deploysolution -url http://MyServer/ -name WSSDistillery.Utilities.Service.InfoPath.wsp -allowGacDeployment -allowCasPolicies -immediate -force
stsadm.exe -o execadmsvcjobs

Now test it, simply go to http://MyServer/_vti_bin/InfoPathAttachment.asmx and you will see the new web service.

4.7 Changes to Custom Web Service

If you make changes to the custom web service, like adding a new method or parameter to your WebMethod you will need to regenerate the WSDL that was described earlier. Then you will need repackage everything including the recompiled DLLs and push all of the changes back out using the WSP solution.

stsadm.exe -o retractsolution -url http://MyServer/ -name WSSDistillery.Utilities.Service.InfoPath.wsp -immediate
stsadm.exe -o execadmsvcjobs
stsadm.exe -o deletesolution -name WSSDistillery.Utilities.Service.InfoPath.wsp

4.8 Making Web Service Discoverable in Visual Studio

This is completely optional, however the MSDN Article mentions that to make the web service discoverable in Visual Studio as a web service alongside the default Windows SharePoint Services web services you need to modify spdisco.aspx file that is located in the ISPI folder in the 12 hive. You will need to make this change on every server in the farm.

<contractRef ref=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(spWeb.Url + "/_vti_bin/InfoPathAttachment.asmx?wsdl"), Response.Output); %>
docRef=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(spWeb.Url + "/_vti_bin/InfoPathAttachment.asmx"), Response.Output); %>
xmlns=" http://schemas.xmlsoap.org/disco/scl/ " />
<discoveryRef ref=<% SPHttpUtility.AddQuote(SPHttpUtility.HtmlEncode(spWeb.Url + "/_vti_bin/InfoPathAttachment.asmx?disco"),Response.Output); %>
xmlns="http://schemas.xmlsoap.org/disco/" />

5.0 Using in InfoPath

I am not going to dive much farther to InfoPath development at this point. If you can get these methods created, you should be good to go from here. Below are a few notes to get you going.

5.1 Web Service Data Connection

Hooking this into you InfoPath form is really straight forward. All you need to do is create a data connection using a web service to http://MyServer/_layouts/InfoPathAttachment.asmx. When creating that web service data connection, do not select submit option, make sure you select retrieve option. I am going to skip the rest of the configuration of the web service data connection as that should be pretty simple.


5.2 Create an Upload Button

The only other little trick you will need to know is that on your InfoPath form, you will need to add a binary data field (base64) into the main source. The reason why is if you try to use the binary field in web service data connection on the form, InfoPath will not allow you to use the attachment control. Now I bet if you crack open the manifest.xsf file you can probably resolve this issue.

Below is a screen shot of a rule I created on a button that will upload a file from the IP from to the SharePoint server. Now all you need to do is create a rule to clear out the main data source attachment field and then create a third rule that will call the GetAttachments web method.

With all of these web methods you could satisfy a use case like:

• User opens a Purchase Request InfoPath form.
• A Purchase Request number is generated in the InfoPath form (out of scope of this posting).
• An attachment folder is created for the Purchase Request.
• User uploads a file associated to the Purchase Request.
• User will see links Purchase Request attachments on the form.

5.3 SharePoint Farm?

If you SharePoint farm that has several front-end web servers that are load balanced, you may run into an issue when making a connection to SharePoint web service when the InfoPath form is running in a web enabled mode. I ran into this issue a long time ago when trying to use the SharePoint Profile web service to get the current users information when using a web enabled InfoPath form (read this blog). You will need to use the URL to the web server directly instead of the load balanced URL. I have never dug any deeper into why this; if you scroll down in the responses of the blog you will see the issue that everyone runs into.

6.0 References

I did get some help along the way:

• http://msdn.microsoft.com/en-us/library/ms464040.aspx
• http://it.toolbox.com/blogs/sharepoint-blog/creating-a-custom-web-service-for-sharepoint-13553
• http://www.sharepointblogs.com/ssa/archive/2006/12/19/creating-a-custom-web-service-for-sharepoint.aspx
• http://social.msdn.microsoft.com/Forums/en-US/sharepointdevelopment/thread/75388e33-19f2-4387-9272-827f240f9ca0/
• http://blah.winsmarts.com/2008-8-InfoPath_2007_-_Extracting_those_pesky_Binary_attachments.aspx