Wednesday, October 31, 2007

November K2 User Group Presentation

Update 4/16/2009 - Here is the recorded presentation

I will be presenting virtually to the K2.Net User Group Meeting Tuesday, November 13, 11am-1pm central time. The actually presentation will between 11:45-12:45 Central Time.

I plan to demonstrate how to build a simple custom SmartObject Service.

For information on how to attend, please go to the following -

Sunday, October 28, 2007

K2 BlackPearl Initial Impression and 101 Training


A few weeks ago K2 sent out an early version of their training VPC for BlackPeal. The training VPCs for 2003 were invaluable for building early iterations of processes, building up demos and proof of concepts, etc.

These notes are not comprehensive and will require some more research on my part as I get more acclimated with all the new features and functionality.

Notes and Improvements

The following are notes that I gathered while go through the initial set of the materials.

  • SmartObjects is the new big thing they have released and it pretty impressive. SmartObjects provide a way to access business data from your line of business databases. With 2003 you had to build up your own service layers and then integrate them into the processes. Now BlackPearl provides the services layer to integrate business data across the enterprise. There are two types of SmartObject. From my initial research the first are SmartObjects which the developer defines where the data is stored in the SmartBox sever. You can now very rapidly create data structures to persist your workflow data to. The second are SmartObjects which you can code up using their SDK which can load up from any disparate datasource available given you have a service of some type to work with. The data is not cached within SmartBox server, you are just building a proxy based on a start interface to retrieve data and surface it up through K2. This deserves a deeper dive and some best practices will develop with time.
  • BlackPearl has tried to make the creation of line rules easier with the addition of Actions. When building events you define actions that an end-user can select from. When the wizard is done BlackPearl will generate all of the lines which can be used to connect to other activities. This assumes is that conditional logic is somewhat simple; which is not always the case. In many cases K2 will create a dropdown for users to select these actions making your development of processes very quick. So if I had a process where a manager should be able to select approve, deny, or resubmit I would create those three actions and within InfoPath a dropdown will be added by K2 where those three values would appear (do not worry, it is not hard coded into the InfoPath form). These actions are also available everywhere throughout workspace, SharePoint web parts, etc. as context menu items allowing the user to take quick action on an event instance without having to open a form.
  • There was a neat new event called Forms Generation Event which is a client event handler to create very basic forms within the new workspace. Although, given how simple it was, I still need to test the boundaries of it to see how feasible it is to use with real heavy duty business requirements are thrown at it. Nothing replaces writing a start custom page.
  • BlackPearl now provides a reporting solution which was not available in 2003. This was a major problem with K2 2003 and the solution was to persist all of your data externally to support reporting. Still is not the best place to store your business data and I still say it is best to keep that data externally (the new SmartObjects help with this best practice). Regardless all of the reporting within BlackPearl is now based completely on Reporting Services. They now provide several wizards to generate your own reports with the K2 Workspace. What was really exciting is that you can write your own custom reports in Reporting Services and then host them within the workspace. Reports associated to workflow may not always be centric and there may be multiple datasrouces where you need to retrieve data. Plus they have written their own data provider to access their data so you have immediate access to all data within K2 to create sophisticated reporting using Reporting Services. This is a huge improvement.
  • Users can now sign up for Notifications for processes, events, activities, destination rules, line rules, start rules, succeeding rules and escalations. This is great because with 2003 I continually had to design around not make K2 a spam machine. Now end-users are empowered with the ability to subscribe to events.
  • BlackPearl provides a web enabled process design tool that is embedded within SharePoint. The tool is better than the SharePoint Designer for creating standard business workflows as you can build workflows with InfoPath forms, they are re-usable, and easier to create custom activities. Still this tool is scoped toward created workflows within SharePoint only as it only is meant for designing workflows for a list or a library. I still need to test the boundaries for this tool. Given how easy it is to create workflows in Visual Studio using K2, I am more inclined to work with it at the moment.
  • BlackPearl now provides a way to create processes with Visio. To be honest when I heard about this at first was excited but skeptical it would be provide a way to create good workflows. During my initial testing of it turned out to work well and I was pleased with it. I will still need to again test the boundaries of this. The great thing about using Visio is that when defining the business process with business users during the elaboration phase I always use Visio to define the process. Although there is not always a one-to-one with the activities and events that I want to expose the business user to. So now if I create the workflows using Visio I can mark them up and them move them over to Visual Studio.
  • The integration with InfoPath 2007 has improved. Things to be excited about is the ability to support web enabled InfoPath forms, have multiple InfoPath forms in a single process, the InfoPath forms are integrated with Visual Studio along with your K2 process and the deployment of the InfoPath form with the K2 process (you no longer have to publish your InfoPath form, K2 will publish it for you). Although when adding .Net managed code to your InfoPath form has some problems due to the new MOSS requirement that any form with .Net manage code must be deployed through central administration. I will be doing a deeper dive into this soon.
  • provided a ton of integration with WSS 3.0 and MOSS. There are administrative event handlers for provisioning new sites, managing user permissions and groups and lists. These are very important as too often I go to companies where their SharePoint implementations have grown out of control because of no governance. Sometimes if the governance is too tight this can become a barrier for usage as business users cannot get what they need in a timely basis. These event handlers can be used to define light-weight business processes to automate the management of an enterprise MOSS implementation.
  • For lists and document libraries in SharePoint MOSS provides event handlers to do almost all of the operations you could need. Create, update, delete, check in/out, manage permission, move, copy, update metadata, download/upload documents, etc.
  • provides search event handlers that allow the developer to build up criteria to find items in SharePoint and then take action on them. I did not use reflector to go underneath the hood to see what it is being done but looks like these events are basically creating CAML queries that can go out and find items.
  • provides some new Records Management event handlers (create and delete records as well as put a hold on a record). My initial research into MOSS Records Management a few months ago I found that there is no workflow around the actual movement of documents into the records center. We have to rely on business users to go out, find a document and then move it to the records center. In many situations it is not that simple as there will be some sort of approval process before the document should be moved over. BlackPearl can be leveraged to build that business process.
  • BlackPearl provides several event handlers around managing web content. As you may know CMS has been rolled into MOSS. In MOSS this means new content management templates, publishing pages, etc. BlackPearl provides event handlers that can create, copy, move, delete, check in/out for publishing pages. As well you can get and update publishing page content, create pages using Conveters and create and Update reusable Content. I can see how this can be used extensively to create workflow around the management content.
  • BlackPearl provides integration with the BDC to expose SmartObject data. As you may know it can become difficult to create the XML configuration for the BDC to read data. If you are already loading in line of business data into your SmartObjects it is really almost no effort to expose any SmartObject through the BDC into SharePoint. It was really simple to hook up a SmartObject into the BDC as K2 has created some custom screens within SharePoint Central Administration to facilitate this. So if you have some SmartObjects with some Oracle, SAP and SQL data all you need to use the K2's BDC integration and you are done.

Miscellaneous Notes

  • Within Visual Studio they have introduced what they call environment configurations. They are similar to K2 2003 StringTable but are reusable between processes.
  • Improved presentation and integration with Visual Studio. Their implementation of WPF for their wizards is really clean and is exciting to see a real implementation of. They also implemented this neat little thing where the developer can draw letters on the Visual Studio canvas which will create a new activity. Is it necessary; not really; but it is neat.
  • The BizTalk events were not present on the VPC. I found out they have not be released yet. This is not terrible as it is still possible to invoke BizTalk through other means but I have used the K2 BizTalk adapters on two projects.
  • Getting into the properties of an event or activity is much easier now.
  • Workflows can be initiated directly from the workspace. In the past, we would have to write little stubs to kick off workflow.
  • The K2 Service Manager is now part of the K2 workspace. The K2 workspace is still geared towards the power user as it can still expose too much information to the common everyday business user.
  • They have created a new concept called "Roles" which seem to be similar to the Destination Queues which were available in 2003.
  • K2 deprecated a couple event handlers, the SQL and Web Service event handlers were gone. Good riddance J

Tuesday, October 16, 2007

MOSS Power User References and Training

Many clients have asked for references for not just a technical training but power user training and references for MOSS. This should compliment an earlier recommendation for technical references and training. I recommend picking up a copy of SharePoint 2007 User's Guide: Learning Microsoft's Collaboration and Productivity Platform by Seth Bates and Tony Smith.

This series so far has been really good for SharePoint 2007. The book lays out step by step instructions on how to do things. If you would like to read an introductory chapter please go here.

I would also highly recommend using the Office SharePoint 2007 Help and How to Site as there is a top of information here with steps on how to do every day sorts of things.

As for training I am going to again refer to MindSharp as they offer a SharePoint 2007 Power Users Course.

Monday, October 15, 2007

BlackPearl and MOSS Workflow Options

1. Background
A common question asked is should you use, Windows Workflow Foundation (WF) or SharePoint designer. The following is some quick things that should be considered.

1.1 General WF
WF architecturally has the look and feel of K2 and we know the K2 BlackPearl release is built completely on WF. Here is a high level context diagram of it.

A few things to keep in mind:

  • Have to build low level interfaces to move data between applications and the workflow definitions.
  • Have to build a hosting application to manage transactions (especially long-running transactions).
  • Have to create a transaction management and logging database.
  • Have to build ACID transaction management using WF activity library.
  • Must create graceful and generic exception handling.
  • No "out of the box" integration with any applications in the Microsoft stack.

This is not a comprehensive list and is based on observations and other projects I have seen try to do complete WF solutions from scratch. WF is great foundation to build solutions upon but it could be equated to "plumbing". If you have done work with BizTalk you may know it is a powerful Enterprise Service Bus pattern solution. BizTalk has orchestrations which would consistently confuse stakeholders into thinking that a simple diagram is drawn and configured and the project is done. We know this is not the case and the same analogy can be drawn for WF.

WF is similar in nature for the workflow world. It is very interface driven, requires lots of custom code and a high-level of effort. Some return on investment is gained with its activity library abstractions but not enough. As well, the level of adoption of pure WF solutions has not gotten off the ground when it comes to building custom applications.

1.2 SharePoint 2007 Workflow

There are two options when doing SharePoint workflow the first is building workflow in SharePoint Designer 2007 and building workflows in Visual Studio.

SharePoint Designer 2007 is a rebuild of MS FrontPage made specifically to work with SharePoint. Some disadvantages of building workflow in SharePoint Designer 2007 are:

  • Meant for business users as a code free workflow solution for managing items within SharePoint only.
  • When using SharePoint Designer 2007 your site can become customized (unghosted). Allowing business users to have elevated privileges to use SharePoint Designer 2007 is not good either.
  • Workflows are not reusable and are bound to the list (there are some known ways of doing it but it is not natural; the official answer is it is "NOT supported" - Porting SharePoint Designer Workflows to Visual Studio). Note this makes it difficult to move them between development, QA and production environments.
  • Limited conditional logic.
  • Cannot add custom code to workflow from designer (read whitepaper in called "Adding Activities to SharePoint Designer 2007" in the EMC Starter Kit).
  • Form integration not robust ( or InfoPath).
  • Centers around workflows to manage documents, sending emails and creating tasks only.
  • Supports sequential flows only.
  • No ability to debug.

The second option is to create custom WFs using the architecture discussed in the first section using Visual Studio. From information I have gathered from other colleagues whom have done pure MOSS WF solutions they we difficult. Some disadvantages of doing MOSS workflow in Visual Studio is:

  • Provide basic activities and events for MOSS only.
  • Difficult to deploy.
  • Robust audit and metric data must be built up.
  • Creating large multi-step processes difficult.
  • Integration with other platforms must be built from scratch.

1.3 References to Learn About Both
To try your hand at both SharePoint Designer and Visual Studio Workflow try this virtual lab.

As well check out the book I recommended here.

Also recommend reviewing these two links Workflow Development in Office SharePoint Designer and Workflow Development Tools Comparison.

2. BlackPearl and MOSS BlackPearl simply becomes an obvious choice once you start pealing back the layers. It is built completely on top of WF and removes every one of the issues stated above. Out of the box:

  • Comes with event handlers to do every operation you need with a SharePoint List or Document Item. Updating metadata, moving, deleting, creating, batch operations, modifying the document permission, checking in/out, etc. They basically provide wizards to generate and customize Features requiring the developer to write less code.
  • They provide several events for publishing content supporting the Content Management Server (CMS) features that were rolled into MOSS.
  • They have some out of the box events to build workflow around Records Management.
  • Creating complex multi-stage InfoPath 2007 processes is supported (this includes web enabled forms).
  • Provide several events for SharePoint site administration. Events to managing users and permissions, managing lists/libraries and provisioning sites and workspaces, etc.
  • They provide an Ajax tool to build workflows on the SharePoint site itself.
  • Workflows can be authored inside of Visio.

There is really so much more as this is the tip of the iceberg for BlackPearl.

Saturday, October 13, 2007

Copy SPListItem.Version History with BlackPearl Part 2

1. Background

Copy Version History with SPListItemVersion to a new SPListItem Part 1 -

In a prior entry we were researching how to move the history from a SPListItem to another SPListItem that will be archived to another location. While doing research on this, we decided to check BlackPearl to see if it supported copying the version information. Using Reflector we were able to see that BlackPearl was not supporting the coping of version history when an SPListItem is copied from one list to another.

2. Resolution
The resolution is simple and still the same was as it would be done in 2003. Basically we need to go modify the code; the only difference is that we are now writing code in Windows Workflow Foundation (WF) instead of in an standard event handler.

Now go to the Event, right click on it and select to view the Event Item.

This will drive you to a screen where you will see the under laying WF code which BlackPearl is built on. Just seeing this is makes me giddy; even the underlying code is presented in a graphical format allowing developer to zero on the exact thing they need to modify. Writing code in this manner forces developers to write more modular code which is easier to understand and maintain.
There are two ways you can make this modification.

2.1 Option 1
One option would be to double click on CopyListItem code event handler and you will be taken to a code screen that you can start modifying.

2.2 Option 2
The second option would be to add an if statement and then add a new code event to the workflow. This is the best solution as it first segregates the custom code from the code. Second all of the K2 SharePoint events use their custom web services which we will not use. Instead we would insert code which was discussed in the part of this article series.

Copy SPListItem.Version (SPListItemVersion) Part 1

7/13/2008 - Going through and cleaning up code samples in popular posts. The old tool I used stunk. At least this is better. I apologize for the inconvenience.

10/14/2009 - I have created a new blog posting with updates as there were some issues with this original posting. Please go here.

1. Introduction

In MOSS you may have a requirement to take list items and move them to another location. A simple scenario would be that you have an item in a list and when a specific value is set you want to move that item to another location. One problem that you will run into is when you copy the item from one list to another is that you will want to ensure that the version history of the item is moved with the item. This article will discuss how you should go about fixing this problem; as well as a quick blurb on how this would be implemented in BlackPearl.

2. Available Methods and Approach

There are several available methods apart of SPListItem that you may be tempted to use. Specifically there are the CopyTo() and CopyFrom() methods. After some research online and using with Reflector to dive into the System.Microsoft.SharePoint neither of these methods can be used because the file name must be passed in has part of the URL parameters for the methods. With a standard list item object the file will be null making these methods useless unless you are working with a document object. As well, these methods do not support the copying of the version history.

The only viable solution is to loop over all of the values in SPListItemVersionCollection and set them into a new SPListItem. The Restore() method of SPListItemVersion cannot be used as it will take a specific version and restore it as a new version within the current SPListItem object. While looping over each SPListItemVersion the call Update() of the new SPListItem object will be called to rebuild the version history into the new object.

3. Solution Outline

An outline of the solution is as follows:

  • Create an Event Handler for the ItemUpdated event.
  • Create a new SPListItem object that will be the item that will be archived.
  • Loop over the Versions property of the source SPListItem. When looping over the item you must loop going backwards as the last item in the SPListItemVersionCollection is the first version saved.
  • Set each field from SPListItemVersion and call update on the new SPListItem (if there are 50 versions, then update will be called 50 times).
    • Note that index zero is always the latest version that a user would edit through SharePoint.
  • Then move the file attachments to the new SPListItem.
    • Note that attached documents to a SPListItem are not versioned. If a document needs to be versioned, it is best to put that document in a document library that is versioned controlled and then link the document to the SPListItem.
  • Deploy the Event Handler as a Feature.

4. Code

4.1 Feature

This article will not discuss the particulars of creating Features but shows them here for reference. NOTE – Change the GUID

Title="Item Archive Event"
Description="This event will archive an item when it is closed"
<ElementManifestLocation="Elements.xml" />

4.2 Elements of Feature

<Elements xmlns="">
Events, Version=,culture=neutral,

4.3 EventHandler Class

public classArchiveItemEventHandler : SPItemEventReceiver {

public overridevoid ItemUpdated(SPItemEventProperties properties) {

4.4 Create Folder

This solution is based on moving items to an archive folder which is dynamically generated every month. The code below creates a new folder in SharePoint based on the current month.

private SPListItem GetArchiveFolder(SPList list) {
//Get the folder for the current month
string folderName = System.DateTime.Now.Month.ToString() +

SPListItem destinationFolder = null;

foreach (SPListItem folder in list.Folders) {
if (folder.Name == folderName) {
destinationFolder = folder;

if (destinationFolder == null) {

//Create new folder
destinationFolder = list.Items.Add(
SPFileSystemObjectType.Folder, null);

if (destinationFolder != null) {
destinationFolder["Name"] =
"Archive Closed Records (" + folderName + ")";

return destinationFolder;

4.5 Copy the Versions into a new Item

This code is the meat of the solution. Couple of more notes:

  • A new archive item SPListItem is created in the archive folder location.
  • The DisableEventFiring() and EnableEventFiring() methods wrap the code. This is needed because the there will be an Update() call for every version.
  • Note again that the looping of the versions starts with the last item in the collection and works backwards.
  • There is code to accommodate Created, Created By, Modified and Modified By. They are read only fields. If they are not set the values will be lost and set with the system account and the time in which the code executed.
  • After all of the versions have been updated into the new item, the attachments are moved into the new item.
  • Finally the source item is deleted.

That is essentially it. The biggest thing to walk away with is that the properties.ListItem.Version[x][y] is two dimensional and that is what gets you access to all of the field values in the version collection.

private void MoveItem(SPListItem sourceItem,
SPListItem destinationFolder,
string newItemLocation) {

//Create a new item
SPListItem archiveItem = destinationFolder.ListItems.Add(


//loop over the soureitem, restore it
for (int i = sourceItem.Versions.Count - 1; i >= 0; i--) {
//set the values into the archive
foreach (SPField sourceField in sourceItem.Fields) {
SPListItemVersion version = sourceItem.Versions[i];

if ((!sourceField.ReadOnlyField) && (sourceField.Type != SPFieldType.Attachments)) {
archiveItem[sourceField.Title] = version[sourceField.Title];
else if (sourceField.Title == "Created"
sourceField.Title == "Created By"
sourceField.Title == "Modified"
sourceField.Title == "Modified By") {

archiveItem[sourceField.Title] = version[sourceField.Title];

//update the archive item and
//loop over the the next version

//now get the attachments, they are not versioned
foreach (string attachmentName in sourceItem.Attachments) {
SPFile file = sourceItem.ParentList.ParentWeb.GetFile(
sourceItem.Attachments.UrlPrefix + attachmentName);

archiveItem.Attachments.Add(attachmentName, file.OpenBinary());



//Now delete the current item.

5. Referencs

Sunday, October 7, 2007

MOSS Book and Training Recommendation

1) Books
There are a ton of MOSS books being published right now. If you are a developer and looking for one; hands down get Microsoft SharePoint: Building Office 2007 Solutions in C# 2005 by Scot Hillier. Scot’s book discusses all of the things you will need to do day one for as a developer from a SharePoint project. He provides step by step instructions on how to create site templates, web parts, features, SSO, object model stuff, some administration, fundamentals and he actually provides an example of a how to create a custom workflow that works. I had messed around with writing WF flows throwing them into MOSS during beta timeframe and failed miserably due to the complete lack of documentation (I would still not go off the deep end and create core workflows in MOSS). Although the workflow he provides is very similar to a free virtual lab I did some time ago. Try it out.

If you want to get Microsoft Office SharePoint Server 2007 Administrator's Companion by Bill English. This is a fine book but most of the information you can get for free on MSDN.

Also if you want to get good training on SharePoint I highly recommend attending a training session with MindSharp.

Wednesday, October 3, 2007

BlackPearl Training VPC

Yes - I have the Blackpearl traing VPC and I am very excited. I have been remiss on making blog enteries on my initial why playing around with first release of BlackPearl. I had built my own VPC and done some playing around with it. I will be working with it over the next two weekends. Right now I am a weekend warrior as I have been busy with MOSS related projects.