BlackPearl 803 Build Released

Exciting news – the 803 build for BlackPearl has been released with some several things that were needed. Here is the notification that was sent out. Some of the good stuff is 64-Bit Support, License Management is now working in the K2 Workspace (used to have to re-run the configuration wizard), tons of performance enhancements (saw the stats at the conference) and a bunch of documentation has been rolled in.

---

K2 blackpearl™, the centerpiece of the new K2 platform, has the K2 [blackpearl] 0803 (4.8075.1.0) release now available.

The 0803 release provides support for 64-bit hardware platforms, support for Worldwide English (non-English Windows, SQL Server, SharePoint, Office and Visual Studio), improved license management, performance enhancements and post-SP1 patches and bug fixes.

64-BIT SUPPORT
64-bit support allows K2 blackpearl components to be installed on 64-bit hardware platforms. Support for the 64-bit platform includes the following components.
> 64-bit SQL Server 2005
> 64-bit SQL Server 2005 Reporting Services
> 64-bit SharePoint 2007 (WSS/MOSS/Forms Server)
> 64-bit K2 blackpearl Server
> 64-bit K2 Workspace

WORLDWIDE ENGLISH (INTERNATIONALIZATION)
Worldwide English support allows K2 blackpearl to run in English on non-English platforms. Support for non-English platforms includes the following components.
> Windows 2003 Server (Standard or better)
> SQL Server 2005
> SQL Server 2005 Reporting Services
> SharePoint 2007 (WSS/MOSS/Forms Server)
> Visual Studio 2005 (Professional or better)
> Visio 2007

LICENSE MANAGEMENT
The K2 license management feature adds the following capabilities.
> Add / Delete license
> Edit existing licenses
> Manage Logged in Users
> Manage License over usage

DISTRIBUTED INSTALLATION ENHANCEMENTS
The K2 Installation Manager and K2 Configuration Manager tools have been greatly enhanced to allow for greater flexibility and robustness when installing K2 blackpearl in distributed environments, including the following.
> Improved support for network load balanced web front ends in SharePoint farms
> Improved support for network load balanced K2 Workspace scenarios

PERFORMANCE ENHANCEMENTS
Numerous performance enhancements have been implemented. Some of the highlights are as follows.
> Workflow Server Performance
> Improved processes execution
> Worklist & Worklist Item performance
> Improved memory usage when starting processes and finishing work list items
> Improved batch execution of K2 processes
> SmartObject Performance
> SmartObject Server data handling enhancements
> Client Data Marshalling enhancement
> SmartObject Synchronization - see http://kb.k2workflow.com/Articles/KB000250.aspxKB000250 - SmartObject GUID Synchronization

IPC EVENT
The following issues have been fixed relating to the IPC Event.
> Process Synchronization using Batch Actioning, IPC Sync Events, and Worklist Navigation
> Worklist Issues with Multiple Destinations, IPC Events and Dynamic Roles
> Not all processes in a solution are displayed when browsing for child processes in an IPC Event
wizard
> When an activity contains an IPC event where the destination user is mapped to the child process and
is used in a client event, both users are able to view the other user's work items.

DOCUMENTATION
Several additions and enhancements have been made to the K2 blackpearl Documentation. The key areas of improvement are as follows.

OVERALL DOCUMENTATION
> Combined all documents to be linked from CHM file for easier discovery
> Added a "Send Feedback" link to all topics
> Added/Improved content for Troubleshooting, License Management, K2 Wizards, SmartObject
(Delete), 2003 Interoperability and InfoPath Templates

GETTING STARTED GUIDE
The K2 [blackpearl] Getting Started Guide has been completely rewritten to provide clear concise information for planning and installation including the following.
> A Getting Started Checklist spreadsheet is embedded in the guide to help you plan, document and
verify your K2 environment
> Software prerequisites list is provided by component and by role
> A new "Before you begin" section has been added that details steps for configuring DNS, Service
Accounts, Permissions, NLB, SPNs, MSMQ and DTC
> Detailed information for installing in a distributed environment has been greatly enhanced
> New topics have been added to the Troubleshooting the Installation section

DEVELOPER REFERENCE (FORMERLY SDK)
The K2 [blackpearl] Developer Reference was introduced mid Q1 and has been updated with the following new content for 0803.

> Database Overview & Schemas

Architecture topics
> Architectural Overview of K2 [blackpearl]
> Architecture of Authoring, Management and Runtime aspects of the K2 platform
> Architecture of the Workflow Server
> Architecture of the SmartObject Server
> Architecture of the Event Bus
> Architecture of the Environment Library
> Architecture of the Workspace Management Console

New "How To" topics
> How to access and return information from the Global Worklist
> How to upload a file to a SmartObject
> How to add a 3rd-party event recorder to the K2 [blackpearl] Server
> How to create a custom MSBuild task to set rights on a process
> How to Generate a Report in PDF format

COMPATABILITY MATRIX
The K2 [blackpearl] Compatibility Matrix has been added to the customer portal at http://portal.k2workflow.com/help/k2blackpearl/k2bpmatrix.aspx.

SharePoint Customizing Display of Profile Data

We recently took over the management and support of an Intranet site. One of the first things our list to knock is to fix the way profile data is being displayed. A couple of the issues we have are:

• Users log in with their AD accounts but there are several data elements that need to originate from different data sources.
• mySites are turned on and we need to fix currently provisioned sites to show this extra data. As well, all new sites need to need to appear correctly.
• There is also the userdisp.aspx page which displays profile data. We need to make sure that is in sync with data displayed on the mySites.

We did some research and found some really good stuff out there that will help us get this problem resolved.

• Using BDC to Augment Profile Data – here is a good guide that shows you how to augment data being pulled with the BDC.
• mySites Redirection – this is a pretty good article that shows how to do redirection of the userdisp.aspx to the mySites page. We are not going to take this approach but worth noting.
• Customizing mySites 1 and customizing mySites 2 – both of these are really solid articles that provide similar approaches for overriding the presentation of mySites. Both entail Feature Stapling to existing sites and using FeatureSiteTemplateAssociation for new sites. This helps get around not messing around with the Microsoft owned site template.
• Customizing User Information Page – this is a three part series on how to modify the userdisp.aspx page and customize it.

Between all of these you should be able to construct a good end-to-end solution to override and display of profile data.

Also take a good look at this Code Plex Project. Really cool.

Email and Escalation Configuration Suggestions

I would like to make a few suggestions after just completing another BlackPearl project.

Re-useable Email Configuration

In the past for K2 2003 I had a complex project where we had tons of emails being sent and the users wanted the ability to configure emails after process deployment. Our solution was to create a database with email subjects and bodies that were configurable through a custom ASP.net page. Problem was that took a little time to through together in K2 2003.

Now with a recent BlackPearl project we had tons of email configurations; lots of them. We made sure we were not creating a spam machine however it was needed. The deal is that even with SmartObjects that did not solve the problem because I needed to embed dynamic content into the subject and body of the content. I wanted to use the drag and drop to get data from both XML and SmartObjects. Plus I did not have lots of time to pull this together. Our solution was to manually get this into like around 40 different email configurations access tradition email events and escalation emails.

What I would like to see is a way to define email subject and body templates that can be used anywhere in the process. I understand that one limitation would be that I could only reference SmartObject and Process level fields but that is good enough. I know there is this whole Notification framework in the K2 Workspace but I rather have this part of the process definition.

Escalation Configuration Usability

Getting into and navigating through escalations could be made easier. Simple things like the ability to re-order the way are listed (even though that does not control the order in which they are fired). Another would be grouping up some of the field configurations into fewer screens to click through because it can be time consuming to click through tons of them. Maybe the ribbon that is in many of the event wizards would be incorporated into the escalation wizard.

Need Escalation Rules

I would really like to see the ability to define conditional logic around escalations. There should be two types of configurations. The first is the ability to configure rules that would register the escalation when the activity instance is created. A use case would be if this is over a $10,000 dollars a user group must be reminded in three days to take action versus a different group of people may be every day. In this use case, it would be expected that I configure two different escalation rules; one for over $10,000 and another for under $10,000

The second is the ability to configure rules when the escalation is raised. The use case would be if a dollar amount has changed from $10,000 to $500 maybe an escalation event that was registered may not apply any more.

Now both of these can be accomplished by adding custom code into the code of template but it would be good to be able to do this through the UI similar to putting in a line or destination rule.

SmartObject Workshop Thoughts

Background

Colin Murphy, Gabriel Malherbe and I held a workshop on SmartObjects at the K2 Insider Conference this April. It was an interesting discussion because the three of us had some conference calls beforehand where tried to gain a better understanding of how to position SmartObjects in the Enterprise Architecture. I wrote a blog that has not been published which all three of us iterated on before the conference. I made some statements in that article (to invoke debate) where I asserted that SmartObjects could be considered SOA. I got beat up a little by colleagues and now admit that SmartObjects are not SOA even though they adhere to many of the tenants of SOA like autonomy, manageability, discoverable, maintainable, exception handling, scalability, reliability, transactionality, security, logging, buildability, interoperability, testability, etc. Maybe it is low on the maturity scale of SOA (level 2 of 5) and I still debate that sometimes that is good enough for many organizations. It can be challenging to get a company to invest to get to a high maturity level of SOA.

So then what are SmartObjects? Here are some excerpts from the slides that we presented.

Before and Now

In the K2 2003 days we would have to go into the client event after the code was generated and start modifying it to get data from external places. Remember that there is no code generation in BlackPearl; it is declarative.

Now all we do is drag and drop smart object references into the canvas to gain access to data; no code.

K2 [blackpearl] Solution

• Blackpearl introduces SmartObjects to solve many of these issues.
• SmartObjects:
• Are reusable business entities which can be deployed centrally and consumed by non-technical workflow authors
• Are a storage location for process data (SmartBox)
• Are a means to quickly access external LOB Data
• Allow developers to aggregate data from multiple backend systems into a single, composite object
• Can be created without writing any code
• Can be accessed outside of the workflow (ADO Provider) and can be reported on

K2 SmartObject in Enterprise Architecture

The following diagram is from a presentation I prepared in November of 2007 for the Texas User Group. At the time I was digging around SmartObjects trying to understand it. From what it looks like my diagram is still spot on.

K2 states that SmartObjects are used to "provision" data into your workflows. K2 uses it as the underpinning to the entire BlackPearl platform. The two things you should know right off are there are SmartObject Services and SmartObjects.

SmartObject Services are:

• The providers that access line of business and expose the data through a common interface.
• They expose methods that can be executed; typically CRUD operations.
• They are registered with the K2 Server and instances of them are created through K2 Workspace.

SmartObjects are:

• Class definitions that have data elements that map into the methods that are provided by the SmartObject Service.
• SmartObjects can be graphically used within K2 to access data.
• SmartObjects have a data provider API which you can easily hook into other tiers of the architecture.

In the end SmartObjects is part of your Data Access layer and they allow you to basically write data providers which are access through SmartObjects.

K2 [blackpearl] Solution

The following are out of the box SmartObject Services that are provided by SharePoint.

• SmartBox (cannot be used for custom SQL Databases)
• SharePoint
• Active Directory – I cannot live without this service when building workflows.
• K2 [blackpearl] – Provides easy access to all data that is within K2.
• Salesforce.com

Blackmarket SmartObject Services

The Blackmarket is a K2's version of CodePlex. It is an area where developers can share code with one another - http://k2underground.com/k2/ProjectLanding.aspx

• Dynamic SQL Stored Procedure Service – Build up Service based on stored procedures in SQL Server. I plan to start using this one because I prefer to be calling stored procedures versus accessing tables and columns directly.
• Dynamic SmartObject Services Service – Build up Service based on schema of a SQL Server instance.
• SharePoint Users in Groups ServiceObject - This ServiceObject when wrapped in a simple SmartObject allows you to leverage SharePoint Groups.
• Office Communication Server Service - The OCS Service Broker allows users to create a SmartObject to send notifications via Microsoft Office Communications Server 2007.
• Dynamic WebService Service– Build up Service based on a web service.
• SmartObject Service Provider for Oracle – Build up Service based on schema of an Oracle instance.
• Dynamic SmartObject Service – Dynamically builds up services that use existing SmartObjects as the datasource. The author says this allows you to create SmartObject Service inheritance.

This catalog of services is going grow as more people start to contribute.

Conclusions

The following are my conclusions after the discussion.

• SmartObjects give you a significant edge in delivering timely solutions with K2. They have provided me the ability to immediately access Active Directory, SQL Databases, etc. We have personally seen that we can now deliver working processes in days instead of weeks because we no longer have to build up custom data access layers and go into the code of K2 and manually access the data. I can use the SmartObjects in a drag and drop fashion in line rules, succeeding rules, preceding rules, emails, destination users/rules, any configuration item in any event wizard, etc. It is ridiculously simple to configure my entire K2 process with data and I have zero custom code to get at the data. Plus I can write C# code in custom event handlers to use SmartObject. The API is really clean, simple and similar to using System.Data.
• Another great thing about SmartObjects is that if you carefully take the time to build them you will start being able to build catalogs of SmartObjects that business users can use when composing their K2 Processes. They will not be exposed to any of the details associated to knowing where the data comes from.
• Right now one concern is the ability to place business rules against the SmartObjects methods. For instance if you use the Dynamic SQL Service which build the interface of the service based on the schema of the database there is no way put business rules on the SmartObject to ensure that a value must be greater than X or cannot be null. So if a business user were to use SmartObject Event in their process they could possibly just start setting invalid data.
• The SmartBox is good for doing early iterations of processes but over the long term you may not want to us them. For instance you have limited control over the schema of the database (because it is generated). As well, you may not want to co-mingle data from different processes in the same database.
• Do not let you SmartObjects replace your Data Warehouse. With SmartObjects it is possible to perform CRUD operations across line of business data sources. It is very tempting to have a SmartObject read from many different places providing a single view of the data despite where the data is being read from. A simple example would have a SmartObject read from Active Directory and SQL Server. You must take performance into consideration when doing something similar to this. Performance issues associated to reading data will be vetted out in the SmartObject Service implementation however if you read in two large data sources and then join them together in the SmartObject layer you will not have the opportunity to index those joins. It would be better to have the SmartObject access data in the Data Warehouse itself.
• This comment will be a little at odds with the first but LINQ, Entity and .Net Data Services make it challenging to position SmartObjects. SmartObjects Services may use them to access data and surface it up through a SmartObject for K2. However many developers will be tempted just use LINQ directly inside your custom server events, middle and UI layers. I have not personally messed with LINQ and Entity as much as I should but from the hype it is pretty rich and provide a full Object Query Language and Object Relational Mapping (ORM). The one nice thing about SmartObjects is that they provide you a way to centralize and publish services.

Notes from April Insider Conference in Denver

I had a fantastic time at the K2 Insider Conference in Denver. Our fearless leader Chris Geier at K2 really put together a top notch conference getting a bunch of really busy people to converge together and share their knowledge of K2. I hope we have another one next year as I will be a first to sign up. It was actually really important that many of us met as K2 BlackPearl has just come out, several us had some implementations under our belts and we needed to discuss what was good, what was bad and get knowledge of what is to come.

There were some things I was not supposed to write about,

Here are some notes I have from the conference:

K2 Website

First if you did not know, their website has been rebranded http://www.k2.com/. It is a little brighter now <g>

Trivia:

• How did they get the "K2" name? It was a build number in one of their very early iterations.
• Where did "BlackPearl" come from? Adriaan's kid is a big fan of Pirates of the Carribean <g>

New Releases

Build 803 is due out in roughly three weeks. It will have:

• Full 64 bit support
• New installer and deployment improvements
• Better support for internationalization
• New patches to support
  
  
  • Visual Studio Stability (that was needed)
  • Workflow performance (some of the stats were impressive)
  • Reporting
  • SmartObject Performance
  • SmartObject Lifecycle
    

Build 805 is due out a little after that with support for various other know bugs, performance, TFS integration, and SAP integration.

Build 806 is due in the Junish timeframe and will have:

• VS 2008 support
• K2 2003 interop support and migration wizards
• K2 Studio RTM (looks really cool)
• Process Portals RTM (was absolutely needed and now provides process managers with a dashboard in SharePoint to see many important information about all processes running)
• Out of Office RTM
• Dynamic SQL Service (rolled in from Blackmarket – Seb is the man)
• Web Service/WCF Workflow template – that is needed!
• BizTalk Templates for 2006 R2 (I used them with K2 2003 – so needed)
  

K2 Studio

I mentioned this above but this is a new skin to the develop K2 workflows. This one will have an office look and feel and will empower business developers to compose workflows outside of visual studio. Visio allows you to compose just the workflow but this environment will allow you to create full solutions with a SmartObject designer. When I saw this I finally got the impression that we can now start turning this over to the business users. However many of my workflows still require custom code events which is not possible in this environment. Maybe if it had a way to reference external dll methods.

Process Management

They will be providing a new functionality that will allow you to change process instances that are now in flight in Visual Studio. Several of us had reservations about process version controlling to ensure that developers are not making tons of ad-hoc modifications to their process without making changes back to their baseline. Regardless this functionality is absolutely needed. I see this being very valuable for that purpose. The great thing about this one is that you can apply patches before an error were to occur. I still recommend that if you have areas of highly volatile business rules that you break those out into sub-processes and isolate them instead of making changes directly to the instances in production.

Process Portals

I have known about it for some time however I have not had the time to break open my Tech Preview VPC. I did see it flight and it awesome. Now they have a site template with some new webparts, reporting service reports, etc that allow you to create process management dashboards in SharePoint. All tasks for the current user and all users are finally visible. There is a process overview web part and you can see all running instances. The K2 Workspace is great for developers, administrators, etc. but sending the business users to this portal is much better. I believe I heard them say they plan to create some new versions of the K2 Workspace based on what type of person you are (admin, dev, or business) and I believe that is the right direction.

End Point Publishing

This got a couple of us really giddy. Now we can publish SmartObjects and Processes and WCF endpoints which can be looked up in a UDDI catalog. This will enable other platforms like Java to really easily start hooking into K2.

SilverLight Designer

Yes they are rebuilding their Ajax designer in SilverLight and it really slick. It will allow business users to compose workflows in SharePoint without any client on their machine. It can pretty much be hosted anywhere and has an extensible model. I thought it was really cool how they reversed the paradigm for composing the workflow. Instead of activity/event centric it is based on the user. So a project manager needs to do this while a program director needs to do that. It would be cool of the K2 Studio environment could also adopt this paradigm for process design.

K2 Connect

They are now providing the ability to hook in SAP using their SmartObject architecture. It was really impressive to see how through the GUI they could map into the BAPI, select data fields and surface the data up through a SmartObject. They plan to start leveraging this to access other tradition ERP datasources.

Things I still like to see

• Tools to assist with the deployment of environments, notifications, custom reports, roles, etc. between disparate environments. It is not uncommon for developers to not have access to production environments.
  
• Some better approaches to doing migrations. For instance since the migration tool is still beta for a while and clients want to do BlackPearl implementations allow them to create new BlackPearl servers. Then when the migration tool is prime time, then migrate the processes to a different clean BlackPearl server. Then migrate the processes over to the permanent BlackPearl server (as if you are doing disaster recovery).
  

Wrox Book

Yes – there is a K2 Wrox that is going to be written. I am responsible for the SmartObjects, Deployment and InfoPath chapters. My hand was sore after signing all the dang paperwork.

There are still tons of other things that were discussed which I will discuss when I am allowed <g>

First K2 Insider Conference April 7 to 11

I am heading off tomorrow to the first K2 Insider Conference tomorrow morning in Denver. I am frantically trying to get some client work done, then I got TPS reports for several projects and an ice hockey in three hours. Ugh. Several went out skiing this weekend but it was just going to work for me given the amount of work on my plate.

I will be making a presentation on some case studies we have done at RDA and then Gabrielle Malherbe, Colin Murphy and myself will be leading a workshop on SmartObjects. There is actually a blog we wrote together that I still have not had time to go over and get up. In it we discuss and debate the positioning of SmartObjects in the Enterprise Architecture. A conclusion (which I fought against for while) was that SmartObjects have many characteristics of SOA but it is not SOA. Even though SmartObjects allow you to easily (and I mean easily) provision and harvest data into your workflows it still does not replace a data access layer. It is just a piece of it. LINQ, Entity, and .Net Data Services provide a richer object query and object relation mapping tools than what you can get with K2 SmartObjects. K2 SmartObjects have their place and right now I compare it to the Business Data Catalog (BDC) that is provided in MOSS. However my opinion on this may change over the week J

ListInstance FeatureID Error

You may be tempted some day to write a ListInstance feature which will create an instance of a list for a list definition that has already been deployed. In this Post in the section entitled "List Instance Feature" I specifically said take note of the FeatureId in my example but did not explain it.

The documentation on MSDN is not great but it does explain how the FeatureId should be used.

• The FeatureId must be set to the FeatureId of the list definition Feature. So in my example of a document library it would be 00BFEA71-E717-4E80-AA17-D0C71B360101.
• If you have a custom list definition, it would be the FeatureId used for the list definition.
• If you have both the list definition and the list instance defined in the same Feature you do not need to specify the FeatureId.

The error you will get in the logs when trying to activate the ListInstance Feature tells you pretty much nothing.

Also not the TemplateType but be set to the ID that is in the list definition.