Jorge's Quest For Knowledge!

All You Need To Know About Identity And Security On-Premises And In The Cloud. It's Just Like An Addiction, The More You Have, The More You Want To Have!

(2015-02-28) FIM Fails With "Unable To Create New WorkflowInstance For WorkflowDefinition"

Posted by Jorge on 2015-02-28


Currently my AD/ADFS/FIM environment was running on one server including all other required software components such as Exchnage, SQL and Sharepoint. I wanted a more flexible configuration so I decided to split up the different roles and reinstalled the environment including everything in it. Configurations were either redone or migrated. In the case of FIM I migrated the configuration using the known migration tools/scripts/methods. At the same time I also reconfigured a few things (e.g. Workflows) in FIM. Because of the additional changes I tested the worflows that were impacted as those were using custom activities. Everything appeared to be fine! The FIM Portal in the new environment was running on Sharepoint Foundation 2013 and with regards to FIM the latest available build at the time of writing was being used.

To uniquely identity every object in the FIM Portal an wanted to assign a global ID to group and user objects. User objects already had that so I only needed to configure stuff for groups. For this task I was using OCG’s Function Evaluator to generate a GUID. As I already had that activity configured somewhere I decided to copy that specific configuration and put it in a new workflow which I configured with "Run On Policy Update (ROPU). Then I created a set and a transition based MPR. I disabled the MPR, committed that and re-enabled it again. Thinking to be done very quickly I checked all requests and I expected to see only completes. WRONG I only saw tons of PostProcessingErrors as shown below.

image

Figure 1: PostProcessing Errors For Requests

Looking at the one of the System Event Request

image

Figure 2: PostProcessing Errors For Requests

Looking at the event viewer

image

Figure 2: "Unable To Create New WorkflowInstance For WorkflowDefinition" Error In The "Forefront Event Viewer" Event Viewer Log

Microsoft.ResourceManagement: Microsoft.ResourceManagement.Workflow.Hosting.WorkflowManagerException: Unable to create new WorkflowInstance for WorkflowDefinition ‘cb6b64a3-9785-4dfe-aab2-5b4e8338eee2’. —> System.Workflow.ComponentModel.Compiler.WorkflowValidationFailedException: The workflow failed validation.
   at System.Workflow.Runtime.Hosting.DefaultWorkflowLoaderService.CreateInstance(XmlReader workflowDefinitionReader, XmlReader rulesReader)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.LoadRootActivity(String xomlText, String rulesText, Byte[] xomlHashCode, Boolean createDefinition, Boolean initForRuntime)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.MruCache.GetOrGenerateDefinition(Type type, String xomlText, String rulesText, Byte[] md5Codes, Boolean initForRuntime, Boolean& exist)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.GetRootActivity(String xomlText, String rulesText, Boolean createNew, Boolean initForRuntime)
   at System.Workflow.Runtime.WorkflowRuntime.InitializeExecutor(Guid instanceId, CreationContext context, WorkflowExecutor executor, WorkflowInstance workflowInstance)
   at System.Workflow.Runtime.WorkflowRuntime.Load(Guid key, CreationContext context, WorkflowInstance workflowInstance)
   at System.Workflow.Runtime.WorkflowRuntime.GetWorkflowExecutor(Guid instanceId, CreationContext context)
   at System.Workflow.Runtime.WorkflowRuntime.InternalCreateWorkflow(CreationContext context, Guid instanceId)
   at System.Workflow.Runtime.WorkflowRuntime.CreateWorkflow(XmlReader workflowDefinitionReader, XmlReader rulesReader, Dictionary`2 namedArgumentValues, Guid instanceId)
   at Microsoft.ResourceManagement.Workflow.Hosting.WorkflowManager.StartWorkflowInstance(Guid workflowInstanceIdentifier, KeyValuePair`2[] additionalParameters)
   — End of inner exception stack trace —
   at Microsoft.ResourceManagement.Utilities.ExceptionManager.ThrowException(Exception exception)
   at Microsoft.ResourceManagement.Workflow.Hosting.WorkflowManager.StartWorkflowInstance(Guid workflowInstanceIdentifier, KeyValuePair`2[] additionalParameters)

After seeing this error, I tried an already existing workflow using the same function evaluator and the same activity configuration. That worked perfectly, therefore there was nothing wrong with the AIC configuration of the activity and the corresponding DLLs.

Because the previous errors did not help in any way and to see if I could get more information I decided to enable tracing for the FIM Service as described through the following blog post "(2013-11-01) Advanced Logging, Event Tracing Or Troubleshooting Within FIM Components".

Basically you comment the default "Default Diagnostics configuration" and remove the comment from the "Advanced Diagnostics Configuration (Full Diagnostics configuration)" and restart the FIM service. Then retrigger the workflow. By the way: DO NOT forget to disable tracing afterwards!!!

REMARK: make sure to specify a path where the log files should be created!!!

In the trace log ("Microsoft.ResourceManagement.Service_tracelog.txt") I saw the following:

    ThreadId=4
    DateTime=2014-12-14T19:18:50.2133779Z
Microsoft.ResourceManagement Error: 3 : Microsoft.ResourceManagement: Microsoft.ResourceManagement.Workflow.Hosting.WorkflowManagerException: Unable to create new WorkflowInstance for WorkflowDefinition ‘cb6b64a3-9785-4dfe-aab2-5b4e8338eee2’. —> System.Workflow.ComponentModel.Compiler.WorkflowValidationFailedException: The workflow failed validation.
   at System.Workflow.Runtime.Hosting.DefaultWorkflowLoaderService.CreateInstance(XmlReader workflowDefinitionReader, XmlReader rulesReader)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.LoadRootActivity(String xomlText, String rulesText, Byte[] xomlHashCode, Boolean createDefinition, Boolean initForRuntime)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.MruCache.GetOrGenerateDefinition(Type type, String xomlText, String rulesText, Byte[] md5Codes, Boolean initForRuntime, Boolean& exist)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.GetRootActivity(String xomlText, String rulesText, Boolean createNew, Boolean initForRuntime)
   at System.Workflow.Runtime.WorkflowRuntime.InitializeExecutor(Guid instanceId, CreationContext context, WorkflowExecutor executor, WorkflowInstance workflowInstance)
   at System.Workflow.Runtime.WorkflowRuntime.Load(Guid key, CreationContext context, WorkflowInstance workflowInstance)
   at System.Workflow.Runtime.WorkflowRuntime.GetWorkflowExecutor(Guid instanceId, CreationContext context)
   at System.Workflow.Runtime.WorkflowRuntime.InternalCreateWorkflow(CreationContext context, Guid instanceId)
   at System.Workflow.Runtime.WorkflowRuntime.CreateWorkflow(XmlReader workflowDefinitionReader, XmlReader rulesReader, Dictionary`2 namedArgumentValues, Guid instanceId)
   at Microsoft.ResourceManagement.Workflow.Hosting.WorkflowManager.StartWorkflowInstance(Guid workflowInstanceIdentifier, KeyValuePair`2[] additionalParameters)
   — End of inner exception stack trace —

…….

Microsoft.ResourceManagement Verbose: 0 : Entered RequestDispatcher with Request Object; RequestIdentifier ’80cf1669-6933-4047-9db2-9c8032d95177′.
    ThreadId=12
    DateTime=2014-12-14T19:18:54.9869166Z
Microsoft.ResourceManagement Verbose: 0 : Add request ’80cf1669-6933-4047-9db2-9c8032d95177′ to cache with RequestStatus ‘Committed’.
    ThreadId=12
    DateTime=2014-12-14T19:18:54.9869166Z
Microsoft.ResourceManagement Information: 1 : RequestDispatcher enter processing pipeline;  RequestIdentifier ’80cf1669-6933-4047-9db2-9c8032d95177′; Operation ‘SystemEvent’; Object ‘Group’; RequestStatus ‘Committed’.
    ThreadId=12
    DateTime=2014-12-14T19:18:54.9869166Z
Microsoft.ResourceManagement Verbose: 0 : RequestDispatcher is processing RequestIdentifier ’80cf1669-6933-4047-9db2-9c8032d95177′ for a ‘SystemEvent’ operation on object ‘Group’ with RequestStatus ‘Committed’.
    ThreadId=12
    DateTime=2014-12-14T19:18:55.1744221Z
Microsoft.ResourceManagement Verbose: 0 : Request ’80cf1669-6933-4047-9db2-9c8032d95177′ status was updated in-memory from ‘Committed’ to ‘PostProcessing’.
    ThreadId=12
    DateTime=2014-12-14T19:18:55.1744221Z
Microsoft.ResourceManagement Verbose: 0 : Request ’80cf1669-6933-4047-9db2-9c8032d95177′ updates have been persisted to permanent storage.
    ThreadId=12
    DateTime=2014-12-14T19:18:57.5766555Z
Microsoft.ResourceManagement Error: 3 : WorkflowManager could not deserialize XOML definition: ‘<ns0:SequentialWorkflow ActorId="00000000-0000-0000-0000-000000000000" RequestId="00000000-0000-0000-0000-000000000000" x:Name="SequentialWorkflow" TargetId="00000000-0000-0000-0000-000000000000" WorkflowDefinitionId="00000000-0000-0000-0000-000000000000" xmlns:ns1="clr-namespace:T4FIM.FunctionEvaluator;Assembly=T4FIM.FunctionEvaluator, Version=4.0.0.2, Culture=neutral, PublicKeyToken=1cff8ccc43c5c5ec" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/workflow&quot; xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml&quot; xmlns:ns0="clr-namespace:Microsoft.ResourceManagement.Workflow.Activities;Assembly=Microsoft.ResourceManagement, Version=4.1.3613.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
    <ns1:WorkflowPart CurrentRequest="{x:Null}" readTarget_Resource="{x:Null}" Destination="globalEmployeeID" DestinationType="Target" ControlTitle="Set GlobalEmployeeID" FunctionText="Replace(Right(Left(GUID(),37),36),&quot;-&quot;,&quot;&quot;)" WorkflowEnabled="True" resolveGrammarActivity_ResolvedExpression="{x:Null}" readReferenceAndAttributes_Resource="{x:Null}" LogMessage="GlobalEmployeeID" x:Name="authenticationGateActivity1" resolveGrammarActivity_GrammarExpression="{x:Null}">
        <ns2:ReceiveActivity.WorkflowServiceAttributes xmlns:ns2="clr-namespace:System.Workflow.Activities;Assembly=System.WorkflowServices, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
            <ns2:WorkflowServiceAttributes ConfigurationName="OCG.Workflow" Name="Workflow" />
        </ns2:ReceiveActivity.WorkflowServiceAttributes>
    </ns1:WorkflowPart>
</ns0:SequentialWorkflow>’.
    ThreadId=12
    DateTime=2014-12-14T19:18:57.6702055Z
Microsoft.ResourceManagement Information: 1 : 347 :  : Invalid Element ‘ReceiveActivity.WorkflowServiceAttributes’ found while deserializing an object of type ‘T4FIM.FunctionEvaluator.WorkflowPart’.
    ThreadId=12
    DateTime=2014-12-14T19:18:57.6702055Z
Microsoft.ResourceManagement Information: 1 : 347 :  : Invalid data found while deserializing an object of type ‘T4FIM.FunctionEvaluator.WorkflowPart’.
    ThreadId=12
    DateTime=2014-12-14T19:18:57.6702055Z
Microsoft.ResourceManagement Error: 3 : Microsoft.ResourceManagement.Workflow.Hosting.WorkflowManagerException: Unable to create new WorkflowInstance for WorkflowDefinition ‘cb6b64a3-9785-4dfe-aab2-5b4e8338eee2’. —> System.Workflow.ComponentModel.Compiler.WorkflowValidationFailedException: The workflow failed validation.
   at System.Workflow.Runtime.Hosting.DefaultWorkflowLoaderService.CreateInstance(XmlReader workflowDefinitionReader, XmlReader rulesReader)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.LoadRootActivity(String xomlText, String rulesText, Byte[] xomlHashCode, Boolean createDefinition, Boolean initForRuntime)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.MruCache.GetOrGenerateDefinition(Type type, String xomlText, String rulesText, Byte[] md5Codes, Boolean initForRuntime, Boolean& exist)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.GetRootActivity(String xomlText, String rulesText, Boolean createNew, Boolean initForRuntime)
   at System.Workflow.Runtime.WorkflowRuntime.InitializeExecutor(Guid instanceId, CreationContext context, WorkflowExecutor executor, WorkflowInstance workflowInstance)
   at System.Workflow.Runtime.WorkflowRuntime.Load(Guid key, CreationContext context, WorkflowInstance workflowInstance)
   at System.Workflow.Runtime.WorkflowRuntime.GetWorkflowExecutor(Guid instanceId, CreationContext context)
   at System.Workflow.Runtime.WorkflowRuntime.InternalCreateWorkflow(CreationContext context, Guid instanceId)
   at System.Workflow.Runtime.WorkflowRuntime.CreateWorkflow(XmlReader workflowDefinitionReader, XmlReader rulesReader, Dictionary`2 namedArgumentValues, Guid instanceId)
   at Microsoft.ResourceManagement.Workflow.Hosting.WorkflowManager.StartWorkflowInstance(Guid workflowInstanceIdentifier, KeyValuePair`2[] additionalParameters)
   — End of inner exception stack trace —

Based upon my experience, when I see something like "deserializing", I know something is wrong with the XOML definition of the new workflow. The question is WHY, as another workflow with the same configuration did work! Because of I decided to compare the XOML definition of workflow that worked and the one that failed

XOML Definition for working WFW

<ns0:SequentialWorkflow x:Name="SequentialWorkflow" ActorId="00000000-0000-0000-0000-000000000000" WorkflowDefinitionId="00000000-0000-0000-0000-000000000000" RequestId="00000000-0000-0000-0000-000000000000" TargetId="00000000-0000-0000-0000-000000000000" xmlns:ns1="clr-namespace:T4FIM.FunctionEvaluator;Assembly=T4FIM.FunctionEvaluator, Version=4.0.0.2, Culture=neutral, PublicKeyToken=1cff8ccc43c5c5ec" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/workflow&quot; xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml&quot; xmlns:ns0="clr-namespace:Microsoft.ResourceManagement.Workflow.Activities;Assembly=Microsoft.ResourceManagement, Version=4.1.3508.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">

    <ns1:WorkflowPart x:Name="authenticationGateActivity16" WorkflowEnabled="True" CurrentRequest="{x:Null}" FunctionText="Replace(Right(Left(GUID(),37),36),&quot;-&quot;,&quot;&quot;)" resolveGrammarActivity_GrammarExpression="{x:Null}" ControlTitle="Set GlobalEmployeeID" LogMessage="GlobalEmployeeID" readTarget_Resource="{x:Null}" readReferenceAndAttributes_Resource="{x:Null}" resolveGrammarActivity_ResolvedExpression="{x:Null}" Destination="globalEmployeeID" DestinationType="Target">
        <ns2:ReceiveActivity.WorkflowServiceAttributes xmlns:ns2="clr-namespace:System.Workflow.Activities;Assembly=System.WorkflowServices, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
            <ns2:WorkflowServiceAttributes Name="Workflow" ConfigurationName="OCG.Workflow" />
        </ns2:ReceiveActivity.WorkflowServiceAttributes>
    </ns1:WorkflowPart>

</ns0:SequentialWorkflow>

XOML Definition for not working WFW

<ns0:SequentialWorkflow ActorId="00000000-0000-0000-0000-000000000000" RequestId="00000000-0000-0000-0000-000000000000" x:Name="SequentialWorkflow" TargetId="00000000-0000-0000-0000-000000000000" WorkflowDefinitionId="00000000-0000-0000-0000-000000000000" xmlns:ns1="clr-namespace:T4FIM.FunctionEvaluator;Assembly=T4FIM.FunctionEvaluator, Version=4.0.0.2, Culture=neutral, PublicKeyToken=1cff8ccc43c5c5ec" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/workflow&quot; xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml&quot; xmlns:ns0="clr-namespace:Microsoft.ResourceManagement.Workflow.Activities;Assembly=Microsoft.ResourceManagement, Version=4.1.3613.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
    <ns1:WorkflowPart CurrentRequest="{x:Null}" readTarget_Resource="{x:Null}" Destination="globalEmployeeID" DestinationType="Target" ControlTitle="Set GlobalEmployeeID" FunctionText="Replace(Right(Left(GUID(),37),36),&quot;-&quot;,&quot;&quot;)" WorkflowEnabled="True" resolveGrammarActivity_ResolvedExpression="{x:Null}" readReferenceAndAttributes_Resource="{x:Null}" LogMessage="GlobalEmployeeID" x:Name="authenticationGateActivity1" resolveGrammarActivity_GrammarExpression="{x:Null}">
        <ns2:ReceiveActivity.WorkflowServiceAttributes xmlns:ns2="clr-namespace:System.Workflow.Activities;Assembly=System.WorkflowServices, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
            <ns2:WorkflowServiceAttributes ConfigurationName="OCG.Workflow" Name="Workflow" />
        </ns2:ReceiveActivity.WorkflowServiceAttributes>
    </ns1:WorkflowPart>
</ns0:SequentialWorkflow>

The differences are highlighted in yellow and green. The yellow difference is taken care of with redirect bindings in the "Microsoft.ResourceManagement.Service.exe.config" file. Instead of updating every workflow, the redirect binding is updated. So that’s not the problem.

The green difference is rather weird! Why is the working workflow using .NET version 3.5.0.0 and why is the not working working workflow using .NET 4.0.0.0?

After changing the XOML definition of the not working workflow from .NET version 4.0.0.0 to 3.5.0.0 the workflow started working again! Yeah!

After fixing the version, I reconfigured the XOML definition through the Normal View. It failed again. Damn! Looking at the XOML definition it reverted back to 4.0.0.0. WTF! This appeared to happen for every workflow with custom activities. Apparently every time I now edit a workflow through the normal View, I need to recheck the XOML definition through the Advanced View to make sure the .NET version is correctly configured to 3.5.0.0. This just makes me sad, as for sure this is forgotten to be checked and stuff breaks again.

So if you workflow suddenly stops working and throws a similar error this could be the reason!

After changing the .NET version in the XOML definition, committing that, disabling the MPR and re-enabling it, everything worked again!

image

Figure 3: Different Request Statuses While The FIM Service Is Executing The ROPU Enabled Workflow

image

Figure 4: Completed Request Status After The FIM Service Has Executed The ROPU Enabled Workflow

YEAH!

Cheers,
Jorge
———————————————————————————————
* This posting is provided "AS IS" with no warranties and confers no rights!
* Always evaluate/test yourself before using/implementing this!
* DISCLAIMER:
https://jorgequestforknowledge.wordpress.com/disclaimer/
———————————————————————————————
############### Jorge’s Quest For Knowledge #############
#########
http://JorgeQuestForKnowledge.wordpress.com/ ########
———————————————————————————————

One Response to “(2015-02-28) FIM Fails With "Unable To Create New WorkflowInstance For WorkflowDefinition"”

  1. Hello Jorge,
    run into the issue some time ago, and found the same solution somewhere in the Net.
    This issues seems to occur only on inplementations with SharePoint 2013.
    On my side this occurs only on Approval Workflows, I put this “Edit .Net Version” in the description to not forget it.
    Action workflows even with custom activities (Powershell on my side) are not affected.

    -Peter

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: