Jorge's Quest For Knowledge!

About Windows Server, ADDS, ADFS, Azure AD, FIM/MIM & AADSync (Just Like An Addiction, The More You Have, The More You Want To Have!)

(2015-02-28) FIM Fails With "Unable To Create New WorkflowInstance For WorkflowDefinition"

Posted by Jorge on 2015-02-28


Currently my AD/ADFS/FIM environment was running on one server including all other required software components such as Exchnage, SQL and Sharepoint. I wanted a more flexible configuration so I decided to split up the different roles and reinstalled the environment including everything in it. Configurations were either redone or migrated. In the case of FIM I migrated the configuration using the known migration tools/scripts/methods. At the same time I also reconfigured a few things (e.g. Workflows) in FIM. Because of the additional changes I tested the worflows that were impacted as those were using custom activities. Everything appeared to be fine! The FIM Portal in the new environment was running on Sharepoint Foundation 2013 and with regards to FIM the latest available build at the time of writing was being used.

-

To uniquely identity every object in the FIM Portal an wanted to assign a global ID to group and user objects. User objects already had that so I only needed to configure stuff for groups. For this task I was using OCG’s Function Evaluator to generate a GUID. As I already had that activity configured somewhere I decided to copy that specific configuration and put it in a new workflow which I configured with "Run On Policy Update (ROPU). Then I created a set and a transition based MPR. I disabled the MPR, committed that and re-enabled it again. Thinking to be done very quickly I checked all requests and I expected to see only completes. WRONG I only saw tons of PostProcessingErrors as shown below.

image

Figure 1: PostProcessing Errors For Requests

-

Looking at the one of the System Event Request

image

Figure 2: PostProcessing Errors For Requests

-

Looking at the event viewer

image

Figure 2: "Unable To Create New WorkflowInstance For WorkflowDefinition" Error In The "Forefront Event Viewer" Event Viewer Log

-

Microsoft.ResourceManagement: Microsoft.ResourceManagement.Workflow.Hosting.WorkflowManagerException: Unable to create new WorkflowInstance for WorkflowDefinition ‘cb6b64a3-9785-4dfe-aab2-5b4e8338eee2′. —> System.Workflow.ComponentModel.Compiler.WorkflowValidationFailedException: The workflow failed validation.
   at System.Workflow.Runtime.Hosting.DefaultWorkflowLoaderService.CreateInstance(XmlReader workflowDefinitionReader, XmlReader rulesReader)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.LoadRootActivity(String xomlText, String rulesText, Byte[] xomlHashCode, Boolean createDefinition, Boolean initForRuntime)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.MruCache.GetOrGenerateDefinition(Type type, String xomlText, String rulesText, Byte[] md5Codes, Boolean initForRuntime, Boolean& exist)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.GetRootActivity(String xomlText, String rulesText, Boolean createNew, Boolean initForRuntime)
   at System.Workflow.Runtime.WorkflowRuntime.InitializeExecutor(Guid instanceId, CreationContext context, WorkflowExecutor executor, WorkflowInstance workflowInstance)
   at System.Workflow.Runtime.WorkflowRuntime.Load(Guid key, CreationContext context, WorkflowInstance workflowInstance)
   at System.Workflow.Runtime.WorkflowRuntime.GetWorkflowExecutor(Guid instanceId, CreationContext context)
   at System.Workflow.Runtime.WorkflowRuntime.InternalCreateWorkflow(CreationContext context, Guid instanceId)
   at System.Workflow.Runtime.WorkflowRuntime.CreateWorkflow(XmlReader workflowDefinitionReader, XmlReader rulesReader, Dictionary`2 namedArgumentValues, Guid instanceId)
   at Microsoft.ResourceManagement.Workflow.Hosting.WorkflowManager.StartWorkflowInstance(Guid workflowInstanceIdentifier, KeyValuePair`2[] additionalParameters)
   — End of inner exception stack trace —
   at Microsoft.ResourceManagement.Utilities.ExceptionManager.ThrowException(Exception exception)
   at Microsoft.ResourceManagement.Workflow.Hosting.WorkflowManager.StartWorkflowInstance(Guid workflowInstanceIdentifier, KeyValuePair`2[] additionalParameters)

-

After seeing this error, I tried an already existing workflow using the same function evaluator and the same activity configuration. That worked perfectly, therefore there was nothing wrong with the AIC configuration of the activity and the corresponding DLLs.

-

Because the previous errors did not help in any way and to see if I could get more information I decided to enable tracing for the FIM Service as described through the following blog post "(2013-11-01) Advanced Logging, Event Tracing Or Troubleshooting Within FIM Components".

Basically you comment the default "Default Diagnostics configuration" and remove the comment from the "Advanced Diagnostics Configuration (Full Diagnostics configuration)" and restart the FIM service. Then retrigger the workflow. By the way: DO NOT forget to disable tracing afterwards!!!

-

REMARK: make sure to specify a path where the log files should be created!!!

-

In the trace log ("Microsoft.ResourceManagement.Service_tracelog.txt") I saw the following:

    ThreadId=4
    DateTime=2014-12-14T19:18:50.2133779Z
Microsoft.ResourceManagement Error: 3 : Microsoft.ResourceManagement: Microsoft.ResourceManagement.Workflow.Hosting.WorkflowManagerException: Unable to create new WorkflowInstance for WorkflowDefinition ‘cb6b64a3-9785-4dfe-aab2-5b4e8338eee2′. —> System.Workflow.ComponentModel.Compiler.WorkflowValidationFailedException: The workflow failed validation.
   at System.Workflow.Runtime.Hosting.DefaultWorkflowLoaderService.CreateInstance(XmlReader workflowDefinitionReader, XmlReader rulesReader)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.LoadRootActivity(String xomlText, String rulesText, Byte[] xomlHashCode, Boolean createDefinition, Boolean initForRuntime)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.MruCache.GetOrGenerateDefinition(Type type, String xomlText, String rulesText, Byte[] md5Codes, Boolean initForRuntime, Boolean& exist)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.GetRootActivity(String xomlText, String rulesText, Boolean createNew, Boolean initForRuntime)
   at System.Workflow.Runtime.WorkflowRuntime.InitializeExecutor(Guid instanceId, CreationContext context, WorkflowExecutor executor, WorkflowInstance workflowInstance)
   at System.Workflow.Runtime.WorkflowRuntime.Load(Guid key, CreationContext context, WorkflowInstance workflowInstance)
   at System.Workflow.Runtime.WorkflowRuntime.GetWorkflowExecutor(Guid instanceId, CreationContext context)
   at System.Workflow.Runtime.WorkflowRuntime.InternalCreateWorkflow(CreationContext context, Guid instanceId)
   at System.Workflow.Runtime.WorkflowRuntime.CreateWorkflow(XmlReader workflowDefinitionReader, XmlReader rulesReader, Dictionary`2 namedArgumentValues, Guid instanceId)
   at Microsoft.ResourceManagement.Workflow.Hosting.WorkflowManager.StartWorkflowInstance(Guid workflowInstanceIdentifier, KeyValuePair`2[] additionalParameters)
   — End of inner exception stack trace —

…….

Microsoft.ResourceManagement Verbose: 0 : Entered RequestDispatcher with Request Object; RequestIdentifier ’80cf1669-6933-4047-9db2-9c8032d95177′.
    ThreadId=12
    DateTime=2014-12-14T19:18:54.9869166Z
Microsoft.ResourceManagement Verbose: 0 : Add request ’80cf1669-6933-4047-9db2-9c8032d95177′ to cache with RequestStatus ‘Committed’.
    ThreadId=12
    DateTime=2014-12-14T19:18:54.9869166Z
Microsoft.ResourceManagement Information: 1 : RequestDispatcher enter processing pipeline;  RequestIdentifier ’80cf1669-6933-4047-9db2-9c8032d95177′; Operation ‘SystemEvent'; Object ‘Group'; RequestStatus ‘Committed’.
    ThreadId=12
    DateTime=2014-12-14T19:18:54.9869166Z
Microsoft.ResourceManagement Verbose: 0 : RequestDispatcher is processing RequestIdentifier ’80cf1669-6933-4047-9db2-9c8032d95177′ for a ‘SystemEvent’ operation on object ‘Group’ with RequestStatus ‘Committed’.
    ThreadId=12
    DateTime=2014-12-14T19:18:55.1744221Z
Microsoft.ResourceManagement Verbose: 0 : Request ’80cf1669-6933-4047-9db2-9c8032d95177′ status was updated in-memory from ‘Committed’ to ‘PostProcessing’.
    ThreadId=12
    DateTime=2014-12-14T19:18:55.1744221Z
Microsoft.ResourceManagement Verbose: 0 : Request ’80cf1669-6933-4047-9db2-9c8032d95177′ updates have been persisted to permanent storage.
    ThreadId=12
    DateTime=2014-12-14T19:18:57.5766555Z
Microsoft.ResourceManagement Error: 3 : WorkflowManager could not deserialize XOML definition: ‘<ns0:SequentialWorkflow ActorId="00000000-0000-0000-0000-000000000000" RequestId="00000000-0000-0000-0000-000000000000" x:Name="SequentialWorkflow" TargetId="00000000-0000-0000-0000-000000000000" WorkflowDefinitionId="00000000-0000-0000-0000-000000000000" xmlns:ns1="clr-namespace:T4FIM.FunctionEvaluator;Assembly=T4FIM.FunctionEvaluator, Version=4.0.0.2, Culture=neutral, PublicKeyToken=1cff8ccc43c5c5ec" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/workflow&quot; xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml&quot; xmlns:ns0="clr-namespace:Microsoft.ResourceManagement.Workflow.Activities;Assembly=Microsoft.ResourceManagement, Version=4.1.3613.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
    <ns1:WorkflowPart CurrentRequest="{x:Null}" readTarget_Resource="{x:Null}" Destination="globalEmployeeID" DestinationType="Target" ControlTitle="Set GlobalEmployeeID" FunctionText="Replace(Right(Left(GUID(),37),36),&quot;-&quot;,&quot;&quot;)" WorkflowEnabled="True" resolveGrammarActivity_ResolvedExpression="{x:Null}" readReferenceAndAttributes_Resource="{x:Null}" LogMessage="GlobalEmployeeID" x:Name="authenticationGateActivity1" resolveGrammarActivity_GrammarExpression="{x:Null}">
        <ns2:ReceiveActivity.WorkflowServiceAttributes xmlns:ns2="clr-namespace:System.Workflow.Activities;Assembly=System.WorkflowServices, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
            <ns2:WorkflowServiceAttributes ConfigurationName="OCG.Workflow" Name="Workflow" />
        </ns2:ReceiveActivity.WorkflowServiceAttributes>
    </ns1:WorkflowPart>
</ns0:SequentialWorkflow>’.
    ThreadId=12
    DateTime=2014-12-14T19:18:57.6702055Z
Microsoft.ResourceManagement Information: 1 : 347 :  : Invalid Element ‘ReceiveActivity.WorkflowServiceAttributes’ found while deserializing an object of type ‘T4FIM.FunctionEvaluator.WorkflowPart’.
    ThreadId=12
    DateTime=2014-12-14T19:18:57.6702055Z
Microsoft.ResourceManagement Information: 1 : 347 :  : Invalid data found while deserializing an object of type ‘T4FIM.FunctionEvaluator.WorkflowPart’.
    ThreadId=12
    DateTime=2014-12-14T19:18:57.6702055Z
Microsoft.ResourceManagement Error: 3 : Microsoft.ResourceManagement.Workflow.Hosting.WorkflowManagerException: Unable to create new WorkflowInstance for WorkflowDefinition ‘cb6b64a3-9785-4dfe-aab2-5b4e8338eee2′. —> System.Workflow.ComponentModel.Compiler.WorkflowValidationFailedException: The workflow failed validation.
   at System.Workflow.Runtime.Hosting.DefaultWorkflowLoaderService.CreateInstance(XmlReader workflowDefinitionReader, XmlReader rulesReader)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.LoadRootActivity(String xomlText, String rulesText, Byte[] xomlHashCode, Boolean createDefinition, Boolean initForRuntime)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.MruCache.GetOrGenerateDefinition(Type type, String xomlText, String rulesText, Byte[] md5Codes, Boolean initForRuntime, Boolean& exist)
   at System.Workflow.Runtime.WorkflowDefinitionDispenser.GetRootActivity(String xomlText, String rulesText, Boolean createNew, Boolean initForRuntime)
   at System.Workflow.Runtime.WorkflowRuntime.InitializeExecutor(Guid instanceId, CreationContext context, WorkflowExecutor executor, WorkflowInstance workflowInstance)
   at System.Workflow.Runtime.WorkflowRuntime.Load(Guid key, CreationContext context, WorkflowInstance workflowInstance)
   at System.Workflow.Runtime.WorkflowRuntime.GetWorkflowExecutor(Guid instanceId, CreationContext context)
   at System.Workflow.Runtime.WorkflowRuntime.InternalCreateWorkflow(CreationContext context, Guid instanceId)
   at System.Workflow.Runtime.WorkflowRuntime.CreateWorkflow(XmlReader workflowDefinitionReader, XmlReader rulesReader, Dictionary`2 namedArgumentValues, Guid instanceId)
   at Microsoft.ResourceManagement.Workflow.Hosting.WorkflowManager.StartWorkflowInstance(Guid workflowInstanceIdentifier, KeyValuePair`2[] additionalParameters)
   — End of inner exception stack trace —

-

Based upon my experience, when I see something like "deserializing", I know something is wrong with the XOML definition of the new workflow. The question is WHY, as another workflow with the same configuration did work! Because of I decided to compare the XOML definition of workflow that worked and the one that failed

-

XOML Definition for working WFW

<ns0:SequentialWorkflow x:Name="SequentialWorkflow" ActorId="00000000-0000-0000-0000-000000000000" WorkflowDefinitionId="00000000-0000-0000-0000-000000000000" RequestId="00000000-0000-0000-0000-000000000000" TargetId="00000000-0000-0000-0000-000000000000" xmlns:ns1="clr-namespace:T4FIM.FunctionEvaluator;Assembly=T4FIM.FunctionEvaluator, Version=4.0.0.2, Culture=neutral, PublicKeyToken=1cff8ccc43c5c5ec" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/workflow&quot; xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml&quot; xmlns:ns0="clr-namespace:Microsoft.ResourceManagement.Workflow.Activities;Assembly=Microsoft.ResourceManagement, Version=4.1.3508.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">

    <ns1:WorkflowPart x:Name="authenticationGateActivity16" WorkflowEnabled="True" CurrentRequest="{x:Null}" FunctionText="Replace(Right(Left(GUID(),37),36),&quot;-&quot;,&quot;&quot;)" resolveGrammarActivity_GrammarExpression="{x:Null}" ControlTitle="Set GlobalEmployeeID" LogMessage="GlobalEmployeeID" readTarget_Resource="{x:Null}" readReferenceAndAttributes_Resource="{x:Null}" resolveGrammarActivity_ResolvedExpression="{x:Null}" Destination="globalEmployeeID" DestinationType="Target">
        <ns2:ReceiveActivity.WorkflowServiceAttributes xmlns:ns2="clr-namespace:System.Workflow.Activities;Assembly=System.WorkflowServices, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
            <ns2:WorkflowServiceAttributes Name="Workflow" ConfigurationName="OCG.Workflow" />
        </ns2:ReceiveActivity.WorkflowServiceAttributes>
    </ns1:WorkflowPart>

</ns0:SequentialWorkflow>

-

XOML Definition for not working WFW

<ns0:SequentialWorkflow ActorId="00000000-0000-0000-0000-000000000000" RequestId="00000000-0000-0000-0000-000000000000" x:Name="SequentialWorkflow" TargetId="00000000-0000-0000-0000-000000000000" WorkflowDefinitionId="00000000-0000-0000-0000-000000000000" xmlns:ns1="clr-namespace:T4FIM.FunctionEvaluator;Assembly=T4FIM.FunctionEvaluator, Version=4.0.0.2, Culture=neutral, PublicKeyToken=1cff8ccc43c5c5ec" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/workflow&quot; xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml&quot; xmlns:ns0="clr-namespace:Microsoft.ResourceManagement.Workflow.Activities;Assembly=Microsoft.ResourceManagement, Version=4.1.3613.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
    <ns1:WorkflowPart CurrentRequest="{x:Null}" readTarget_Resource="{x:Null}" Destination="globalEmployeeID" DestinationType="Target" ControlTitle="Set GlobalEmployeeID" FunctionText="Replace(Right(Left(GUID(),37),36),&quot;-&quot;,&quot;&quot;)" WorkflowEnabled="True" resolveGrammarActivity_ResolvedExpression="{x:Null}" readReferenceAndAttributes_Resource="{x:Null}" LogMessage="GlobalEmployeeID" x:Name="authenticationGateActivity1" resolveGrammarActivity_GrammarExpression="{x:Null}">
        <ns2:ReceiveActivity.WorkflowServiceAttributes xmlns:ns2="clr-namespace:System.Workflow.Activities;Assembly=System.WorkflowServices, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">
            <ns2:WorkflowServiceAttributes ConfigurationName="OCG.Workflow" Name="Workflow" />
        </ns2:ReceiveActivity.WorkflowServiceAttributes>
    </ns1:WorkflowPart>
</ns0:SequentialWorkflow>

-

The differences are highlighted in yellow and green. The yellow difference is taken care of with redirect bindings in the "Microsoft.ResourceManagement.Service.exe.config" file. Instead of updating every workflow, the redirect binding is updated. So that’s not the problem.

The green difference is rather weird! Why is the working workflow using .NET version 3.5.0.0 and why is the not working working workflow using .NET 4.0.0.0?

After changing the XOML definition of the not working workflow from .NET version 4.0.0.0 to 3.5.0.0 the workflow started working again! Yeah!

-

After fixing the version, I reconfigured the XOML definition through the Normal View. It failed again. Damn! Looking at the XOML definition it reverted back to 4.0.0.0. WTF! This appeared to happen for every workflow with custom activities. Apparently every time I now edit a workflow through the normal View, I need to recheck the XOML definition through the Advanced View to make sure the .NET version is correctly configured to 3.5.0.0. This just makes me sad, as for sure this is forgotten to be checked and stuff breaks again.

-

So if you workflow suddenly stops working and throws a similar error this could be the reason!

-

After changing the .NET version in the XOML definition, committing that, disabling the MPR and re-enabling it, everything worked again!

image

Figure 3: Different Request Statuses While The FIM Service Is Executing The ROPU Enabled Workflow

-

image

Figure 4: Completed Request Status After The FIM Service Has Executed The ROPU Enabled Workflow

-

YEAH!

-

Cheers,
Jorge
———————————————————————————————
* This posting is provided "AS IS" with no warranties and confers no rights!
* Always evaluate/test yourself before using/implementing this!
* DISCLAIMER:
https://jorgequestforknowledge.wordpress.com/disclaimer/
———————————————————————————————
############### Jorge’s Quest For Knowledge #############
#########
http://JorgeQuestForKnowledge.wordpress.com/ ########
———————————————————————————————

Posted in Forefront Identity Manager (FIM) Portal, Troubleshooting, Uncategorized, Workflow | Leave a Comment »

(2015-02-24) Migrating ADFS Databases From SQL To WID

Posted by Jorge on 2015-02-24


For whatever reason you may be interested in migrating your ADFS databases, currently hosted on WID, to SQL server. Information regarding that can be read through the following links:

-

Now some of you might think: "is it possible to migrate the ADFS databases, currently hosted on SQL server, to WID?"

-

Now why would you want to do that?

  • SQL features related to ADFS are not being used (Token Replay Prevention and Artifact Resolution)
  • Saving costs on SQL server licenses
  • Simplifying DR and high availability for ADFS

-

Now to answer the questions…NO, it is not possible to migrate ADFS databases from SQL server to WID!

-

I tried this myself. I did a backup of the ADFS databases on SQL server and then tried to restore those same databases on WID.

image

Figure 1: SQL Management Studio Connected To WID (Local) and SQL Server (Remote)

-

As soon as you try to do the restore, you will see an error similar to the following

image

Figure 2: Failing To Restore A Database Previously Hosted On SQL Server To WID

-

Now why is this? The simple answer is: WID basically uses an older version of SQL than SQL server itself. You can restore a database from a lower version of SQL to a higher version of SQL, but you CANNOT restore a database from a higher version of SQL to a lower version of SQL!

You can read more about this here.

-

Cheers,
Jorge
———————————————————————————————
* This posting is provided "AS IS" with no warranties and confers no rights!
* Always evaluate/test yourself before using/implementing this!
* DISCLAIMER:
https://jorgequestforknowledge.wordpress.com/disclaimer/
———————————————————————————————
############### Jorge’s Quest For Knowledge #############
#########
http://JorgeQuestForKnowledge.wordpress.com/ ########
———————————————————————————————

Posted in Active Directory Federation Services (ADFS), DB On SQL, DB On WID, Uncategorized, Upgrading | Leave a Comment »

(2015-02-20) Finding Attributes Configured To Be Never Audited

Posted by Jorge on 2015-02-20


When an attribute is defined with the bit 2^8 (=256) in the searchFlags property, the attribute is configured to be never audited. When “Directory Services Access” is enabled AND the correct SACL has been configured on objects for a specific action on the attribute by specific actors, access (read or write) those attributes will be audited by default. When “Directory Services Changes” is enabled AND the correct SACL has been configured on objects for a specific action on the attribute by specific actors, the values (before and after the change) of those attributes will also be audited by default. To disable the behavior of also auditing values for one or more attributes, those attributes can be configured not to be change audited at all. This might be of interest for those attributes that have been configured as confidential and auditing has been enabled on the object.

-

ADFIND

ADFIND -h R1FSRWDC1.IAMTEC.NET -schema -f "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=256))" -dn

OR

ADFIND -h R1FSRWDC1.IAMTEC.NET -bit -schema -f "(&(objectClass=attributeSchema)(searchFlags:AND:=256))" -dn

-

image

Figure 1: Example Output

-

AD PoSH Module

Get-ADObject -Server R1FSRWDC1.IAMTEC.NET -SearchBase $((Get-ADRootDSE).schemaNamingContext) -LDAPFilter "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=256))" | %{$_.DistinguishedName}

-

ADSI Through PoSH

$targetDC = "R1FSRWDC1.IAMTEC.NET"
$rootDSE = [ADSI]"LDAP://$targetDC/RootDSE"
$schemaNamingContext = $rootDSE.schemaNamingContext
$search = New-Object DirectoryServices.DirectorySearcher([ADSI]"")
$Search.SearchRoot = "LDAP://$targetDC/$schemaNamingContext"
$search.filter = "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=256))"
$search.FindAll() | %{$_.Properties.distinguishedname}

-

PS: replace the FQDN of the DC with your info

-

PS: the opposite of this query can be found by replacing (searchFlags:1.2.840.113556.1.4.803:=256) with (!(searchFlags:1.2.840.113556.1.4.803:=256))

-

More information:

-

Cheers,
Jorge
———————————————————————————————
* This posting is provided "AS IS" with no warranties and confers no rights!
* Always evaluate/test yourself before using/implementing this!
* DISCLAIMER:
https://jorgequestforknowledge.wordpress.com/disclaimer/
———————————————————————————————
############### Jorge’s Quest For Knowledge #############
#########
http://JorgeQuestForKnowledge.wordpress.com/ ########
———————————————————————————————

Posted in Active Directory Domain Services (ADDS), AD Queries, Schema, Tooling/Scripting | Leave a Comment »

(2015-02-16) Finding Attributes Marked To Be Copied When Duplicating The Object

Posted by Jorge on 2015-02-16


When an attribute is defined with the bit 2^4 (=16) in the searchFlags property, the attribute is configured to be copied when duplicating the object. When using template objects with generic information that applies to all newly created objects, it can be interesting to configure the new attribute in such a way, so that its attribute value is copied from the reference/template object to the new created object. If this flag is configured, it is the responsibility of the tool used, and NOT the directory service, to honor the setting or not and behave accordingly. Make sure the tools used, interpreter this flag correctly when set.

-

ADFIND

ADFIND -h R1FSRWDC1.IAMTEC.NET -schema -f "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=16))" -dn

OR

ADFIND -h R1FSRWDC1.IAMTEC.NET -bit -schema -f "(&(objectClass=attributeSchema)(searchFlags:AND:=16))" -dn

OR

ADFIND -sc COPY -dn (sorted output: ADFIND -sc COPYL -dn)

-

image

Figure 1: Example Output

-

AD PoSH Module

Get-ADObject -Server R1FSRWDC1.IAMTEC.NET -SearchBase $((Get-ADRootDSE).schemaNamingContext) -LDAPFilter "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=16))" | %{$_.DistinguishedName}

-

ADSI Through PoSH

$targetDC = "R1FSRWDC1.IAMTEC.NET"
$rootDSE = [ADSI]"LDAP://$targetDC/RootDSE"
$schemaNamingContext = $rootDSE.schemaNamingContext
$search = New-Object DirectoryServices.DirectorySearcher([ADSI]"")
$Search.SearchRoot = "LDAP://$targetDC/$schemaNamingContext"
$search.filter = "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=16))"
$search.FindAll() | %{$_.Properties.distinguishedname}

-

PS: replace the FQDN of the DC with your info

-

PS: the opposite of this query can be found by replacing (searchFlags:1.2.840.113556.1.4.803:=16) with (!(searchFlags:1.2.840.113556.1.4.803:=16))

-

More information:

-

Cheers,
Jorge
———————————————————————————————
* This posting is provided "AS IS" with no warranties and confers no rights!
* Always evaluate/test yourself before using/implementing this!
* DISCLAIMER:
https://jorgequestforknowledge.wordpress.com/disclaimer/
———————————————————————————————
############### Jorge’s Quest For Knowledge #############
#########
http://JorgeQuestForKnowledge.wordpress.com/ ########
———————————————————————————————

Posted in Active Directory Domain Services (ADDS), AD Queries, Schema, Tooling/Scripting | Leave a Comment »

(2015-02-10) Finding Attributes With A Subtree Index

Posted by Jorge on 2015-02-10


When an attribute is defined with the bit 2^6 (=64) in the searchFlags property, the attribute is configured to have a subtree index. This index allows Virtual List View (VLV) operations to be more efficient when using a specific attribute as the sort key. In a (very) large DS it also prevents the VLV query to terminate with the "Critical Extension Unavailable" error as a subtree index prevents the use of the special internal table called TEMP. The size of this table can vary, but by default the maximum number of entries is configured to be 10000 (the MaxTempTableSize setting of the Default Query Policy). VLV is a GUI technique that can be used when, ordered lists containing a large number of entries, need to be displayed. When the LDAP protocol is extended to use VLV (request: 2.16.840.1.113730.3.4.9; response: 2.16.840.1.113730.3.4.10), a window that contains a small number of visible list entries is drawn. The visible portion of the list can be relocated to different points in the list by means of scrolling, slider bars, and cursor keys as well as PAGE UP/DOWN keys. The user experience is that the full list can be browsed at will, even though it might contain millions of entries. In fact, the complete list contents are never required at any one time. Rather than retrieve the complete list from wherever it is stored (typically, from disk or a remote server), the server retrieves only the information that is required to display the part of the list that is currently in view on the client. This improves both the response from the server and the speed at which results are listed at the client.

-

ADFIND

ADFIND -h R1FSRWDC1.IAMTEC.NET -schema -f "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=64))" -dn

OR

ADFIND -h R1FSRWDC1.IAMTEC.NET -bit -schema -f "(&(objectClass=attributeSchema)(searchFlags:AND:=64))" -dn

OR

ADFIND -sc SINDEXED -dn (sorted output: ADFIND -sc SINDEXEDL -dn)

-

image

Figure 1: Example Output

-

AD PoSH Module

Get-ADObject -Server R1FSRWDC1.IAMTEC.NET -SearchBase $((Get-ADRootDSE).schemaNamingContext) -LDAPFilter "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=64))" | %{$_.DistinguishedName}

-

ADSI Through PoSH

$targetDC = "R1FSRWDC1.IAMTEC.NET"
$rootDSE = [ADSI]"LDAP://$targetDC/RootDSE"
$schemaNamingContext = $rootDSE.schemaNamingContext
$search = New-Object DirectoryServices.DirectorySearcher([ADSI]"")
$Search.SearchRoot = "LDAP://$targetDC/$schemaNamingContext"
$search.filter = "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=64))"
$search.FindAll() | %{$_.Properties.distinguishedname}

-

PS: replace the FQDN of the DC with your info

-

PS: the opposite of this query can be found by replacing (searchFlags:1.2.840.113556.1.4.803:=64) with (!(searchFlags:1.2.840.113556.1.4.803:=64))

-

More information:

-

Cheers,
Jorge
———————————————————————————————
* This posting is provided "AS IS" with no warranties and confers no rights!
* Always evaluate/test yourself before using/implementing this!
* DISCLAIMER:
https://jorgequestforknowledge.wordpress.com/disclaimer/
———————————————————————————————
############### Jorge’s Quest For Knowledge #############
#########
http://JorgeQuestForKnowledge.wordpress.com/ ########
———————————————————————————————

Posted in Active Directory Domain Services (ADDS), AD Queries, Indexing, Schema, Tooling/Scripting | Leave a Comment »

(2015-02-06) Finding Attributes With A Tuple Index

Posted by Jorge on 2015-02-06


When an attribute is defined with the bit 2^5 (=32) in the searchFlags property, the attribute is configured to have a tuple index. A regular index, assuming the attribute syntax also supports it, also is optimized to support queries using trailing wildcards (e.g. ‘name=jorge*’). To be able to support medial queries, where wildcards in queries are anywhere but at the end of the string (e.g. ‘name=*pinto’ or ‘name=jorge*pinto’), a tuple index should be enabled for the attribute(s) if (an) application(s) is/are having bad/poor medial query performance. An important thing to be aware of is that tuple indexes will increase the AD database (NTDS.DIT) more than regular indexes. In addition, the performance impact of new attribute insertion depends on the number of attributes being inserted at once. The more attributes being inserted, the higher the impact.

-

ADFIND

ADFIND -h R1FSRWDC1.IAMTEC.NET -schema -f "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=32))" -dn

OR

ADFIND -h R1FSRWDC1.IAMTEC.NET -bit -schema -f "(&(objectClass=attributeSchema)(searchFlags:AND:=32))" -dn

OR

ADFIND -sc TUPLE -dn (sorted output: ADFIND -sc TUPLEL -dn)

-

image

Figure 1: Example Output

-

AD PoSH Module

Get-ADObject -Server R1FSRWDC1.IAMTEC.NET -SearchBase $((Get-ADRootDSE).schemaNamingContext) -LDAPFilter "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=32))" | %{$_.DistinguishedName}

-

ADSI Through PoSH

$targetDC = "R1FSRWDC1.IAMTEC.NET"
$rootDSE = [ADSI]"LDAP://$targetDC/RootDSE"
$schemaNamingContext = $rootDSE.schemaNamingContext
$search = New-Object DirectoryServices.DirectorySearcher([ADSI]"")
$Search.SearchRoot = "LDAP://$targetDC/$schemaNamingContext"
$search.filter = "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=32))"
$search.FindAll() | %{$_.Properties.distinguishedname}

-

PS: replace the FQDN of the DC with your info

-

PS: the opposite of this query can be found by replacing (searchFlags:1.2.840.113556.1.4.803:=32) with (!(searchFlags:1.2.840.113556.1.4.803:=32))

-

More information:

-

Cheers,
Jorge
———————————————————————————————
* This posting is provided "AS IS" with no warranties and confers no rights!
* Always evaluate/test yourself before using/implementing this!
* DISCLAIMER:
https://jorgequestforknowledge.wordpress.com/disclaimer/
———————————————————————————————
############### Jorge’s Quest For Knowledge #############
#########
http://JorgeQuestForKnowledge.wordpress.com/ ########
———————————————————————————————

Posted in Active Directory Domain Services (ADDS), AD Queries, Indexing, Schema, Tooling/Scripting | Leave a Comment »

(2015-02-02) Finding Attributes With A Container Index

Posted by Jorge on 2015-02-02


When an attribute is defined with the bit 2^1 (=2) in the searchFlags property, the attribute is configured to have a containerized index. This index indexes the value of the attribute relative to the name of the container or OU. Since the index is container-based, its size will be smaller and probably faster when performing one level queries. A one level query is a query against objects in a single container/OU.

-

ADFIND

ADFIND -h R1FSRWDC1.IAMTEC.NET -schema -f "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=2))" -dn

OR

ADFIND -h R1FSRWDC1.IAMTEC.NET -bit -schema -f "(&(objectClass=attributeSchema)(searchFlags:AND:=2))" -dn

OR

ADFIND -sc CINDEXED -dn (sorted output: ADFIND -sc CINDEXEDL -dn)

-

image

Figure 1: Example Output

-

AD PoSH Module

Get-ADObject -Server R1FSRWDC1.IAMTEC.NET -SearchBase $((Get-ADRootDSE).schemaNamingContext) -LDAPFilter "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=2))" | %{$_.DistinguishedName}

-

ADSI Through PoSH

$targetDC = "R1FSRWDC1.IAMTEC.NET"
$rootDSE = [ADSI]"LDAP://$targetDC/RootDSE"
$schemaNamingContext = $rootDSE.schemaNamingContext
$search = New-Object DirectoryServices.DirectorySearcher([ADSI]"")
$Search.SearchRoot = "LDAP://$targetDC/$schemaNamingContext"
$search.filter = "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=2))"
$search.FindAll() | %{$_.Properties.distinguishedname}

-

PS: replace the FQDN of the DC with your info

-

PS: the opposite of this query can be found by replacing (searchFlags:1.2.840.113556.1.4.803:=2) with (!(searchFlags:1.2.840.113556.1.4.803:=2))

-

More information:

-

Cheers,
Jorge
———————————————————————————————
* This posting is provided "AS IS" with no warranties and confers no rights!
* Always evaluate/test yourself before using/implementing this!
* DISCLAIMER:
https://jorgequestforknowledge.wordpress.com/disclaimer/
———————————————————————————————
############### Jorge’s Quest For Knowledge #############
#########
http://JorgeQuestForKnowledge.wordpress.com/ ########
———————————————————————————————

Posted in Active Directory Domain Services (ADDS), AD Queries, Indexing, Schema, Tooling/Scripting | Leave a Comment »

(2015-01-29) Finding Attributes With A Regular Index

Posted by Jorge on 2015-01-29


When an attribute is defined with the bit 2^0 (=1) in the searchFlags property, the attribute is configured to have a regular index. When applications are querying AD on a regular basis, or very frequently, using the same attributes over and over again, attribute(s) used can be, or better yet should be, indexed to improve query performance. The values of an indexed attribute are placed in a special table in a sorted order, so that a query using the attribute is completed much faster by just looking at a subset of all the data in the DS.

-

Important points to be aware of about indexes:

  • Indexing attributes increases the AD database (NTDS.DIT), therefore it costs disk space;
  • Indexes are built locally by every DC and are not replicated;
  • Configuring an attribute for indexing impacts DC performance immediately because of the index is built right away, unless deferred indexing (W2K12 DCs and higher) has been configured. With deferred indexing configured, the indexing operation only starts after the DCs reboots (probably restarting the service is enough) or the operational attribute updateSchemaNow is triggered. Deferred indexing is configured by enabling the 18th bit on the forest-wide attribute dsHeuristics.
  • The speed at which an index is created is dependent on how much data must be indexed and also the hardware on which the DC is running;
  • Queries that contain bitwise operations on an indexed attribute (e.g. systemFlags, userAccountControl) negates the usefulness of the index. A bitwise operation cannot be directly looked up in the index table, and the entire set of values in the index will have to be enumerated and tested;
  • Queries that contain a NOT of an indexed attribute negates the use of the index for that portion of the query. It requires the enumeration of all objects in the search scope to determine which objects do not have the attribute or which objects have permissions applied that deny the trustee/actor to view the attribute value;
  • Linked attributes are implicitly indexed, therefore no need to explicitly index;
  • Indexes can be built and will work correctly for attributes with multiple values or non-unique values.

-

ADFIND

ADFIND -h R1FSRWDC1.IAMTEC.NET -schema -f "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=1))" -dn

OR

ADFIND -h R1FSRWDC1.IAMTEC.NET -bit -schema -f "(&(objectClass=attributeSchema)(searchFlags:AND:=1))" -dn

OR

ADFIND -sc INDEXED -dn (sorted output: ADFIND -sc INDEXEDL -dn)

-

image

Figure 1: Example Output

-

AD PoSH Module

Get-ADObject -Server R1FSRWDC1.IAMTEC.NET -SearchBase $((Get-ADRootDSE).schemaNamingContext) -LDAPFilter "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=1))" | %{$_.DistinguishedName}

-

ADSI Through PoSH

$targetDC = "R1FSRWDC1.IAMTEC.NET"
$rootDSE = [ADSI]"LDAP://$targetDC/RootDSE"
$schemaNamingContext = $rootDSE.schemaNamingContext
$search = New-Object DirectoryServices.DirectorySearcher([ADSI]"")
$Search.SearchRoot = "LDAP://$targetDC/$schemaNamingContext"
$search.filter = "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=1))"
$search.FindAll() | %{$_.Properties.distinguishedname}

-

PS: replace the FQDN of the DC with your info

-

PS: the opposite of this query can be found by replacing (searchFlags:1.2.840.113556.1.4.803:=1) with (!(searchFlags:1.2.840.113556.1.4.803:=1))

-

More information:

-

Cheers,
Jorge
———————————————————————————————
* This posting is provided "AS IS" with no warranties and confers no rights!
* Always evaluate/test yourself before using/implementing this!
* DISCLAIMER:
https://jorgequestforknowledge.wordpress.com/disclaimer/
———————————————————————————————
############### Jorge’s Quest For Knowledge #############
#########
http://JorgeQuestForKnowledge.wordpress.com/ ########
———————————————————————————————

Posted in Active Directory Domain Services (ADDS), AD Queries, Indexing, Schema, Tooling/Scripting | Leave a Comment »

(2015-01-25) Finding Attributes Marked To Be Preserved On Deletion Or Recycling

Posted by Jorge on 2015-01-25


When an attribute is defined with the bit 2^3 (=8) in the searchFlags property, the attribute is marked to be preserved when the object is deleted (Recycle Bin is OFF) or when the object is manually or automatically recycled (Recycle Bin is ON). Attributes that are good candidates to be preserved on deletion of the object are for example the password attributes. That helps you to get back up-and-running without setting new passwords and the communication around that when someone makes a mistake and deletes an OU with a large number of user objects. While the Recycle Bin optional feature IS NOT enabled, preserving additional attribute values on the tombstone object (the deleted object) is good if it fits a specific/required purpose. For example, by preserving the data, the impact on the user community is lowered as much as possible. HOWEVER, while the Recycle Bin optional feature IS enabled, all data on an AD object is already preserved by default on a deleted object, whether this flag is enabled or not. Enabling this flag then preserves the data when the deleted object transforms into a recycled object manually or automatically. However recycled objects cannot be undeleted, therefore there is no value in preserving this data on recycled objects. A word of advise: if you have configured additional attributes to be preserved on tombstoned objects, while the Recycle Bin optional feature IS NOT enabled, do not forget to reconfigure those attributes is you are going to enable the Recycle Bin optional feature.

-

ADFIND

ADFIND -h R1FSRWDC1.IAMTEC.NET -schema -f "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=8))" -dn

OR

ADFIND -h R1FSRWDC1.IAMTEC.NET -bit -schema -f "(&(objectClass=attributeSchema)(searchFlags:AND:=8))" -dn

OR

ADFIND -sc TOMBSTONE -dn (sorted output: ADFIND -sc TOMBSTONEL -dn)

-

image

Figure 1: Example Output

-

AD PoSH Module

Get-ADObject -Server R1FSRWDC1.IAMTEC.NET -SearchBase $((Get-ADRootDSE).schemaNamingContext) -LDAPFilter "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=8))" | %{$_.DistinguishedName}

-

ADSI Through PoSH

$targetDC = "R1FSRWDC1.IAMTEC.NET"
$rootDSE = [ADSI]"LDAP://$targetDC/RootDSE"
$schemaNamingContext = $rootDSE.schemaNamingContext
$search = New-Object DirectoryServices.DirectorySearcher([ADSI]"")
$Search.SearchRoot = "LDAP://$targetDC/$schemaNamingContext"
$search.filter = "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=8))"
$search.FindAll() | %{$_.Properties.distinguishedname}

-

PS: replace the FQDN of the DC with your info

-

PS: the opposite of this query can be found by replacing (searchFlags:1.2.840.113556.1.4.803:=8) with (!(searchFlags:1.2.840.113556.1.4.803:=8))

-

More information:

-

Cheers,
Jorge
———————————————————————————————
* This posting is provided "AS IS" with no warranties and confers no rights!
* Always evaluate/test yourself before using/implementing this!
* DISCLAIMER:
https://jorgequestforknowledge.wordpress.com/disclaimer/
———————————————————————————————
############### Jorge’s Quest For Knowledge #############
#########
http://JorgeQuestForKnowledge.wordpress.com/ ########
———————————————————————————————

Posted in Active Directory Domain Services (ADDS), AD Queries, Schema, Tooling/Scripting | Leave a Comment »

(2015-01-21) Finding Attributes Used In Ambiguous Name Resolution (ANR)

Posted by Jorge on 2015-01-21


When an attribute is defined with the bit 2^2 (=4) in the searchFlags property, the attribute is used in ambiguous name resolution (ANR). Ambiguous Name Resolution (ANR) is an efficient search algorithm in Active Directory that allows you to specify complex filters involving multiple naming-related attributes in a single (filter) clause (e.g. (ANR=jorge)). It can be used to locate objects in Active Directory when you know something about the name of the object, but not necessarily which naming attribute has the information. While ANR is usually used to locate user objects, it can be used to find any class of object in Active Directory. ANR is most used in address book lookups. The list of attributes used in ANR is different per AD version. ANR enabled attributes must be indexed.

-

ADFIND

ADFIND -h R1FSRWDC1.IAMTEC.NET -schema -f "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=4))" -dn

OR

ADFIND -h R1FSRWDC1.IAMTEC.NET -bit -schema -f "(&(objectClass=attributeSchema)(searchFlags:AND:=4))" -dn

OR

ADFIND -sc ANR -dn (sorted output: ADFIND -sc ANRL -dn)

-

image

Figure 1: Example Output

-

AD PoSH Module

Get-ADObject -Server R1FSRWDC1.IAMTEC.NET -SearchBase $((Get-ADRootDSE).schemaNamingContext) -LDAPFilter "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=4))" | %{$_.DistinguishedName}

-

ADSI Through PoSH

$targetDC = "R1FSRWDC1.IAMTEC.NET"
$rootDSE = [ADSI]"LDAP://$targetDC/RootDSE"
$schemaNamingContext = $rootDSE.schemaNamingContext
$search = New-Object DirectoryServices.DirectorySearcher([ADSI]"")
$Search.SearchRoot = "LDAP://$targetDC/$schemaNamingContext"
$search.filter = "(&(objectClass=attributeSchema)(searchFlags:1.2.840.113556.1.4.803:=4))"
$search.FindAll() | %{$_.Properties.distinguishedname}

-

PS: replace the FQDN of the DC with your info

-

PS: the opposite of this query can be found by replacing (searchFlags:1.2.840.113556.1.4.803:=4) with (!(searchFlags:1.2.840.113556.1.4.803:=4))

-

More information:

-

Cheers,
Jorge
———————————————————————————————
* This posting is provided "AS IS" with no warranties and confers no rights!
* Always evaluate/test yourself before using/implementing this!
* DISCLAIMER:
https://jorgequestforknowledge.wordpress.com/disclaimer/
———————————————————————————————
############### Jorge’s Quest For Knowledge #############
#########
http://JorgeQuestForKnowledge.wordpress.com/ ########
———————————————————————————————

Posted in Active Directory Domain Services (ADDS), AD Queries, Schema, Tooling/Scripting | Leave a Comment »

 
%d bloggers like this: