(2020-04-19) It’s That Time Again, You Need To Roll-Over The Certificates In ADFS

This weekend I was involved in rolling over the ADFS Token Signing and Token Encryption certificates while a huge amount of application were connected using WS-Federation, SAML or OAuth. With regards to rolling over the certificates, only the WS-Federation and SAML based apps are impacted, not the OAuth apps.

Short answer: preparation, communication and automation are KEY!

–

This ADFS system was designed and implemented by me years ago. In the beginning, there were just a few apps and it grew gradually. After a moment it was starting to grow like crazy. More and more app devs smelled the federated SSO cool stuff and wanted that for their app(s). Why? Default globally supported protocols between organizations, apps, etc, and all based on federation. Supported by ADFS, many other on-premises federation systems and cloud based systems, such as e.g. Azure AD. The focus of this blog post is apps connected to ADFS as the federation provider, for both downstream identity providers and upstream service providers.

–

When I design this system, after the technology, I also designed the required processes to make sure the technology was used in the correct way. If you do not design the processes in addition to the technology, it becomes a mess and because certificates are used it will drop dead, exactly 1 year after installing/configuring it. Why will it drop dead after 1 year? The default Token Signing and Token Encryption certificates are ADFS managed and expire after 1 year. All apps not (being) able to consume to the federation metadata URL automatically will drop dead if no action is taken in time! And rushing with no process in place is a good cocktail for possible failure. If you have a handful of apps, you should be OK, but if you have 100+, or 200+ or even more apps connected to ADFS, you seriously need a process to get this done in time and correctly. Regarding the process, that ADFS has internally, please see: (2013-05-14) ADFS Managed Certificates Supporting Auto Certificate Rollover

–

Processes:

• Backup/Export of the complete ADFS configuration
• Provisioning of applications onto ADFS
• Recertification of any connected application, keep or deprovision
• Notification of expiring certificates in ADFS (about 3 months ahead of time)
• Certificate change in ADFS
• Validation of specified e-mail addresses against AD

–

During the onboarding of ANY application the following was recorded:

1. Does an application support automatic consumption of the federation metadata URL or does it support its consumption manually through an XML file?
2. Does an application support just 1 or 2 Token Signing certificates from an identity provider?
3. When an application supports the consumption of the federation metadata file, AND it supports only 1 Token Signing certificate, which certificate will it consume when it reads the federation metadata file as it contains multiple Token Signing certificates from the identity provider
4. E-mail addresses from at least 3 “owners”
5. E-mail address of either a functional mailbox or distribution list with a support team behind

–

[AD.1]

If it supports automatic consumption of federation metadata through a URL (for ADFS: /federationmetadata/2007-06/federationmetadata.xml">https://<federation service FQDN>/federationmetadata/2007-06/federationmetadata.xml) it processes the information automatically. If it does not, you may need to first export the federation metadata to an XML file to be able to import it into the application. Even though the federation metadata URL is used, it does not mean an application uses it automatically on a regular basis (e.g. every day). It is possible that a admin need to run a script, or push some button or whatever

–

[AD.2]

If an application supports 2 Token Signing certificates from an identity provider, a smooth certificate rollover is supported without ANY pain. If only 1 Token Signing certificate is supported it means, the certificate must be replaced at (nearly) the same time as the ADFS admin switches the primary and secondary Token Signing certificate

–

[AD.3]

This scenario should be very rare, but unfortunately some apps do it incorrectly. When the federation metadata from the identity provider is imported, and only 1 Token Signing certificate is used, and the federation metadata from the identity provider contains multiple Token Signing certificates, which is consumed (used) by the application? The first Token Signing certificate occurrence, or the last occurrence? If it is the last occurrence, then that is good, but the federation metadata XML file must only be imported at the same time when the switch is made in ADFS. If it is the first occurrence, then the app is not following best practice and it will break as soon as the switch in ADFS is done.

ADFS by default:

• Publishes ALL Token Signing certificates in the federation metadata
• The oldest Token Signing certificate is published first and the newest Token Signing certificate is published last
• Switching Token Signing certificates in ADFS DOES NOT change the order of the Token Signing certificates in the federation metadata
• Publishes only the primary (active) Token Encryption certificate in the federation metadata
• Switching Token Encryption certificates in ADFS removes the old Token Encryption certificate from the federation metadata, and adds the new Token Encryption certificate to the federation metadata
• ADFS signs the federation metadata with the primary (active) Token Signing certificate

So if the first occurrence of the Token Signing certificate is consumed, after the switch in ADFS, the only way to solve this is:

• Edit the federation metadata XML file and remove the first occurrence of the Token Signing certificate (old Token Signing certificate, marked as “use=’signing’”, appears 3 times in the federation metadata XML), and keep the last occurrence (the newest) Token Signing certificate
  OR
• Remove the oldest Token Signing certificate as a secondary from ADFS, export the federation metadata to an XML file and import it into the application

Which option you use depends on whether or not an application checks for the signing of the federation metadata. If it does not, you can use either option, but it is easier to edit and you can keep the old Token Signing certificate as secondary if for whatever unlikely reason want to rollback (not recommended when having many applications (define “many”!) as it may turn out in a logistics nightmare

–

[AD.4]

A minimum of three e-mail addresses are required to support the yearly recertification process where a decision is made to either keep or deprovision the federation trust in ADFS. A minimum of three mail addresses as people come and people go (internally or externally), and chances are very low that all three people move at the same time (unless there is a reorganization, sigh!). If any of the listed people changes roles

–

[AD.5]

The e-mail address of a functional mailbox or distribution list with people from a (technical) support team behind it. This is mainly used by the “Certificate Change in ADFS” process. The chance of failure must be brought to a minimum by not having dependency of e-mail addresses from people, but rather have e-mail address of a functional mailbox or distribution list with people behind it. The people from a (technical) support team are responsible to updating the application with the new Token Signing when ADFS changes the certificate.

–

Now having all this information, you need to be able to store it somewhere that makes it easily manageable and provides the ability to process it through scripting (automation). Within ADFS, every single trust has a notes or description field, and the field can perfectly be used to store this information. However it is a string formatted field with a maximum of 3000 characters. I thought of the idea to implement an XML structure to be able to read from and write to it as needed. Works like magic as we can script against it too. But how about using CMDB? Well, we wanted to be certain of the up-to-date-ness of the information, we wanted to be able to script against it and we wanted it as close as possible to the trust configuration in ADFS.

–

Now, the whole certificate change in ADFS looks like something similar to:

1. Define "Product Backlog Items (PBI)" on the backlog, plan into sprints and assign to people
2. Organize initial meeting with colleagues to explain process, answer any questions and discuss PBIs
3. Plan meeting with (higher) management and deliver a presentation about the certificate change, discussing what, when, why, how, current state (number of trusts), risks, impact, rollback (none!), etc.
4. Raise an RFC to change certificates in ADFS and push to achieve the highest risk/impact category possible
5. Check and fix if needed e-mail addresses from application owners and support teams
6. Request new certificates for ADFS (SSL, Token Signing and Token Encryption)
7. Install all new certificates in the personal certificate store of all ADFS servers
8. Install the new SSL certificate in the personal certificate store of all WAP servers
9. Configure the new Token Signing certificate as a secondary Token Signing certificate
10. Configure the new Token Encryption certificate as a secondary Token Encryption certificate
11. Configure the new Svc Comm/SSL Certificate on all ADFS servers and WAP servers
12. Recheck and fix if needed e-mail addresses from application owners and support teams
13. Send the "First Notification" to all support teams and application owners
14. Notify the global service committee about the upcoming change, provide information and request to chase teams/owners
15. Send the "Reminder Notification" to all support teams and application owners
16. Switch the Token Signing/Encryption Certificate in ADFS
17. Send the "Switch Executed Notification" to all support teams and application owners
18. Guide and support support teams and application owners that request or needed guidance
19. Organize a closure meeting with colleagues to discuss any feedback from support teams and application owners, colleagues involved, and if applicable possible improvements for the process
20. Process the feedback and improvements from the closure meeting for the next time

–

End Result: SUCCESS! Oops I did it again!

–

I have done this multiple times now, and even though there is nothing to it (that’s what I think!) it is always exciting due to the huge amount of applications involved! The most difficult part is explaining this to non-technical people where you have to say things like: “there is no rollback”, “there will be issues with apps”, “fix forward only”, etc. It just scare the crap out of them and I understand that. Another one not to forget is, the support team, and sometimes even the vendor or service provideers, managing the application do not have the technical knowledge regarding federation and SSO

–

But, been there, done that, got the t-shirt, all sizes and all colours!

–

–

Cheers,
Jorge

————————————————————————————————————————————————————-
This posting is provided "AS IS" with no warranties and confers no rights!
Always evaluate/test everything yourself first before using/implementing this in production!
This is today’s opinion/technology, it might be different tomorrow and will definitely be different in 10 years!
DISCLAIMER: https://jorgequestforknowledge.wordpress.com/disclaimer/
————————————————————————————————————————————————————-
########################### Jorge’s Quest For Knowledge ##########################
#################### http://JorgeQuestForKnowledge.wordpress.com/ ###################
————————————————————————————————————————————————————-

(2014-03-14) Gathering Architectural Details From Your ADFS Infrastructure – ADFS Certs

If ADFS was installed in the past by someone else and there is little to no documentation, how do you know if you are you using ADFS Managed Self-Signed Certs Or CA Issued Certs for your Token Signing and Token Encryption certs? Keep reading to find out how to determine that!

–

ADFS Managed Self-Signed Certs Or CA Issued Certs For Your Token Signing and Token Encryption Certs?

To determine if ADFS Managed Self-Signed certificates are being used for the Token Signing cert and the Token Encryption cert, you can execute the "Get-ADFSProperties" CMDlet and check if the property "AutoCertificateRollOver" is set to True. When set to True ADFS Managed Self-Signed certificates are being used. When set to False ADFS Managed Self-Signed certificates are NOT being used and you are then using CA issued certificates. All yellow marked properties below are used if the property "AutoCertificateRollOver" is set to True. For more information see (2013-05-14) ADFS Managed Certificates Supporting Auto Certificate Rollover.

One important thing to remember is:

• The value for CertificateSharingContainer is NOT populated when ADFS is in StandAlone mode
• The value for CertificateSharingContainer is populated when ADFS most likely in Farm mode and when ADFS Managed Self-Signed certificates are actually being used OR have been used (after changing to CA issued certs)
• If CA issued certs are being used, and you were using ADFS Managed Self-Signed certificates before that, you can safely delete the Certificate Sharing Container from AD (test first and be able to fully restore deleted objects as needed!)

Figure 1: Are ADFS Managed Self-Signed Certificates Being Used For The Token Signing Cert And The Token Encryption Cert Or Not?

–

With ADFS Managed Self-Signed Certificates the "Issued To" and the "Issued By" are always the same. The part before the dash (-) is always "ADFS Signing" and the part after the dash (-) is always "<the FQDN of the Federation Service>"

Figure 2: Are ADFS Managed Self-Signed Certificate For The Token Signing Cert

–

With ADFS Managed Self-Signed Certificates the "Issued To" and the "Issued By" are always the same. The part before the dash (-) is always "ADFS Encryption" and the part after the dash (-) is always "<the FQDN of the Federation Service>"

Figure 3: Are ADFS Managed Self-Signed Certificate For The Token Encryption Cert

–

Either the ADFS MMC or the PowerShell CMDlet "Get-ADFSCertificate" will tell you which certificates are in use by ADFS. Also see: (2013-05-13) Certificates Used In Active Directory Federation Services (ADFS) v2.x

–

Cheers,
Jorge
———————————————————————————————
* This posting is provided "AS IS" with no warranties and confers no rights!
* Always evaluate/test yourself before using/implementing this!
* DISCLAIMER: https://jorgequestforknowledge.wordpress.com/disclaimer/
———————————————————————————————
############### Jorge’s Quest For Knowledge #############
######### http://JorgeQuestForKnowledge.wordpress.com/ ########
———————————————————————————————

(2013-05-14) ADFS Managed Certificates Supporting Auto Certificate Rollover

In the following blog post Certificates Used In Active Directory Federation Services (ADFS) v2.x I wrote about the certificates used by ADFS v2.x. In summary, you can use CA issued certificates for all certificates required by ADFS or you can use ADFS managed self-signed certificates for both the Token Signing Certificate and the Token Decryption Certificate. When automated certificate rollover has been enabled, ADFS managed self-signed certificates for both the Token Signing Certificate and the Token Decryption Certificate will be used!

–

Let’s first have a look at the characteristics self-signed certificates in general. It is considered a bad practice to use self-signed certificates because by default those certificates are not trusted by any entity. Self-signed certificates do not leverage CRLs and cannot be revoked, only replaced. And to my knowledge you cannot store its private key in an HSM. However, in the case of ADFS, the valid usage of a self-signed certificate, and also CA Issued Certificates, (for both the Token Signing Certificate at the IdP STS and the Token Decryption Certificate at the SP STS/App) is determined by the existence of the certificate, its configuration as the certificate for a specific purpose and the publication of such in the federation metadata. Something to be aware of is that other parties, such as both STS’es and applications, still must become aware of the information available in the federation metadata, either manually or automatically.

–

The usage of ADFS managed self-signed certificates lowers the management overhead that comes with the usage of certificates in common. ADFS managed self-signed certificates, and their corresponding private keys are not stored on individual servers, but rather in the ADFS configuration database. In the context of security, all ADFS STS servers and all servers hosting the ADFS configuration database should be secured, physically and logically, in at least the same way as securing RWDCs. Access to these servers should ONLY be allowed to highly skilled knowledgeable and authorized people that understand the working of ADFS. Why? Let’s do a comparison!

In a Windows world, resources are secured through Windows Security Groups. To gain access to those resources you must be a member of the corresponding Windows Security Groups and the DC must hand out an Access Token that contains those Windows Security Groups. If the RWDC has been compromised, the person that compromised the RWDC will also have access to any resource secured by Windows Security Groups because it will be able to generate Access Tokens with any required Windows Security Groups.

In a Claims Based Authorization world, resources are secured through specific Claims. To gain access to those resources you must be able to provide the corresponding Claims and the ADFS STS must hand out a Security Token that contains those Claims. If the ADFS STS has been compromised, the person that compromised the ADFS STS will also have access to any resource secured by Claims because it will be able to generate Security Tokens with any required Claims.

To be able to test this all, I have changed the default values to lower values as shown below. Later on I will explain each and every property.

Figure 1: Automated Certificate Rollover Related Properties

–

AutoCertificateRollover

When set to True, Automated Certificate Rollover is enabled for both the Token Signing Certificate and Token Decryption Certificate. CA issued certificates cannot be used for both.

When set to False, Automated Certificate Rollover is disabled for both the Token Signing Certificate and Token Decryption Certificate. CA issued certificates must be used for both.

–

CertificateThresholdMultiplier

The default value is 1440 minutes (24 hours/1 day).

This is a multiplier unit used by the other parameters and it is measured in the number of minutes.

–

CertificateRolloverInterval

The default value is 720 minutes (12 hours).

This is the frequency in minutes in which the federation service checks the current ADFS managed self-signed certificates.

–

CertificateDuration

The default value is 365 units.

This is the validity period of ADFS managed self-signed certificate. Defined by the number of multiplier units. In this custom scenario, the validity of the ADFS managed self-signed certificate is 10 units of 30 minutes, or 300 minutes (5 hours).

–

CertificateGenerationThreshold

The default value is 20 units.

This is the period of time before expiration of the current primary certificate, a new ADFS managed self-signed certificate is generated and marked as secondary. Defined by the number of multiplier units. In this custom scenario, a new ADFS managed self-signed will be generated and marked as secondary 2 hours (4 units of 30 minutes, 120 minutes) before the expiration of the current primary ADFS managed self-signed certificate.

–

CertificatePromotionThreshold

The default value is 5 units.

This is the period of time after which the previously newly generated ADFS managed self-signed certificate will be promoted to become the primary ADFS Managed self-signed certificate. Defined by the number of multiplier units. In this custom scenario, a new ADFS managed self-signed will be promoted to primary 1 hour (2 units of 30 minutes, 60 minutes) after it was generated.

–

CertificateCriticalThreshold

The default value is 2 units.

This is the period of time before expiration of the current primary certificate, a new ADFS managed self-signed certificate is generated and marked as primary immediately. This only occurs is everything else before failed and went down the drain. Defined by the number of multiplier units. In this custom scenario, a new ADFS managed self-signed will be generated and promoted to primary immediately half an hour (1 units of 30 minutes, 30 minutes) before the expiration of the current primary ADFS managed self-signed certificate.

–

When you put everything in a diagram, it looks like as shown below.

Figure 2: Automated Certificate Rollover Related Properties In A Diagram

–

As an example I will show the corresponding event IDs, for every moment, except the one for “CertificateCriticalThreshold”

Figure 3: The Generation Of The Very First ADFS Managed Self-Signed Token Decryption Certificate (“Valid From”)

–

Figure 4: The Generation Of The Very First ADFS Managed Self-Signed Token Signing Certificate (“Valid From”)

–

Figure 5: The Federation Service Checking The Current Certificates (CertificateRollOverInterval)

–

Figure 6: The Generation Of The (New) Next ADFS Managed Self-Signed Token Decryption Certificate And Configured As Secondary (CertificateGenerationThreshold)

–

Figure 7: The Generation Of The (New) Next ADFS Managed Self-Signed Token Signing Certificate And Configured As Secondary (CertificateGenerationThreshold)

–

Figure 8: The Promotion Of The Secondary ADFS Managed Self-Signed Token Decryption Certificate To Primary

–

Figure 9: The Promotion Of The Secondary ADFS Managed Self-Signed Token Signing Certificate To Primary

–

Figure 10: The Removal Of The (Old/Expired) Secondary ADFS Managed Self-Signed Token Signing Certificate (“Valid To”)

–

Figure 11: The Removal Of The (Old/Expired) Secondary ADFS Managed Self-Signed Token Decryption Certificate (“Valid To”)

–

For additional details also see:

• AD FS 2.0: Understanding AutoCertificateRollover Threshold Properties

–

Cheers,
Jorge
———————————————————————————————
* This posting is provided "AS IS" with no warranties and confers no rights!
* Always evaluate/test yourself before using/implementing this!
* DISCLAIMER: https://jorgequestforknowledge.wordpress.com/disclaimer/
———————————————————————————————
############### Jorge’s Quest For Knowledge #############
######### http://JorgeQuestForKnowledge.wordpress.com/ ########
———————————————————————————————