Active Directory - The UnicodePwd Mystery of AD LDS
Active Directory’s Lightweight Directory Services uses sophisticated procedure to convert and authenticate passwords.
By Frank C. Rettig
Recently, one of my clients needed to expose certain information over the Internet from Active Directory. The intent was to provide authentication (for logon purposes) to an external application without exposing those select attributes. You may be wondering what is the point of authenticating outside of the application. It puts the onus on the application users, as opposed to the provider, thus reducing the owner’s liability.
Exposing a domain controller to the Internet is normally a bad practice, whether that exposure comes directly from the production environment or through a perimeter network. The natural alternative is to place a Windows Server 2008 server with Active Directory Lightweight Directory Services (AD LDS) role running in the perimeter network.
AD LDS was originally known as Active Directory Application Mode (ADAM). It was first an add-on for Windows 2003, included with Windows 2003 R2, and is now available as a server role for Windows 2008 or Windows 7. In many ways, it’s almost like AD. As some have said, it’s a toned-down version that lacks full enterprise abilities. Implementing AD LDS is practically the same on any version of Windows, although Windows Server 2008 R2 is the best choice because it includes Windows PowerShell v2. This new version of Windows PowerShell provides enhanced manageability of AD/AD LDS attributes that simply aren’t available in the other OS versions.
Whether your focus is AD or AD LDS, both can have the unicodePwd attribute. This attribute is stored in the directory/application partition. This assumes the server has been promoted to an AD domain controller, or the proper LDIF template that has the attribute for an AD LDS server was applied.
With AD or AD LDS, applications use the Lightweight Directory Access Protocol (LDAP) version 3.0 to query the directory. To do that, the application (which could be LDP.EXE, CSVDE.EXE, LDIFDE.EXE, ADSIEdit.exe or some homegrown application) must provide the account and the password. Seems simple enough, so why then is it anything but? Providing the password is the easy part. Getting the password stored in the unicodePwd attribute is the challenge. Here’s why: With most interactive directories or applications, we’re usually prompted to enter the password in its native form. So if the password was “car,” that’s what we would enter. Such is the case when accessing a forest or domain for AD, or when accessing an AD LDS instance using ADSIedit or LDP to assign or reset a password for a particular account.
When you need to make bulk additions or changes all at once, the LDIFDE.EXE program is the tool of choice. In order to add or update the unicodePwd attribute, Microsoft requires that the value of the unicodePwd attribute be in unicode-base64 format.
So how is the unicode-base64 value calculated? This is where the fun starts. Hopefully, we’ll dispel the mystery to the unicodePwd (and other attributes that follow the same encoding) once and for all.
First, note that the following sentences are annotated. Refer to steps 1–12 in Figure 1. Microsoft requires the password be enclosed in double quotes, and then each character (including the quotes) must be converted to its unicode equivalent. This means it must first find the ASCII equivalent for each character, and then derive the hexadecimal value from each ASCII value, then pad each hex value with two zeros (because Windows conforms to UTF16) so that 16 bits are represented.
Once converted, these padded hex sets are all combined into one long string. Because base64 is based on 6 bits, this string is parsed every six characters from the left. If an incomplete hex sextet remains, it’s padded to the right with zeros until six characters are represented. Then each hex sextet is converted to binary. As is the case with binary, parsing occurs from the right, and every six characters.
Though unnecessary, for an incomplete binary sextet, pad to the left with zeros until six characters are represented (remember that leading zeros still equal zero, but this makes it easier to read since the other sets also have six characters). Now convert each parsed binary value into its numeric 11 equivalent. Use that number as the index to look up against a base64 table (Figure 2), and get its character. When a complete sextet is only made up of padded zeros, these are represented by the equal sign.
Figure 1 shows you, in steps 1–12, how the word car is converted to IgBjAGEAcgAiAA==. Right padding is required when preparing the unicode for base64, which can be seen by the text in blue highlight. As to the left padding represented by the text in red highlight, this is just to make it consistent, and easier to read when compared to the other binary sextets.
Figure 1 Steps to create a unicode-base64 password.
Figure 2 Base64 Mapping.
If you’re wondering if this funky logic is correct, get a copy of the Stringconverter.exe program from gbordier.com/gbtools/stringconverter.htm, and run the following DOS prompt command string:
stringconverter \"car\" /encode /unicode IgBjAGEAcgAiAA==
As you can see, the results are the same.
Creating Unicode-Base64 Password
For simple initial testing, using the following Excel functions (Figure 1 was created in Excel) will let you quickly generate the correct unicode-base64 of any text entered:
- CODE() – Returns the numeric value of a character
- DEC2HEX() – Converts a decimal number to hexadecimal
- Concatenate() – Joins several strings into one string
- BIN2DEC() – Converts a binary number into decimal
- MID() – Returns the character from the middle of a string
The best way to deal with generating lots of unicodePwd values is to write a script. Doing so within Excel using VBA lets you see all the generated values, and keep them within the spreadsheet for future reference. Using another VBA script, I was able to instantly generate an LDIF file for importing into the AD LDS instance.
Uploading the Unicode-Base64 UnicodePwd Value
Because the unicodePwd is manipulated using the LDIFDE.exe program, uploading the unicodePwd must conform to the LDIF file format. Because the engagement involved the Windows Server 2008 R2 AD LDS role, this LDIF file format is what I followed so basic information was uploaded. Note that the user account of John Doe is the first one that has the unicodePwd value of car as IgBjAGEAcgAiAA==.
dn: CN=JohnDoe,OU=Accounts,DC=CONTOSO,DC=COM Changetype: add objectClass: top objectClass: person objectClass: organizationalPerson objectClass: user cn: John Doe givenName: John sn: Doe userPrincipalName: johnd@Contoso.COM mail: johnd@Contoso.COM unicodePwd::IgBjAGEAcgAiAA==\ msDS-UserAccountDisabled: FALSE dn: CN=Jess Wanders,OU=Accounts,DC=CONTOSO,DC=COM Changetype: add objectClass: top objectClass: person objectClass: organizationalPerson objectClass: user cn: Jess Wanders givenName: Jess sn: Wanders userPrincipalName: JessW@Contoso.COM mail: JessW@Contoso.COM unicodePwd::IgA3ACQANQBNAHMAIwA0AEQAaQBHACIA msDS-UserAccountDisabled: FALSE
So to import this file, assuming its file name is Accounts.ldf, enter the following syntax at the Windows Server 2008 R2 command prompt console:
C:\Windows\system32\LDIFDE –i –f Accounts.ldf.
Binding and Testing the Account’s Password
After uploading one or more newly created accounts to the application partition, and connecting or binding to the LDAP service with the account’s userPrincipalName and unicodePwd, it‘s important to be sure the unicode-base64 format of the unicodePwd is correct. The tool that works best and is easy to use is LDP.exe. You might think that IgBjAGEAcgAiAA== is the correct password to enter when using this, but it’s not. It’s what we originally started with: car.
Once you’ve connected to the AD LDS instance using LDP, bind with the UPN of the uploaded account and enter the password. If you correctly followed the logic to create the unicodePwd unicode-base64 value, you’ll see a positive confirmation displayed on the LDP window that you are in fact bound to that AD LDS instance with that account.
Using Windows PowerShell to Update the UnicodePwd Attribute
Now, it wouldn’t be fair if I said that LDIFDE.exe was the only tool around capable of performing bulk changes to the unicodePwd attribute. Then you wouldn’t have learned how to generate the unicode-base64 value.
In Windows Server 2008 R2, Windows PowerShell v2 brought a bit more granularity in its ability to manipulate select attributes in AD and AD LDS. Working with AD LDS, I came up with the syntax in Figure 3 (note that for the sake of readability, I placed each command option on its own line) to achieve the same results that my VBA script does within the Excel spreadsheet.
It’s obvious that with just a few lines of Windows PowerShell code, you can quickly make the necessary changes without having to jump through hoops when creating the unicode-base64 value for each password. This is a powerful statement of Windows PowerShell’s capabilities, but it comes at the expense of processing time, considering all that it does in the background. This is especially evident if processing hundreds, or thousands, of accounts.
So if you occasionally need to create or update a few accounts every now and then, use Windows PowerShell. If you always create or update hundreds or thousands on a regular basis, using LDIFDE.exe is much quicker in the long run.
Figure 3 Windows PowerShell v2 script to update the account password.
Import-Csv c:\scripts\accounts.csv | New-ADUser –Name $_.commonName –GivenName $_.givenName –Surname $_.sn -EmailAddress $_.email -Type user -UserPrincipalName $_.userPrincipalName –Server LDS01:389 | Set-ADAccountPassword -Identity $_.distinguishedName -NewPassword (ConvertTo-SecureString -AsPlainText $_.Password -Force) -Reset -Server LDS01:389 | Enable-ADAccount -Identity $_.distinguishedName -Server LDS01:389
Whether working with AD or AD LDS (or even ADAM for that matter), there are several ways to update the unicodePwd attribute. If you have just one account to change, use LDP.exe or ADSIedit to get it done. If there’s just a handful every now and then, using Windows PowerShell v2 against Windows Server 2008 R2 AD/AD LDS works great.
However, if you’re still working with Windows Server 2003 or later, and have hundreds to thousands of accounts that need constant updating, do your prep work upfront to generate the unicode-base64 value for the unicodePwd attribute. Then use LDIFDE.exe to upload those changes all at once.
Frank Rettig is a consultant in the U.S. Public Sector practice, located in the Washington, D.C., area, with 26 years experience working domestically and internationally. He specializes in directory integration, identity management, mobility solutions and government computing standards. Frank can be reached at email@example.com.
I would like to thank the Microsoft Information Security ACE Services team members Roger Grimes (Principal Security Architect) and Shawn Rabourn (Senior Security Consultant) for validating this article, and USPS Federal Services’ Anthony de Lagarde (Senior Consultant) for content review.