Local Admin Password Maintainer

by Ryan 18. February 2015 17:02

Active Directory is great for robust, centralized management of a large amount of I.T. assets.  But even once you have Active Directory, you're still left with that problem of what to do with local administrator accounts on all of the domain members.  You probably don't want to disable the local admin account, because you'll need it in case the computer is ever in a situation where it can't contact a domain controller.  But you don't have a good way of updating and maintaining the local Administrator password across your entire environment, either.  Everyone knows better than to use Group Policy Preferences to update the local administrator password on domain members, as it is completely unsecure.  Most other solutions involve sending the administrator passwords across the network in clear-text, require an admin to manually run some scripts or software every time that may not work well in complicated networks, and they still leave you with the same local administrator password on every machine... so if an attacker knocks over any one computer in your entire domain, he or she now has access to everything.

This is the situation Local Admin Password Maintainer seeks to alleviate.  LAPM easily integrates into your Active Directory domain and fully automates the creation of random local administrator passwords on every domain member.  The updated password is then transmitted securely to a domain controller and stored in Active Directory.  Only users who have been given the appropriate permissions (Domain Administrators and Account Operators, by default) may view any password.

The solution is comprised of two files: Install.ps1, which is the one-time install script, and LAPM.exe, an agent that will periodically (e.g., once a month,) execute on all domain members.  Please note that these two files will always be digitally signed by me.

Minimum Requirements

  • Active Directory. You need to be a member of both Domain Admins and Schema Admins to perform the install. You must perform the installation on the forest schema master.
  • Forest and domain functional levels of 2008 or better. This software relies on a feature of Active Directory (confidential attributes) that doesn't technically require any certain forest or domain functional level, but enforcing this requirement is an easy way of ensuring that all domain controllers in your forest are running a modern version of Windows.
  • I do not plan on doing any testing of either the install or the agent on Windows XP or Server 2003.  I could hypothetically make this work on XP/2003 SP1, but I don't want to.  If you're still using those operating systems, you aren't that concerned with security anyway.
  • A Public Key Infrastructure (PKI,) such as Active Directory Certificate Services, or otherwise have SSL certificates installed on your domain controllers that enable LDAP over SSL on port 636.  This is because LAPM does not allow transmission of data over the network in an unsecure manner.  It is possible to just bang out some self-signed certificates on your domain controllers, and then distribute those to your clients via Group Policy, but I do not recommend it.
  • The installer requires Powershell 4. Which means you need Powershell 4 on your schema master. Which means it needs to be 2008 R2 or greater.  I could port the install script to an older version of Powershell, but I haven't done it yet.
  • The Active Directory Powershell module. This should already be present if you've met the requirements thus far.
  • The Active Directory Web Service should be running on your DCs. This should already be present if you've met the requirements thus far.
  • LAPM.exe (the "agent") will run on anything Windows Vista/Server 2008 or better, 32 or 64 bit.  I just don't feel like porting it back to XP/2003 yet.


Copyright ©2015 Joseph Ryan Ries. All Rights Reserved.



Installation Instructions

  • Download the installation package found below, and unzip it anywhere on your Active Directory domain controller that holds the Schema Master FSMO role.  (Use the netdom query fsmo command if you forgot which DC is your Schema Master.)
  • If necessary, use the Unblock-File Powershell cmdlet or use the GUI to unblock the downloaded zip file.
  • You can verify the integrity of the downloaded files like so:

  • If you need to change your Powershell execution policy in order to run scripts on your DC, do so now with Set-ExecutionPolicy RemoteSigned.
  • Execute the Install script by typing .\Install.ps1 in the same directory as the script and LAPM.exe.

  • The installation script will perform several prerequisite checks to ensure your Active Directory forest and environment meet the criteria. It will also create a log file that stores a record of everything that takes place during this install session.  If you see any red [ERROR] text, read the error message and try to correct the problem that is preventing the install script from continuing, then try again. (E.g. SSL certificate not trusted, you're not on the Schema Master, etc.)  It's important that you read and consider the warning text, especially the part about how extending the Active Directory schema is a permanent operation.
  • Type yes at the warning prompt to commit to the installation.

  • The installation will now make a small schema modification by adding the LAPMLocalAdminPassword attribute to the Active Directory schema, adding that attribute to the computer object, and then adding an access control entry (ACE) to the root of the domain that allows the SELF principal the ability to write to that attribute.  That means that a computer has the right to modify its own LAPMLocalAdminPassword attribute, but not the attribute of another computer. (A computer does not have the ability to read its own LAPMLocalAdminPassword attribute. It is write-only.)

  • Finally, the install script copies LAPM.exe to the domain's SYSVOL share. This is so all domain members will be able to access it.
  • You are now done with the script and are in the post-installation phase.  You have one small thing left to do.
  • Open Group Policy Management on your domain controller.

  • Create a new GPO and link it to the domain:

  • Name the new GPO Local Admin Password Maintainer.
  • Right click on the new GPO and choose Edit. This will open the GPO editor.
  • Navigate to Computer Configuration > Preferences > Control Panel Settings > Scheduled Tasks.

  • Right-click in the empty area and choose New > Scheduled Task (At least Windows 7).

  • Choose these settings for the new scheduled task. It is very important that the scheduled task be run as NT Authority\System, also known as Local System.

  • This task will be triggered on the first of every month.  It's advisable to configure the random delay shown in the screenshot above, as this will mitigate the flood of new password uploads to your domain controllers on the first of the month.

  • For the program to execute, point to \\YourDomain\SYSVOL\YourDomain\LAPM.exe. Remember that the second "YourDomain" in the path is a reparse point/symlink that looks like "domain" if you view it in File Explorer.  For the optional argument, type BEGIN_MAGIC, in all capital letters.  It is case sensitive.
  • Lastly, the "Remove this item when it is no longer applied" setting is useful.  Unchecking "allow this task to be run on demand" can also be useful.  As an administrator, you have some leeway here to do what makes the most sense for your environment.  You might even choose to scope this GPO to only a certain OU if you only want a subset of the members of your domain to participate in Local Admin Account Maintainer.

  • Click OK to confirm, and you should now have a new scheduled task that will execute on all domain members.
  • Close the Group Policy editor.

Don't worry if the scheduled task also applies to domain controllers.  LAPM.exe detects whether it is running on a domain controller before it does anything, and exits if it is.

It also doesn't matter what the local administrator's name is, in case the account has been renamed. LAPM uses the SID.

LAPM logs successes and failures to the Windows Application event log.  Here is an example of what you might see if a client can't connect to a DC for some reason, like if SSL certificates aren't configured correctly:

In an event like this, LAPM.exe exits before changing the local administrator password, so the password will just stay what it was until the next time the scheduled job runs.

LAPM will generate a random, 16-character long password.  The "randomness" comes from the cryptographically secure PRNG supplied by the Windows API.

Success looks like this:

Now, notice that the standard domain user "Smacky the Frog" is unable to read the LAPMLocalAdminPassword attribute from Active Directory:

However, a Domain Administrator or Account Operator can!

Of course, you can also see it in the GUI as well, with Active Directory Users and Computers with advanced view turned on, for example.

So there you have it. Be smart, test it out in a lab first, and then enjoy your 30-day, random rotating local admin passwords!

As I continue to update this software package, new versions will be published on this page.


LAPM-1.0.zip (54.4KB)

Configuring HP ILO Settings and TLS Certificates With Powershell

by Ryan 6. February 2015 17:02

I've been configuring HP ILOs lately. And of course, the cardinal rule in I.T. is that if you're going to do something more than once, then you must start automating it.  And of course, if you want to automate something, then you fire up Powershell.

Luckily, HP is playing ball with the HP Scripting Tools for Windows Powershell. The cmdlets are not half bad, either. Essentially, what I needed to do was configure a bunch of ILOs, including renaming them, setting some IPv6 settings, and putting valid SSL/TLS certificates on them.

First, let's save the ILOs address (or hostname,)  username and password for future use:

[String]$Device   = ''
[String]$Username = 'Admin'
[String]$Password = 'P@ssword'

Next, let's turn off IPv6 SLAAC and DHCPv6 (for ILO 3s, firmware ~1.28 or so, and above):

Set-HPiLOIPv6NetworkSetting -Server $Device `
                            -Username $Username `
                            -Password $Password `
                            -AddressAutoCfg Disable

Set-HPiLOIPv6NetworkSetting -Server $Device `
                            -Username $Username `
                            -Password $Password `
                            -DHCPv6Stateless Disable

Set-HPiLOIPv6NetworkSetting -Server $Device `
                            -Username $Username `
                            -Password $Password `
                            -DHCPv6Stateful Disable

Next I wanted to set the FQDN to what I wanted it to be... it was important that I turned DHCP off first, because the ILO wanted to set the domain name using DHCP and thus locked it from being edited, even though no DHCP server was actually on the network:

Set-HPiLOServerName -Server $Device `
                    -Username $Username `
                    -Password $Password `
                    -ServerName 'server1-ilo.contoso.com'

Now I wanted to put a valid SSL/TLS certificate on the ILO. So, I needed to first generate a Certificate Signing Request (CSR) on the ILO:

Get-HPiLOCertificateSigningRequest -Server $Device `
                                   -Username $Username `
                                   -Password $Password

IP                          :
HOSTNAME                    : server1-ilo.contoso.com
STATUS_TYPE                 : OK
STATUS_MESSAGE              : OK
                              -----END CERTIFICATE REQUEST-----

Nice... now copy and paste the entire CSR text block, including the -----BEGIN and END------ bits, and submit that your certificate authority.  Then, the administrator of the certificate authority has to approve the request.

This is the one piece where automation breaks down, in my opinion, and some manual intervention is necessary. This is not a technical limitation, though... it's by design.  The idea is that the entire basis of SSL/TLS public key cryptography is that it's based on trust.  And that trust has to come from other sources such as the Certificate Authority administrator phoning the requestor and verifying that it was actually them making the request, or getting some additional HR info, or whatever.  If, at the end of the day, there was no extraordinary measure taken to really verify the requestor's identity, then you can't really trust these certificates.

Anyway, once the CA has signed your CSR, you just need to import the signed certificate back into the ILO:

Import-HPiLOCertificate -Server $Device `
                        -Username $Username `
                        -Password $Password `
                        -Certificate (Get-Content C:\mycert.cer -Raw) 

Assuming no errors were returned, then you're done and your HP ILO will now reboot, and when it comes back up, will be using a valid SSL certificate.

Also, HP ILOs cannot read certificates if they are using PKCS #1 v2.1 format. Add that to the huge pile of devices that cannot read an X509 standard that came out in 2003.

Testing Authenticated NTP Configurations

by Ryan 23. January 2015 08:01

It's been a long time since I posted, I know. I've been busy both with actual work, and also working on a personal project that involves getting way better at old-school C programming. I've been following Casey Muratori and his Handmade Hero series for a couple of months now, and I've been really inspired to write more C.

More on what I've been working on as it develops.

Also, my friend and co-conspirator Wesley sent me a gift for accomplishing the "Serverfault 10k Challenge" in 2014:

This majestic creature embodies a never-ending thirst for knowledge and symbolizes the triumphs and tribulations of the sysadmin.  It's not a Microsoft MVP award...

... it's better.

Alright so the topic of today's post: Authenticated NTP.  NTP is one of the very oldest protocols on the internet, it has had very few vulnerabilities reported over its 30+ year lifespan, and is ubiquitous in virtually every computer network on the planet. (Because most computers are awful at keeping time.) There are many different versions of it and spinoffs from the reference implementation. People tend to find NTP a boring protocol, but it's one of my favorite internet protocols.  Most people just point their router at pool.ntp.org and never think about NTP again.

Until the day comes that you want to enable authentication in your NTP architecture. Authentication is the mechanism that allows a message recipient to verify that the response came from the intended sender and wasn't tampered with in transit. Within a Windows Active Directory domain, we already have this. Domain members use the Kerberos session keys they already have to create authenticated Windows Time messages with domain controllers.  But that means if you're not a member of the domain, you can't participate.

The administrator of an NTP server who wishes to send you authenticated NTP messages will probably send you an ntp.keys file, or at least a password for you to use. An example ntp.keys file looks like this:

# ntpkey_MD5key_ntp.domain.com.2825130701
# Thu Jan 15 20:00:01 2015
 1 MD5  JzF&f})0ocK1{H9	# MD5 key
 2 MD5  Dv(0v@W8vJ8%#*2	# MD5 key
 3 MD5  N(BzeyvYx$qzs5]	# MD5 key
 4 MD5  TVd2*DXtu-mewLs	# MD5 key
 5 MD5  F9UTa)8AQ9O9561	# MD5 key
 6 MD5  F9}{%$d9vs3Dpxb	# MD5 key
 7 MD5  D]Z*OOr56ukpiD6	# MD5 key
 8 MD5  TTr$OIR9+f74J28	# MD5 key
 9 MD5  EC3F9Zr%-3190&0	# MD5 key
10 MD5  Ndi5+]F^3x3Gdeb	# MD5 key
11 MD5  S+27&8(ba30qM@5	# MD5 key
12 MD5  CnO8)=CyG)QBj]}	# MD5 key
13 MD5  Em62oK!RXhw#y9_	# MD5 key
14 MD5  K-l(^UE@&T(Zj5B	# MD5 key
15 MD5  Gcff1nJb(CuF$*!	# MD5 key
16 MD5  W-*5^xbp3@v8br)	# MD5 key

There aren't any tools that ship with Windows that will help you test this stuff.  The Windows implementation of NTP ("Windows Time") is good enough to keep Windows working within the context of an AD domain, but it's not a full-featured reference NTP implementation.  So, I downloaded the Windows port of NTP from ntp.org. You can use the ntpdate program to query an NTP server using the shared secrets in your ntp.keys file to verify that authentication is successful:

C:\Users\Ryan>ntpdate.exe -b -d -k C:\Users\Ryan\ntp.keys -a 1 ntp.domain.com
receive: authentication passed
receive: authentication passed
server, port 123
stratum 2, precision -20, leap 00, trust 000
refid [], delay 0.03090, dispersion 0.00066
transmitted 4, in filter 4
Authenticated NTP: Check. I'll probably write more about this topic in the future, but I have to perform some experiments first.

One Way of Exporting Nicer CSVs with Powershell

by Ryan 16. December 2014 11:12

One of the ever-present conundrums in working with computers is that data that looks good and easily readable to a human, and data that is easy and efficient for a computer to process, are never the same.

In Powershell, you see this "immutable rule" manifest itself in that, despite all the various Format-* cmdlets available to you, some data will just never look good in the console.  And if it looks good in the console, chances are you've mangled the objects so that they've become useless for further processing over the pipeline.  This is essentially one of the Powershell "Gotcha's" espoused by Don Jones, a term that he refers to as "Format Right."  The principal is that if you are going to format your Powershell output with a Format-* cmdlet, you should always do so at the end of the statement (e.g., on the right side.)  The formatting should be the last thing you do in an expression, and you should never try to pass something that has been formatted over the pipeline.

CSV files, in my opinion, are a kind of happy medium, because they are somewhat easy for humans to read (especially if the human has an application like Microsoft Excel or some such,) and CSV files are also relatively easy for computers to read and process.  Therefore, CSVs are a popular format for transporting data and feeding it to computers, while still being legible to humans.

When you use Export-Csv to write a bunch of objects out to a CSV file:

# Get Active Directory groups, their members, and memberships:
Get-ADGroup -Filter * -SearchBase 'CN=Users,DC=domain,DC=local' -Properties Members,MemberOf | `
    Select Name, Members, MemberOf | `
    Export-Csv -NoTypeInformation -Path C:\Users\ryan\Desktop\test.csv 

And those objects contain arrays or lists as properties, you'll get something like this in your CSV file:


Uh... that is not useful at all.  What's happened is that instead of outputting the contents of the Active Directory group members and memberOf attributes, which are collections/arrays, Powershell has instead output only the names of the .NET types of those collections.

What we need is a way to expand those lists so that they'll go nicely into a CSV file.  So I usually do something like the script excerpt below.  This is just one possible way of doing it; I by no means claim that it's the best way or the only way.

#Get all the AD groups:
$Groups = Get-ADGroup -Filter * -SearchBase 'OU=MyOU,DC=domain,DC=com' -Properties Members,MemberOf

#Create/initialize an empty collection that will contain a collection of objects:
$CSVReadyGroups = @()

#Iterate through each one of the groups:
Foreach ($Group In $Groups)
    #Create a new object to hold our "CSV-Ready" version of the group:
    $CSVReadyGroup = New-Object System.Object #Should probably be a PSObject
    #Add some properties to the object.
    $CSVReadyGroup | Add-Member -Type NoteProperty -Name 'Name'     -Value  $Group.Name
    $CSVReadyGroup | Add-Member -Type NoteProperty -Name 'Members'  -Value  $Null
    $CSVReadyGroup | Add-Member -Type NoteProperty -Name 'MemberOf' -Value  $Null

    # If the group has any members, then run the code inside these brackets:
    If ($Group.Members)
        # Poor-man's serialization.
        # We are going to convert the array into a string, with NewLine characters 
        # separating each group member. Could also be more concise just to cast
        # as [String] and do  ($Group.Members -Join [Environment]::NewLine)

        $MembersString = $Null
        Foreach ($GroupMember In $Group.Members)
            $MembersString += $GroupMember + [Environment]::NewLine
        #Trim the one extra newline on the end:
        $MembersString = $MembersString.TrimEnd([Environment]::NewLine)
        #Add to our "CSV-Ready" group object:
        $CSVReadyGroup.Members = $MembersString

    # If the group is a member of any other groups, 
    # then do what we just did for the Members:
    If ($Group.MemberOf)
        $MemberOfString = $Null
        Foreach ($Membership In $Group.MemberOf)
            $MemberOfString += $Membership + [Environment]::NewLine
        $MemberOfString = $MemberOfString.TrimEnd([Environment]::NewLine)
        $CSVReadyGroup.MemberOf = $MemberOfString

    #Add the object we've created to the collection:
    $CSVReadyGroups += $CSVReadyGroup

#Output our collection:
$CSVReadyGroups | Export-Csv -NoTypeInformation -Path C:\Users\ryan\Desktop\test.csv

Now you will have a CSV file that has readable arrays in it, that looks good when you open it with an application such as Excel.

Have You Been Pwned by CVE-2014-6324/MS14-068?

by Ryan 19. November 2014 08:11

In case you haven't heard, there is a critical [Windows implementation of] Kerberos bug that you need to be updating, right now.

More information on the vulnerability can be found here.

In the "Detection Guidance" section of the above blog post, you will see that you can detect if the vulnerability has been exploited on an unpatched machine by analyzing the Security event logs. Specifically, looking at Event ID 4624 logon events, and taking note that the "Security ID" and "Account Name" fields in that event description match.  If they don't, chances are high that you have been a victim of a privilege escalation attack.

I whipped up a detection script to check all the domain controllers:

#Requires -Module ActiveDirectory
Set-StrictMode -Version Latest
Get-Job | Remove-Job -Force
[String]$DomainName = $(Get-ADDomain).Name
$DCs = $(Get-ADDomain).ReplicaDirectoryServers

:NextDC Foreach ($DC In $DCs)
    Start-Job -ScriptBlock {
        [Int]$PotentialMS14068s = 0
        Write-Output "Fetching Security event log from $DC."
            $Events = Get-EventLog -LogName Security -InstanceId 4624 -ComputerName $DC -ErrorAction Stop
            Write-Error "An error occurred while reading event log from $DC.`r`n$($_.Exception.Message)"

        :NextEvent Foreach ($Event In $Events)
            $MessageLines  = $Event.Message -Split [Environment]::NewLine
            [String]$SecurityID    = [String]::Empty
            [String]$AccountName   = [String]::Empty
            [String]$AccountDomain = [String]::Empty

            # Server 2012 Format
            If ($MessageLines[13].Trim() -Like 'Security ID:*')
                $SecurityID    = ($MessageLines[13].Trim() -Split ':')[1].Trim()
                $AccountName   = ($MessageLines[14].Trim() -Split ':')[1].Trim()
                $AccountDomain = ($MessageLines[15].Trim() -Split ':')[1].Trim() 
            # Server 2008 R2 Format
            If ($MessageLines[11].Trim() -Like 'Security ID:*')
                $SecurityID    = ($MessageLines[11].Trim() -Split ':')[1].Trim()
                $AccountName   = ($MessageLines[12].Trim() -Split ':')[1].Trim()
                $AccountDomain = ($MessageLines[13].Trim() -Split ':')[1].Trim()

            If (($SecurityID -EQ [String]::Empty) -OR ($AccountName -EQ [String]::Empty) -OR ($AccountDomain -EQ [String]::Empty))
                Write-Error "Event log message format unrecognized on $DC!"
                $Event | Format-List
                Break NextEvent

            If ($AccountDomain -Like $DomainName -And $SecurityID -NotLike 'S-1-5-18')
                $SID = New-Object System.Security.Principal.SecurityIdentifier($SecurityID)
                $Username = $SID.Translate([System.Security.Principal.NTAccount])        
                If ($Username -Like '*\*')
                    $Username = ($Username -Split '\\')[-1]
                If ($Username -Like '*@*')
                    $Username = ($Username -Split '@')[0]
                If ($Username -NE $AccountName)
                    $Event | Format-List
        Write-Output "Finished with $DC. $PotentialMS14068s interesting events found."
    } -ArgumentList $DC

While ($(Get-Job -State Running).Count -GT 0)
    Get-Job -State Completed | Receive-Job   
    Start-Sleep -Seconds 10 

The script uses Powershell jobs to achieve some parallelism, because if you have more than one or two domain controllers in your environment, this quickly becomes a Herculean, time-consuming task.  The script will display potential security event log events that may indicate exploits currently being used in your environment.

About Me

Ryan Ries
Texas, USA
Systems Engineer

I am a systems engineer with a focus on Microsoft tech, but I can run with pretty much any system that uses electricity.  I'm all about getting closer to the cutting edge of technology while using the right tool for the job.

This blog is about exploring IT and documenting the journey.

Blog Posts (or Vids) You Must Read (or See):

Pushing the Limits of Windows by Mark Russinovich
Mysteries of Windows Memory Management by Mark Russinovich
Accelerating Your IT Career by Ned Pyle
Post-Graduate AD Studies by Ned Pyle
MCM: Active Directory Series by PFE Platforms Team
Encodings And Character Sets by David C. Zentgraf
Active Directory Maximum Limits by Microsoft
How Kerberos Works in AD by Microsoft
How Active Directory Replication Topology Works by Microsoft
Hardcore Debugging by Andrew Richards
The NIST Definition of Cloud by NIST

MCITP: Enterprise Administrator


Profile for Ryan Ries at Server Fault, Q&A for system administrators


GitHub: github.com/ryanries


I do not discuss my employers on this blog and all opinions expressed are mine and do not reflect the opinions of my employers.