My Entry for the Advanced Event #4 of the 2013 Scripting Games

by Ryan 21. May 2013 12:35

We're on the downhill stretch now. Honestly I'm kind of glad.  These scripts are fun to write, and great practice, but it's work.  I can tell that I'm not the only one loosing steam, as the number of votes on other people's entries has gone way down.  Anyway, about the script I wrote: I like that the #Requires -Modules statement at the top automatically loads the AD module for you if it's not already loaded. I still didn't do the BEGIN/PROCESS/END blocks this time either, which I fail to see how it matters at all, since I'm not dealing with pipeline input... but I'm sure I'll still get crowd scores of 1 and 2 stars for it.  That and dudes with 640x480 monitors going "some of your code goes off the screen why don't you splat!?"  :P

#Requires -Version 3
#Requires -Modules ActiveDirectory
Function Get-RandomADUser
{
<#
.SYNOPSIS
    Retrieves random users from Active Directory and generates an HTML report.
.DESCRIPTION
    Retrieves random users from Active Directory, generates an HTML report,
    and then returns the users to the pipeline. 
    Use the -Verbose switch if you want to see console output.
    This Cmdlet requires PS 3 and the Active Directory module. The AD module
    will be loaded automatically if it isn't already.
.PARAMETER Count
    The number of random users to get from Active Directory. Minimum is 1,
    maximum is Int16.MaxValue (32767) and the default is 20.
.PARAMETER Filename
    The filename to write the HTML report to. The filename must end in
    html or htm. The default is .\RandomADUsers.html.
.EXAMPLE
    Get-RandomADUser
 
    Gets 20 random users from AD, outputs a report to .\RandomADUsers.html.
.EXAMPLE
    Get-RandomADUser -Count 100 -Filename C:\reports\rpt.html.
 
    Gets 100 random users from AD, outputs a report to C:\reports\rpt.html.
#>
 
    [CmdletBinding()]
    Param([Parameter()]
            [ValidateRange(1, [Int16]::MaxValue)]
            [Int16]$Count = 20,
          [Parameter()]
            [ValidateScript({($_.ToLower().Split('.')[-1] -EQ "html" -OR $_.ToLower().Split('.')[-1] -EQ "htm") -AND (Test-Path -IsValid $_)})]
            [String]$Filename = ".\RandomADUsers.html") 
 
    Try
    {
        Write-Verbose "Retrieving users from Active Directory..."
        $Users = Get-ADUser -Filter * -Properties Department, Title, LastLogonDate, PasswordLastSet, Enabled, LockedOut -ErrorAction Stop | Get-Random -Count $Count
        Write-Verbose "$($Users.Count) users retrieved from Active Directory."
    }
    Catch
    {
        Write-Error "Unable to retrieve users from Active Directory: $($_.Exception.Message)"
        Return
    }   
    Try
    {
        Write-Verbose "Generating report $Filename..."
        $Header = @'
        <title>Random Active Directory User Audit</title>
            <style type=""text/css"">
                <!--
                    TABLE { border-width: 1px; border-style: solid;  border-color: black; }
                    TD    { border-width: 1px; border-style: dotted; border-color: black; }
                -->
            </style>
'@
        $Pre  = "<p><h2>Random Active Directory User Audit for $Env:USERDNSDOMAIN</h2></p>"
        $Post = "<hr><p style=`"font-size: 10px; font-style: italic;`">This report was generated on $(Get-Date)</p>"
        $Users | ConvertTo-HTML -Property SamAccountName, Department, Title, LastLogonDate, PasswordLastSet, Enabled, LockedOut -Head $Header -PreContent $Pre -PostContent $Post | Out-File $Filename     
        Return $Users
    }
    Catch
    {
        Write-Error "Unable to generate report: $($_.Exception.Message)"
    }
}

My Entry for the Advanced Event #3 of the 2013 Scripting Games

by Ryan 14. May 2013 09:09

Halfway done.  Here's my third entry for this year's Powershell games.  I used a workflow this time, mostly in an attempt to garner favor from the voters for using new features exclusive to PS3.  Even though the multithreading with jobs that I did in the last event is a neat idea, it really doesn't perform very well.  The workflow will likely perform better, though I don't know if it's going to handle the throttling of thread creation if I handed it a list of 500 computers.

#Requires -Version 3
Function New-DiskSpaceReport
{
	<#
		.SYNOPSIS
			Gets hard drive information from one or more computers and saves it as HTML reports.
		.DESCRIPTION
			Gets hard drive information from one or more computers and saves it as HTML reports.
			The reports are saved to the specified directory with the name of the computer in
			the filename. The list of computers is processed in parallel for increased speed.
			Use the -Verbose switch if you want to see console output, which is very useful if you
			are having problems generating all the desired reports.
		.PARAMETER ComputerName
			One or more computer names from which to get information. This can be a
			comma-separated list, or a file of computer names one per line. The alias
			of this parameter is -Computer. The default value is the local computer.
		.PARAMETER Directory
			The directory to write the HTML files to. E.g., C:\Reports. The directory
			must exist. The default is the current working directory.
		.INPUTS
			[String[]]$ComputerName
			This is an array of strings representing the hostnames of the computers
			for which you want to retrieve information. This can also be supplied by
			(Get-Content file.txt). This can be piped into the cmdlet.
		.INPUTS
			[String]$Directory
			The directory to save the HTML reports to. The directory must exist.
		.OUTPUTS
			HTML files representing the information obtained from all
			the computers supplied to the cmdlet.
		.EXAMPLE
			New-DiskSpaceReport
			
			This will generate a report for the local computer and output the HTML file to
			the current working directory.			
		.EXAMPLE
			New-DiskSpaceReport -ComputerName server01,server02,server03 -Directory C:\Reports
			
			This will generate three HTML reports for the servers and save them in the C:\Reports
			directory.
		.EXAMPLE
			New-DiskSpaceReport -Computer (Get-Content .\computers.txt)
			
			This will generate HTML reports for all the computers in the computers.txt file and
			save the reports in the current working directory.
		.EXAMPLE
			,(Get-Content .\computers.txt) | New-DiskSpaceReport -Directory C:\Reports
			
			This will generate HTML reports for all the computers in the computers.txt file and
			save the reports in C:\Reports. Please note the leading comma in this example.
		.NOTES
			Scripting Games 2013 Advanced Event 3
	#>
	[CmdletBinding()]
	Param([Parameter(ValueFromPipeline=$True)]
			[Alias('Computer')]
			[String[]]$ComputerName = $Env:Computername,
		  [Parameter()]
			[ValidateScript({Test-Path $_ -PathType Container})]
			[String]$Directory = (Get-Location).Path)
	
	Write-Verbose -Message "Writing reports to $Directory..."
	
	WorkFlow BuildReports
	{
		Param([String[]]$Computers, [String]$Directory)
		ForEach -Parallel ($Computer In $Computers)
		{			
			InlineScript
			{				
				Write-Verbose -Message "Generating report for $Using:Computer..."
				$Header = @'
				<title>Disk Free Space Report</title>
				<style type=""text/css"">
					<!--
						TABLE { border-width: 1px; border-style: solid;  border-color: black; }
						TD    { border-width: 1px; border-style: dotted; border-color: black; }
					-->
				</style>
'@
				$Pre  = "<p><h2>Local Fixed Disk Report for $Using:Computer</h2></p>"
				$Post = "<hr><p style=`"font-size: 10px; font-style: italic;`">This report was generated on $(Get-Date)</p>"
				Try
				{					
					$LogicalDisks = Get-WMIObject -Query "SELECT * FROM Win32_LogicalDisk WHERE DriveType = 3" -ComputerName $Using:Computer -ErrorAction Stop | Select-Object -Property DeviceID,@{Label='SizeGB';Expression={"{0:N2}" -F ($_.Size/1GB)}},@{Label='FreeMB';Expression={"{0:N2}" -F ($_.FreeSpace/1MB)}},@{Label='PercentFree';Expression={"{0:N2}" -F (($_.Freespace/$_.Size)*100)}};
					$LogicalDisks | ConvertTo-HTML -Property DeviceID, SizeGB, FreeMB, PercentFree -Head $Header -PreContent $Pre -PostContent $Post | Out-File -FilePath $(Join-Path -Path $Using:Directory -ChildPath $Using:Computer`.html)
					Write-Verbose -Message "Report generated for $Using:Computer."
				}
				Catch
				{
					Write-Verbose -Message "Cannot build report for $Using:Computer. $($_.Exception.Message)"
				}
			}
		}
	}
	
	If($PSBoundParameters['Verbose'])
	{
		BuildReports -Computers $ComputerName -Directory $Directory -Verbose
	}
	Else
	{
		BuildReports -Computers $ComputerName -Directory $Directory
	}
}

My Entry for the Advanced Event #2 of the 2013 Scripting Games

by Ryan 10. May 2013 11:21

More Powershell! I'm somewhat proud of this script.

#Requires -Version 3
Function Get-ComputerInfo
{
	<#
		.SYNOPSIS
			Gets some basic system information about one or more remote Windows computers.
		.DESCRIPTION
			Gets some basic system information about one or more remote Windows computers.
			Specifically designed to be able to fetch information from any version of
			Windows computer from Windows 2000 up. This Cmdlet takes only one parameter,
			-ComputerName. ComputerName can be a single computer name or IP address, or it
			can be an array of computer names. You can also use a file of computer hostnames,
			one per line. This function will return the information gathered from all
			of the computers. Remember to use a leading comma when piping an array to
			this cmdlet. See the examples for more details. Powershell 3.0 is the minimum
			required on the machine that runs this cmdlet, though the target computers 
			do not need Powershell at all. Use Get-Help Get-ComputerInfo -Examples  to see
			usage examples. Example 8 is my favorite!
		.PARAMETER ComputerName
			One or more computer names from which to get information. This can be a
			comma-separated list, or a file of computer names one per line. The alias
			of this parameter is -Computer.
		.PARAMETER MaxThreads
			Default is 4. This is the maximum number of threads that are allowed to
			run simultaneously. This is useful because network operations can block
			for a long time, making threading desirable. However, when using a very 
			large list of computers, spawning a huge number of concurrent threads can
			be detrimental to the system, so thread creation should be throttled.
			The max is 32. The alias for this parameter is -Threads.
		.INPUTS
			[String[]]$ComputerName
			This is an array of strings representing the hostnames of the computers
			for which you want to retrieve information. This can also be supplied by
			(Get-Content file.txt). This can be piped into Get-ComputerInfo.
		.OUTPUTS
			A collection of objects representing the information obtained from all
			the computers supplied to the cmdlet.
		.EXAMPLE
			Get-ComputerInfo server1,server2,server3
		.EXAMPLE
			Get-ComputerInfo -ComputerName server1,server2,server3 | Format-Table
		.EXAMPLE
			Get-ComputerInfo -ComputerName (Get-Content .\computers.txt) -MaxThreads 8
		.EXAMPLE
			,(Get-Content .\computers.txt) | Get-ComputerInfo -Threads 12
			
			(Please note the leading comma in this example.)
		.EXAMPLE
			,("server1","server2","server3") | Get-ComputerInfo
			
			(Please note the leading comma in this example.)
		.EXAMPLE
			$Computers = @("server1","server2","server3")
			,$Computers | Get-ComputerInfo
		
			(Please note the leading comma in this example.)
		.EXAMPLE
			"server1" | Get-ComputerInfo
		.EXAMPLE
			Get-ComputerInfo -ComputerName ($(Get-ADComputer -Filter *).Name) | Out-GridView
		.NOTES
			Scripting Games 2013 Advanced Event 2
	#>
	[CmdletBinding()]
	Param([Parameter(Mandatory = $True, ValueFromPipeline=$True, HelpMessage = 'Computer names to scan, e.g. server01,server02,server03')]
			[Alias('Computer')]
			[String[]]$ComputerName,
		  [Parameter(Mandatory = $False)]
			[Alias('Threads')]
			[ValidateRange(1, 32)]
			[Int]$MaxThreads = 4)
	
	# This is the collection of objects that this function will eventually return.
	$ComputerInfoCollection = @()
	
	# By using the unique job name of "GetComputerInfo", we avoid interfering with any other
	# unrelated jobs that might be running by coincidence.
	$JobName = "GetComputerInfo"
	
	# Clear any old jobs with the same name before we begin. -EA Stop ensures that errors will be caught.
	Try
	{
		Get-Job -Name $JobName -ErrorAction Stop | Remove-Job -Force
	}
	Catch
	{
		# No jobs with the name $JobName were running. We don't care.
	}
	
	# This is the work to be performed by each thread in a Start-Job command.
	$Work = {
		$ComputerInfo = [PSCustomObject]@{ Name = $Args[0]; IPAddresses = $null; OSCaption = $null; MegaBytesRAM = $null; CPUSockets = $null; TotalCores = $null; }
		Try
		{			
			$ComputerInfo.IPAddresses = $([System.Net.Dns]::GetHostEntry($Args[0])).AddressList
		}
		Catch
		{
			# The hostname did not resolve to an IP address, so there is no reason to keep going.
			$ComputerInfo.IPAddresses = "Could not resolve name!"
			Return $ComputerInfo
		}
		Try
		{
			$ComputerInfo.OSCaption = $(Get-WMIObject Win32_OperatingSystem -ComputerName $Args[0] -ErrorAction Stop).Caption
		}
		Catch
		{
			$ComputerInfo.OSCaption = "$($_.Exception.Message)"
		}
		Try
		{
			$ComputerInfo.MegaBytesRAM = [Math]::Round($(Get-WMIObject Win32_ComputerSystem -ComputerName $Args[0] -ErrorAction Stop).TotalPhysicalMemory / 1MB, 0)
		}
		Catch
		{
			$ComputerInfo.MegaBytesRAM = "$($_.Exception.Message)"
		}
		Try
		{
			$CPUInfo = Get-WMIObject Win32_Processor -ComputerName $Args[0] -ErrorAction Stop
			
            # SocketDesignation does not exist on Server 2000
            # $ComputerInfo.CPUSockets = $CPUInfo.SocketDesignation.Count
            # Also, Win 2000 does not care about Hyperthreading and does not distinguish
            # cores from sockets AFAIK, so TotalCores will be null if Win 2000. Not a big deal IMO.
            $ComputerInfo.CPUSockets = $CPUInfo.DeviceID.Count
			ForEach($CPU In $CPUInfo)
			{
				$Cores += $CPU.NumberOfCores
			}
			$ComputerInfo.TotalCores = $Cores
		}
		Catch
		{
			$ComputerInfo.CPUSockets = "$($_.Exception.Message)"
			$ComputerInfo.TotalCores = "$($_.Exception.Message)"
		}
		
		Return $ComputerInfo
	}
	
	ForEach($Computer In $ComputerName)
	{
		While($(Get-Job -State "Running" | Where-Object Name -EQ $JobName).Count -GE $MaxThreads)
		{
			# Max number of concurrent running threads reached - sleep until one is available.
			Start-Sleep -Milliseconds 500
		}
		Start-Job -Name $JobName -ScriptBlock $Work -ArgumentList $Computer | Out-Null
	}
	
	# Wait for all jobs to finish.
	# Get-Job -State "Running" -Name $JobName does not work for some reason, so let's do it in two steps.
	While(Get-Job -State "Running" | Where-Object Name -EQ $JobName)
	{
		Start-Sleep -Milliseconds 500
	}
	
	# Jobs are done, let's collect the results and store it in our collection.
	ForEach($Job In Get-Job -Name $JobName)
	{
		$ComputerInfoCollection += Receive-Job $Job
	}
	
	Return $ComputerInfoCollection
}

Probably the Craziest Powershell One-Liner I've Written To Date

by Ryan 7. May 2013 13:49

Someone at work asked me to identify duplicate computers in two separate AD forests, and remove the one that was no longer needed.  It's assumed as part of business policy that there should not be duplicate server hostnames anywhere in the company - even if they reside in different forests or domains.  But for some reason or another, a computer might get migrated from DomainA to DomainB, but the computer object stays behind in the old domain, etc.  So I decided to just collect all the computers from DomainA and DomainB (in ForestA and ForestB respectively), point out the computer accounts that had the same name in each domain, and list their PwdLastSet attribute next to their name.  If the machine had not updated its password in over 30 days in DomainA, while the machine password was up to date in DomainB, then it was reasonably safe to assume that the machine had been migrated out of DomainA and into DomainB, or vice versa.

I only had Powershell v2 on hand, so I didn't have the relative luxury of automatic foreach, etc.  After collecting the computer objects like $DomainAComputers = Get-ADComputer -Filter * -Properties *, check out this hideous monstrosity I came up with to compare them in a single line:

PS C:\Users\ryan> foreach($C In $(Compare-Object $($DomainAComputers|?{!($_.DistinguishedName.Contains("Disabled Accounts"))}|%{$_.Name}) $($DomainBComputers|?{!($_.DistinguishedName.Contains("Disabled Accounts"))}|%{$_.Name}) -IncludeEqual | ? { $_.SideIndicator -eq "==" })) { $o = $($DomainAComputers|?{$_.Name -eq $C.InputObject}); $n = $($DomainBComputers|?{$_.Name -eq $C.InputObject}); $o.DnsHostName + "`t" + $o.PasswordLastSet + "`t" + $n.DnsHostName + "`t" + $n.PasswordLastSet }

The output looks like this:

computer1.domainA.com    04/17/2013    computer1.domainB.com    01/21/2010
computer2.domainA.com    05/05/2013    computer2.domainB.com    10/11/2011
etc...

You can easily see now that the two computers in DomainA are active, while the computer objects of the same name in DomainB are stale, so I'll delete them.

Now don't get me wrong - this is not elegant or clever. It's thoroughly unreadable and ugly and I'd not brag about it except to say, "Haha, look how much s*#! I can cram on one single line of Powershell!"

A couple things that I thought were interesting:

  • Get-ADComputer gives you a free pseudo-attribute called PasswordLastSet, which is a nicely formatted DateTime. But it's not a "real" attribute of the object in Active Directory. Rather, it's the Powershell cmdlet's courtesy attribute where it automatically converts the real attribute - PwdLastSet - from file time (epoch seconds) to a .NET DateTime object. Many of the Active Directory cmdlets work that way.
  • Compare-Object -ExcludeDifferent didn't seem to work and I'm not sure why.  So I had to just use -IncludeEqual instead and isolate the names that were equal.

My Entry for the Advanced Event #1 of the 2013 Scripting Games

by Ryan 30. April 2013 10:49

I've been pretty excited about the annual scripting games. This is only my second Games, but they have been a terrific Powershell learning experience for me. This year it's being run by Don Jones and his gang:

http://scriptinggames.org/

http://powershell.org/wp/

Their PHP-based website has already shown to be a little buggy, and I will eat road salt before I enable Java in my browser, so I won't be using their chat room, but you have to cut them some slack as it's a brand new site that has never been used before.

When people comment on the scripts you submit, it can be humbling but is also a good learning experience for being able to tell what people wanted out of your script but didn't get. There's not any error-handling to speak of in this script - I knew that was a risk I was taking by submitting a script with no error handling, but the event description stated that I should "display all errors clearly," which is exactly what the script does with no error handling. Still, I could have still used error handling to make it a little more elegant. Also, I guess I've got to break down and start doing the -WhatIf and -Confirm junk, even though I don't exactly want to, it's going the extra mile.

Without further ado:

#Requires -Version 3
Function Move-OldFiles
{
	<#
		.SYNOPSIS
			Move files that are older than a specified number of days. (Default is 90 days.)
			Use the verbose switch if you want to see output, otherwise the Cmdlet shows only errors.
		.DESCRIPTION
			Move files that are older than a specified number of days (default 90) from the 
			source directory to a destination directory. Directory recursion is on by default,
			but can be disabled with the -NoRecurse switch. The subdirectory structure will be 
			preserved at the destination. By default, all files are moved, but a file pattern
			can be specified with the -Pattern parameter. By default, files that already exist at
			the destination and are readonly are not overwritten, unless the -Force switch is used.
			This cmdlet works with drive letters as well as UNC paths. By default, only errors are shown.
			Use the -Verbose switch if you want to see more output. This function requires Powershell 3.
		.PARAMETER SourceDirectory
			Specifies the source directory from which you want to move files. E.g. C:\Logs or C:\Logs\
			This must be a valid directory. The alias for this parameter is Source.			
		.PARAMETER DestinationDirectory
			Specifies the destination directory to which you want to move files. E.g. E:\Archives or
			E:\Logs\Old\ or \\SERVER02\Share\Logs. This must be a valid directory. The alias for
			this parameter is Destination.
		.PARAMETER OlderThan
			The number of days that a file's age must exceed before it will be moved. This is
			an optional parameter whose default is 90 days. This parameter must be a positive
			integer. The alias for this parameter is Age.
		.PARAMETER Pattern
			This is an optional filename filter. E.g., *.log or *.txt or Report*.html.
			The alias for this parameter is Filter.
		.PARAMETER NoRecurse
			This is a switch that indicates whether the cmdlet will process files in subdirectories
			underneath the specified source directory. By default, recursion is on. Optional.
		.PARAMETER Force
			This is a switch that indicates whether files that already exist at the destination
			and are readonly will be overwritten. By default they are not overwritten. Optional.
		.EXAMPLE
			PS C:\> Move-OldFiles -Source C:\Application\Log -Destination \\NASServer\Archives -OlderThan 90 -Filter *.log
		.EXAMPLE
			PS C:\> Move-OldFiles C:\Logs \\NASServer\Archives 90 *.log
		.EXAMPLE
			PS C:\> Move-OldFiles -SourceDirectory C:\Logs -DestinationDirectory \\NAS\Archives -Age 31 -Pattern *.log -Force
		.EXAMPLE
			PS C:\> Move-OldFiles C:\Logs \\NAS\Archives
	#>
	[CmdletBinding()]
	Param([Parameter(Position = 0, Mandatory = $True, HelpMessage = 'Source directory, e.g. C:\Logs')]
			[ValidateScript({Test-Path $_ -PathType Container})]
			[Alias('Source')]
			[String]$SourceDirectory,
	      [Parameter(Position = 1, Mandatory = $True, HelpMessage = 'Destination directory, e.g. \\NASServer\Archives')]
			[ValidateScript({Test-Path $_ -PathType Container})]
			[Alias('Destination')]
			[String]$DestinationDirectory,
		  [Parameter(Position = 2, Mandatory = $False, HelpMessage = 'The number of days old the file must be in order to be moved.')]
			[ValidateScript({$_ -GT 0})]
			[Alias('Age')]
			[Int]$OlderThan = 90,
		  [Parameter(Position = 3, Mandatory = $False, HelpMessage = 'The file pattern to match, e.g. *.log')]
			[Alias('Filter')]
			[String]$Pattern = "*",
		  [Parameter(Position = 4, Mandatory = $False, HelpMessage = 'Disable directory recursion, i.e. only copy the directory specified.')]
			[Switch]$NoRecurse = $False,
		  [Parameter(Position = 5, Mandatory = $False, HelpMessage = 'Specify to overwrite existing readonly files at the destination.')]
			[Switch]$Force = $False)
	
	$Start = Get-Date
    If(!($SourceDirectory.EndsWith("\")))
    {
	    $SourceDirectory = $SourceDirectory + "\"
    }
    If(!($DestinationDirectory.EndsWith("\")))
    {
        $DestinationDirectory = $DestinationDirectory + "\"
    }
	
	Write-Verbose "Source Directory:       $SourceDirectory"
	Write-Verbose "Destination Directory:  $DestinationDirectory"
	Write-Verbose "Move Files Older Than:  $OlderThan Days"
	Write-Verbose "Filename Filter:        $Pattern"
	Write-Verbose "Exclude Subdirectories: $NoRecurse"
	Write-Verbose "Overwrite if Readonly:  $Force"
	
	If($NoRecurse)
	{
		$SourceFiles = Get-ChildItem -Path $SourceDirectory -Filter $Pattern -File | Where-Object LastWriteTime -LT (Get-Date).AddDays($OlderThan * -1)
		Write-Verbose "$($SourceFiles.Count) files found in $SourceDirectory matching pattern $Pattern older than $OlderThan days."
	}
	Else
	{
		$SourceFiles = Get-ChildItem -Path $SourceDirectory -Filter $Pattern -File -Recurse | Where-Object LastWriteTime -LT (Get-Date).AddDays($OlderThan * -1)
		Write-Verbose "$($SourceFiles.Count) files found in $SourceDirectory matching pattern $Pattern older than $OlderThan days."
	}
	
	[Int]$FilesMoved = 0
	ForEach($File In $SourceFiles)
	{
		Write-Verbose "Moving $($File.FullName)"
		$DestinationFullName = $DestinationDirectory + $($File.FullName).Replace($SourceDirectory, $null)
		$DestinationFileDirectory = $DestinationFullName.Replace($DestinationFullName.Split('\')[-1], $null)
		If($PSBoundParameters['Verbose'])
		{
			Write-Progress -Activity "Move-OldFiles" `
						   -Status "Moving files..." `
						   -CurrentOperation "Transferring $($File.FullName)`..." `
						   -PercentComplete $([Math]::Round($FilesMoved / $SourceFiles.Count * 100, 0))
		}		
		If($Force)
		{
			If(!(Test-Path $DestinationFileDirectory -PathType Container))
			{
				Write-Verbose "Creating directory $DestinationFileDirectory"
				New-Item $DestinationFileDirectory -Type Directory | Out-Null
			}
		    Move-Item -Path $File.FullName -Destination $DestinationFullName -Force | Out-Null
		}
		Else
		{
			If(!(Test-Path $DestinationFileDirectory -PathType Container))
			{
				Write-Verbose "Creating directory $DestinationFileDirectory"
				New-Item $DestinationFileDirectory -Type Directory | Out-Null
			}
		    Move-Item -Path $File.FullName -Destination $DestinationFullName | Out-Null
		}
		$FilesMoved++
	}
	$End = Get-Date
	Write-Verbose "$($SourceFiles.Count) files were moved in $([Math]::Round(((New-TimeSpan $Start $End).TotalSeconds), 1)) seconds."
}

Get-FQDNInfo.ps1

by Ryan 22. April 2013 09:13

Someone recently asked me if I could write a script for them.  He had a list of several hundred fully qualified domain names (internet URLs essentially) in a file, and he had to get the IP address(es) of each FQDN and then some whois information about those IP addresses.  Running down a list of names and resolving them scriptomatically is a breeze of course, but the whois stuff sounded a little more tricky.  Luckily, ARIN has a sweet REST API ready to go, that made the whole script a snap.

I took the time to return all that data as objects so the output can be pipelined, and there is also an optional "save to CSV" parameter.  I think there are a couple more ways in which the script could be improved, but it works for now.  The output looks like this:

Get-FQDNInfo.ps1

And here's the whole script:

<#
.SYNOPSIS	
	Feed me a bunch of FQDNs, one per line, and I will give you as much info as
	I can about that IP address.
	
.DESCRIPTION
	This script takes an input file. The input file contains a list of FQDNs, one per line.
	With each FQDN, the script will attempt to resolve the name, and then find as much
	info as it can using ARIN REST services.
	
.PARAMETER InFile
	Specify a text file containing the FQDNs you want to scan.
	Each FQDN goes on a separate line. For example:
	
	host.foo.com
	barney.purpledinosaur.com
	et.phonehome.org

.PARAMETER OutFile
	Optional file to write the results to.

.INPUTS
	[System.String]$InFile - The name of the input file to read.
.INPUTS
	[System.String]$OutFile - Optional, the name of the output file to write.

.OUTPUTS
	[System.Object]$FQDNInfoCollection - Contains resolved FQDNInfo objects.

.OUTPUTS
	[System.IO.File]$OutFile - Optional, the output file to write.
	
.EXAMPLE
	PS C:\> .\Get-FQDNInfo.ps1 .\fqdns.txt outfile.txt

.EXAMPLE
	PS C:\> "fqdns.txt" | .\Get-FQDNInfo.ps1
	
.NOTES
	Name  : Get-FQDNInfo.ps1
	Author: Ryan Ries
	Email : ryanries09@gmail.com
	Date  : April 19, 2013

.LINK	
	http://www.myotherpcisacloud.com
#>

Param([Parameter(Mandatory=$True,ValueFromPipeline=$True)][ValidateScript({Test-Path $_ -PathType Leaf})][String]$InFile, [Parameter(Mandatory=$False,ValueFromPipeline=$False)][String]$OutFile)
$FQDNInfoCollection = @()
$EntriesProcessed = 0
Foreach($FQDN In Get-Content $InFile)
{
	$FQDNInfo = New-Object System.Object
	$FQDNInfo | Add-Member -Type NoteProperty -Name "FQDN"        -Value $FQDN
	$FQDNInfo | Add-Member -Type NoteProperty -Name "AddressList" -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "CSVSafeList" -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "NetRange"    -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "CIDR"        -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "NetName"     -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "NetType"     -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "RegDate"     -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "Updated"     -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "Comment"     -Value $null
	$FQDNInfo | Add-Member -Type NoteProperty -Name "SOA"         -Value $null
	
	Try	{ $FQDNInfo.AddressList = $([System.Net.Dns]::GetHostEntry($FQDN)).AddressList }
	Catch {	}
	If($FQDNInfo.AddressList -ne $null)
	{
		ForEach($A In $FQDNInfo.AddressList) { $FQDNInfo.CSVSafeList += "$($A)|" }
		$FQDNInfo.CSVSafeList = $FQDNInfo.CSVSafeList.TrimEnd('|')		
		Try
		{
			$ARINData = $(Invoke-WebRequest http://whois.arin.net/rest/ip/$($FQDNInfo.AddressList[0].ToString())`.txt).Content
			$ARINData = $ARINData.Split([Environment]::NewLine)
			Foreach($l In $ARINData)
			{
				If($l.StartsWith("NetRange:"))    { $FQDNInfo.NetRange = $l.SubString(16) }
				Elseif($l.StartsWith("CIDR:"))    { $FQDNInfo.CIDR     = $l.SubString(16) }
				Elseif($l.StartsWith("NetName:")) { $FQDNInfo.NetName  = $l.SubString(16) }
				Elseif($l.StartsWith("NetType:")) { $FQDNInfo.NetType  = $l.SubString(16) }
				Elseif($l.StartsWith("RegDate:")) { $FQDNInfo.RegDate  = $l.SubString(16) }
				Elseif($l.StartsWith("Updated:")) { $FQDNInfo.Updated  = $l.SubString(16) }
				Elseif($l.StartsWith("Comment:")) 
				{ 
					$FQDNInfo.Comment += $l.SubString(16)
					$FQDNInfo.Comment += " "
				}
			}
		}
		Catch { }
		& nslookup -q=soa $FQDN 2>&1> $Env:UserProfile`\temp.txt
		Foreach($_ In Get-Content $Env:UserProfile`\temp.txt)
		{
			If($_.Contains("primary name server =")) { $FQDNInfo.SOA = $_.Split('=')[1].Trim() }
		}		
	}	
	$FQDNInfoCollection += $FQDNInfo
	$EntriesProcessed   += 1
	Write-Host $EntriesProcessed "FQDNs processed."
}

If($OutFile.Length -gt 0)
{
	$FQDNInfoCollection | Export-CSV $OutFile -NoTypeInformation	
}
return $FQDNInfoCollection

Tags:

Powershell

ShareDiscreetlyWebServer v1.0.0.3

by Ryan 9. April 2013 13:17

I wrote a web service.  I call it "ShareDiscreetly".  Creative name, huh?

I wrote the server in C# .NET 4.0.  It runs as a Windows service.

ShareDiscreetlyWebServer serves a single purpose: to allow two people to share little bits of information - secrets - such as passwords, etc., in a secure, discreet manner.  The secrets are protected both in transit and at rest, using the FIPS-approved AES-256 algorithm with asymmetric keys supplied by an X.509 certificate.

Oh, and I made sure that it's thoroughly compatible with Powershell so that the server can be used in a scriptable/automatable way.

You can read a more thorough description of the server as you try it out here.

Please let me know if you find any bugs, exploits, or if you have any feature requests!

Powershell: Get Content Faster with ReadCount!

by Ryan 3. April 2013 13:41

Do you use Powershell?  Do you use Get-Content in Powershell to read files?  Do you sometimes work with large text files?

If you answered yes to any of the questions above, then read on - this post is for you!

I have a very simple tip that I used today in a script I was writing.  Thought I'd share.

Let's say you have a large text file, such as a packet log from a DNS server that you're debugging.  It might be 300 megabytes and millions of lines.  I was writing a script to parse the file and collect some statistics that I was after.

$LogFile = Get-Content $FileName
ForEach($_ In $LogFile)
{
    Do-Stuff
}

When I ran this script against a 52MB text file, the script executed in about 22 seconds.  When I ran the script on a 150MB text file, Powershell proceeded to consume over 3GB of RAM within a few seconds, the script never finished, and after bringing my laptop (Win7 x64, 4GB RAM, 4CPU, PS v3, .NET 4.5) to a crawl for about 5 minutes, Powershell just gave up and returned to the prompt without outputting anything.  I guess it was some sort of memory leak.  But come on... a 150MB file is not even that big...

So I started looking through the help for Get-Content, and it turns out there's an easy workaround:

$LogFile = Get-Content $FileName -ReadCount 0
ForEach($_ In $LogFile)
{
    Do-Stuff
}

The -ReadCount parameter specifies how many lines of content are sent through the pipeline at a time. The default is 1. A value of 0 sends all of the content through at one time.

Now when I run the script against the 52MB file, it completes in 2.8 seconds, and when I run it on the 150MB text file, it finishes in 10.2 seconds!

Tags:

Powershell

DNS over HTTP

by Ryan 31. March 2013 12:39

I was discussing with some fellow IT admins, the topic of blocking certain websites so that employees or students couldn't access them from the work or school network.  This is a pretty common topic for IT in most workplaces.  However, I personally don't want to be involved in it.  I realize that at some places, like schools for instance, filtering of some websites may be a legal or policy requirement.  But at the workplace, if an employee wants to waste company time on espn.com, that is an issue for HR and management to take up with that employee.  And again in my opinion, it's not about how much time an employee spends on ESPN or Reddit either, but simply whether that employee delivers satisfactory results.  I don't want to handle a people problem with a technical solution.  I don't want to be the IT guy that derives secret pleasure from blocking everyone from looking up their fantasy football scores.  (Or whatever it is people do on espn.com.)  I could spend my entire career until I retire working on a web proxy, blocking each and every new porn site that pops up.  If there's one thing the internet has taught me, it's that there will always be an infinite number of new porn sites.

On the other extreme of black listing, someone then suggested white listing.  Specifically, implementing "DNS white listing" in their environment for the purpose of restricting what internet sites users were allowed to access to only a handful of internet sites.  Well that is a terrible idea.  The only proper way of doing this in my opinion is to use a real web proxy, such as ISA or TMG or Squid.  But I could not help but imagine how I might implement such a system, and then how I might go about circumventing it from the perspective of a user.

OK, well for my first half-baked idea, I can imagine standing up a DNS server, disabling recursion/forwarders on that DNS server, and putting my "white list" of records on that DNS server.  Then, by way of firewall, block all port 53 access to any other IP except my special DNS server.  Congratulations, you just made your users miserable, and have done almost nothing to actually improve the security of your network or prevent people from accessing other sites.  Now the users just have to find another way of acquiring IP addresses for sites that aren't on your white list.

Well how do I get name resolution back if I can't use my DNS server?  I have an idea... DNS over HTTP!

The guys at StatDNS have already thought about this.  And what's awesome, is that they've created a web API for resolving names to IPs over HTTP.  Here's what I did in 5 minutes of Powershell:

PS C:\> Function Get-ARecordOverHTTP([string]$Query) { $($($(Invoke-WebRequest http://api.statdns.com/$Query/a).Content | ConvertFrom-Json).Answer).rdata }

PS C:\> Get-ARecordOverHTTP google.com
173.194.70.101
173.194.70.100
173.194.70.138
173.194.70.102
173.194.70.139
173.194.70.113

PS C:\> Get-ARecordOverHTTP myotherpcisacloud.com
168.61.52.184

Simple as that. How cool is Powershell, seriously?  One line to create a function that accepts a name and returns a list of IPs by interacting with an internet web service.  Pretty awesome if you ask me.

As long as you have port 80 open to StatDNS, you have internet name resolution.  Now, to wrap this into a .NET-based Windows service...

Excuse Me Sir, You Got Your Rally In My Powershell

by Ryan 28. March 2013 18:04

If you work in the tech industry, especially in the area of software development, you might use or at least be familiar with Rally. It's a web-based project management tool that follows the principles of the Agile development lifecycle.  I've even seen Rally used to organize and track projects that were not actually software development projects.  I've seen this because I do not specifically work in the area of software development, yet I use Rally anyway.

I don't really care much for its web interface though. In my opinion, it often takes way more clicking around in the GUI than it should. Sometimes the GUI isn't intuitive and you end up making mistakes like deleting an entire User Story when you meant to just delete a single Task.

So I started thinking to myself.  "Self," I said. "I wonder if Rally has a REST API? If they did, I could interface with it in Powershell and automate a lot of simple tasks."

Well lo and behold they sure do.  Let's see how freaking simple this is with Powershell:

$Rally = New-WebServiceProxy https://rally1.rallydev.com/slm/webservice/1.41/meta/132645/rally.wsdl -Crednetial (Get-Credential) -Namespace Rally

And just like that, we're connected to Rally.  To verify, you could check the currently logged on user:

$Rally.getCurrentUser()

Now, let's pull some data from Rally. Let's retrieve, say, my 20 most recently-created Tasks in Rally:

PS C:\> $Query = $Rally.Query($null, "Task", "(Owner.Name = `"ryan@domain.com`")", "CreationDate desc", $True, 1, 20)
PS C:\> $Query.Results.Name

Write some code
Write some more code
Fix this defect
Fix that defect
Fix all the defects
Save the company from bankruptcy
Reconcile marriage
Catch the guy who keeps running the copier out of paper and not refilling it
etc.

Playing with REST services and web services is so delightfully easy with Powershell

About

Name: Ryan Ries
Location: Texas, USA
Occupation: Systems Engineer 

I am a Windows engineer and Microsoft advocate, but I can run with pretty much any system that uses electricity.  I'm all about getting closer to the cutting edge of technology while using the right tool for the job.

This blog is about exploring IT and documenting the journey.


Blog Posts (or Vids) You Must Read (or See):

Pushing the Limits of Windows by Mark Russinovich
Mysteries of Windows Memory Management by Mark Russinovich
Accelerating Your IT Career by Ned Pyle
Post-Graduate AD Studies by Ned Pyle
MCM: Active Directory Series by PFE Platforms Team
Encodings And Character Sets by David C. Zentgraf
Active Directory Maximum Limits by Microsoft
How Active Directory Replication Topology Works by Microsoft


MCITP: Enterprise Administrator

Profile for Ryan Ries at Server Fault, Q&A for system administrators

Twitter

LOPSA

 

I do not discuss my employers on this blog and all opinions expressed are mine and do not reflect the opinions of my employers.