Skip to content

Duplicate Files With PowerShell

February 26, 2012

Probably one of the coolest script by looking into the content comparison from
http://blog.codeassassin.com/2007/10/13/find-duplicate-files-with-powershell/

param ([string] $Path = (Get-Location))

function Get-MD5([System.IO.FileInfo] $file = $(throw ‘Usage: Get-MD5 [System.IO.FileInfo]‘))
{
# This Get-MD5 function sourced from:
# http://blogs.msdn.com/powershell/archive/2006/04/25/583225.aspx
$stream = $null;
$cryptoServiceProvider = [System.Security.Cryptography.MD5CryptoServiceProvider];
$hashAlgorithm = new-object $cryptoServiceProvider
$stream = $file.OpenRead();
$hashByteArray = $hashAlgorithm.ComputeHash($stream);
$stream.Close();

## We have to be sure that we close the file stream if any exceptions are thrown.
trap
{
if ($stream -ne $null) { $stream.Close(); }
break;
}

return [string]$hashByteArray;
}

$fileGroups = Get-ChildItem $Path -Recurse `
| Where-Object { $_.Length -gt 0 } `
| Group-Object Length `
| Where-Object { $_.Count -gt 1 };

foreach ($fileGroup in $fileGroups)
{
foreach ($file in $fileGroup.Group)
{
Add-Member NoteProperty ContentHash (Get-MD5 $file) -InputObject $file;
}

$fileGroup.Group `
| Group-Object ContentHash `
| Where-Object { $_.Count -gt 1 };
}


http://powershell.com/cs/blogs/tips/archive/2008/11/17/finding-duplicate-files.aspx
function Find-Duplicates {   $Input | ForEach-Object {      if ($lookup.ContainsKey($_.Name.toLower()))       {          "$($_.FullName) is a duplicate in $($lookup.$($_.Name.toLower()))"       } else       {          $lookup.Add($_.Name.toLower(), $_.FullName)       }   }}$lookup = @{}dir | Find-Duplicatesdir $env:windir | Find-Duplicatesdir $env:windir\system32 | Find-Duplicates


http://powershell.com/cs/blogs/tips/archive/2008/11/17/finding-duplicate-files.aspx


Bottom line.  Need more PowerShell practice.

Advertisements
Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: