- Folder Structure
- The CrawlLogs_Scripts folder
- Create the data counter collection
- Crawlcounters.config file:
- Create the Start_Collector task:
- Create the Stop_Collector task:
- Create the set-execpolicy unrestricted task:
- Run the deletelogs powershell script:
- Create the set-execpolicy allsigned task:
This post describes a fully scripted perfmon counter collection deployment used to collect data about SharePoint crawls - complete with scheduled tasks to run the data collection and a PowerShell script to remove old logs
First, on the Index server create 2 folders and grant everyone the Modify permission –
Next I use XCOPY to copy the CrawlLogs_Scripts from my desktop to the server:
cmd.exe /c XCOPY "\_SOURCE\Crawllogs_scripts" "I:\LogFiles\Crawllogs_scripts" /E /V /I /Y > "c:\logs\copy_Crawllogs_scripts.log"
The CrawlLogs_Scripts folder
This contains the PowerShell scripts, batch files and config file that will be used to configure crawl logging
Create the data counter collection
Do this by running the create_data_collector.bat – the contents of this file are listed below – this creates the “Osearch-crawl-counters” counter, in .csv format, named “crawllog”, in the “I:\Logfiles\CrawlLogs” path, using the crawlcounters.config file
logman create counter Osearch-crawl-counters -f csv -max 250 -cnf -si 00:00:15 -o "I:\LogFiles\Crawllogs\crawllog" -cf "I:\LogFiles\Crawllogs_scripts\crawlcounters.config"
This is used to list the perfmon counters that will be logged – in this case:
"\Office Server Search Gatherer\*"
"\Office Server Search Archival Plugin(Portal_Content)\*"
"\Office Server Search Gatherer Projects(Portal_Content)\*"
"\Office Server Search Indexer Catalogs(Portal_Content)\*"
"\Office Server Search Schema Plugin(Portal_Content)\*"
The crawl in our environment runs every weekday from 17:00 – so we’ll now set the counters to collect data over this period
Create the Start_Collector task:
Now we create a scheduled task to run the “start_collector.bat” at 16:30 Mon-Fri using the SPFarm account credentials
cmd.exe /c if not exist C:\WINDOWS\tasks\Start_Collector.job SCHTASKS /create /tn "Start_Collector" /tr "I:\LogFiles\Crawllogs_scripts\start_collector.bat" /sc weekly /D "MON,TUE,WED,THU,FRI" /st 16:30 /ru domain\SPfarm /rp password > "c\logs\add-schtask-startcollector.log"
Create the Stop_Collector task:
Now we create a scheduled task to run the “stop_collector.bat” at 04:30 Mon-Fri using the SPFarm account credentials
cmd.exe /c if not exist C:\WINDOWS\tasks\Stop_Collector.job SCHTASKS /create /tn "Stop_Collector" /tr "I:\LogFiles\Crawllogs_scripts\stop_collector.bat" /sc weekly /D "MON,TUE,WED,THU,FRI" /st 04:30 /ru DOMAIN\SPfarm /rp password > "c:\logs\add-schtask-stopcollector.log"
Create the set-execpolicy unrestricted task:
Next we create a scheduled task to set the PowerShell execution policy to unrestricted at 05:29 Mon-Fri – this will enable us to then run the deletelogs.ps1 – a script that deletes counter logs over 10 days old and thus prevents the drive from filling up
cmd.exe /c if not exist C:\WINDOWS\tasks\Set-execpolicy-un.job SCHTASKS /create /tn "Set-execpolicy-un" /tr "I:\LogFiles\Crawllogs_scripts\set_un.bat" /sc weekly /D "MON,TUE,WED,THU,FRI" /st 05:29 /ru DOMAIN\SPfarm /rp password > "c:\install\logs\add-schtask3.log"
Run the deletelogs powershell script:
This task runs 1 minute after the execution policy has been set to unrestricted
cmd.exe /c if not exist C:\WINDOWS\tasks\deletelogs.job SCHTASKS /create /tn "deletelogs" /tr "I:\LogFiles\Crawllogs_scripts\deletelogs.bat" /sc weekly /D "MON,TUE,WED,THU,FRI" /st 05:30 /ru DOMAIN\SPfarm /rp password > "c:\install\logs\add-schtask4.log"
This task runs the deletelogs.bat:
powershell -command "& 'I:\LogFiles\Crawllogs_scripts\deletelogs.ps1' "
The deletlogs.bat runs the deletelogs.ps1
$TargetFolder = "I:\LogFiles\Crawllogs"
$cFiles = get-childitem $TargetFolder
foreach ($cFile in $cFiles)
If ($cFile.CreationTime -lt ((get-date).AddDays(-10)))
write-host "Deleting File $cfile" -foregroundcolor "Red"
$targetfile = "$TargetFolder\$cfile"
Create the set-execpolicy allsigned task:
This task resets the PowerShell execution policy at 05:35 - after the deletelogs.ps1 has been run
cmd.exe /c if not exist C:\WINDOWS\tasks\Set-execpolicy-re.job SCHTASKS /create /tn "Set-execpolicy-re" /tr "I:\LogFiles\Crawllogs_scripts\set_re.bat" /sc weekly /D "MON,TUE,WED,THU,FRI" /st 05:35 /ru DOMAIN\SPfarm /rp password > "c:\install\logs\add-schtask5.log"
It runs the set_re.bat:
powershell -command set-executionpolicy allsigned
At the end of all this you get log files for the last 10 days that detail all major criteria of crawl activity – very useful
Hope this helps – I did all this because next I added the PDF Ifilter and I wanted to measure the impact
In the next blog I’ll go over how I scripted the PDF IFilter install on a MOSS 2007 environment