Extracting data from ULS logs using logparser

Had an interesting problem today. Received an ULS log file that was 18GB large. Although I knew the approximate timeframe where the problem started happening the normal PowerShell commands I use to break the logs did not work.

Basically the challenge was: how to extract a specific timeframe from an 18GB log file.

Answer(as suggested by the great CristiG): using logparser. Initially meant for IIS logs, we can use logparser also for ULS with a little bit of adaptation. The main challenge was parsing the Timeframe column that is not in the standard format that logparser understands.

The resulting script looks like:

select [Timestamp],Process,TID,Area,Category,EventID,Level,Message,Correlation into test3.log from problem.log
where TO_TIME(TO_TIMESTAMP(STRCAT(STRCAT(STRCAT(STRCAT(STRCAT(STRCAT(TRIM(SUBSTR(Timestamp,6,4)),'-'),TRIM(SUBSTR(Timestamp,0,2))),'-'),TRIM(SUBSTR(Timestamp,3,2))),' '),TRIM(SUBSTR(Timestamp,11,8))), 'yyyy-MM-dd hh:mm:ss')) between timestamp('01:33:00', 'hh:mm:ss') and timestamp('01:43:00', 'hh:mm:ss')

After installing logparser and adding it to your path variable you can run the following command to filter the logs: LogParser -i:tsv -o:tsv file:test2.sql (given that you have the input file and the script file are in the same directory).

Logparser can be downloaded for free from: http://www.microsoft.com/en-us/download/details.aspx?id=24659