Log Parser

This Log Parser suite was created over the years based on personal experience and suggestions from friends and colleagues. 

Parse Server Side Log

This utility was written to allow a single thread to be pulled from a multi-thread log file.  This program has one required command line parameter, and one optional.  The first parameter is the name of the log file to be processed. The second is optional, and is the thread ID to be pulled out of the log file.  If the second parameter is not provided, then it must be provided in the ParseServerSideLog.cfg file.  An example of calling this program from command line would be

java -cp .\lib\parse_logs.jar com.apl.ParseServerSideLog arfilter.log 0000008036 > output.log

This command line assumes that the jar file is located in a lib folder under the current folder, that the log file to be parsed is local in the current directory, and that thread 8036 exists as a thread in the input log.  It will output the values that match that thread into a redirected file of output.log.

If you want to utilize the program with the provided cfg file, the cfg file would be updated to contain

threadID:             0000008036

Then, you would be able to provide the following command line

java -cp .\lib\parse_logs.jar com.apl.ParseServerSideLog arfilter.log > output.log

This provides the same output as above, but allows for use of the ‘drag and drop’ functionality of log file onto batch file (provided in zip)

Parse String

This utility was written as a derivative of Parse Server Side Log.  But instead of being hard coded to just pull out a Thread ID, it is designed to be able to pull out any string you want to pull out of a text file.  Additionally, it has another parameter that is used that allows the string in question be removed from the resulting file instead of be the only content of the file.  This program requires the use of a command line parameter, and a config file named ParseString.cfg.  An example of calling this program from command line would be

java -cp .\lib\parse_logs.jar com.apl.ParseString arfilter.log > output.log

This command line assumes that the jar file is located in a lib folder under the current folder, that the log file to be parsed is local in the current directory.  The config file has two parameters:

string:   <string you want to look for>

You simply set the string to the value you want to look for

makeNegative: true/false

If set to true, the system will exclude any line that contains your string, if false, only your string will be returned

The batch file provided in the zip allows for  ‘drag and drop’ functionality of log file onto batch file.

Remove Failed/Disabled

This utility came about because of a discussion with a co-worker about how much easier it would be to read workflow logs if we didn’t need to look at all of the workflow that didn’t actually do anything, so this would be workflow that had a run-if that failed qualification, and there were no else actions, or things that were skipped because of GoTo actions, basically, making the file as clean as possible to only look at things that WERE happening.  This program has one command line parameter, which is the file you want to process

java -cp .\lib\parse_logs.jar com.apl.RemoveFailedDisabled arfilter.log > output.log

This command line assumes that the jar file is located in a lib folder under the current folder, that the log file to be parsed is local in the current directory.

The resultant output log file should be more concise than the file you started with

Remove Thread Info

This program came out of frustration of constantly needing to scroll left and right in my text editor to be able to see what workflow was doing, especially in the case of Active Links and Filters being logged in the same file.  This program leaves the <Function> tags at the beginning of each line, but removes everything between that and the actual data in the log, removing all of the thread and related information.  This program has one command line parameter, which is the file you want to process

java -cp .\lib\parse_logs.jar com.apl.RemoveThreadInfo arfilter.log > output.log

This command line assumes that the jar file is located in a lib folder under the current folder, that the log file to be parsed is local in the current directory.

The resultant output log file should be more ‘horizontally compact’ than the original file.  You may want to run this utility AFTER you have isolated the transaction in question, because the thread, rpc queue, user information will be removed after this utility is run

Split Server Side Log

This utility came because of a need to not parse out a single thread from a server side log, but to be able to take all threads and put them into separate files.  We all know that this can be done at the time of logging by turning on the ‘Log Per Thread’ option on the logging tab, but I have found that option to be ‘ugly’ for most logging, and have historically not used the option much, but this program gives you the ability to take a single log file produced without that option turned on, and generate the same files it would have if the option was turned on at capture time.  This program has one command line parameter, which is the file you want to process

java -cp .\lib\parse_logs.jar com.apl.SplitServerSideLog arfilter.log

This command line assumes that the jar file is located in a lib folder under the current folder, that the log file to be parsed is local in the current directory.

This program is a bit different than the others because you aren’t providing an output file redirect. This program creates one file for each thread in the source file, and names the file with the thread id.log as the name.  The output from this program on the display is relatively simple.  For every 1000 lines it processes, it puts a . on the screen, and it puts 100 dots per line, making each line worth 100,000 log lines that it parsed.  It keeps track of the number of lines that it has output to the screen so you can keep rough count of how many lines it has processed.

Transform RegEx

This program began as a derivative of the Remove Thread Info program.  What this program does is it takes a ‘Regular Expression’, or RegEx, and performs a translation on the source row converting what you search for into what you want it to be.  This program has one command line parameter, which is the file you want to process

java -cp .\lib\parse_logs.jar com.apl.TransformRegEx arfilter.log > output.log

This command line assumes that the jar file is located in a lib folder under the current folder, that the log file to be parsed is local in the current directory.

This program requires the TransformRegEx.cfg to be in the same location as the program.  This config file has two parameters:

regEx: > <TID.* > /\\*

regEx is the Java Regular Expression that you want to search a line for, the example above is looking for the string “ > <TID”, then the .* indicates to look as far as it needs with as many characters as possible (the last instance of) “ > /*”….but because this is programming, and * is a ‘special’ character to Regular Expressions, the * needs to be turned into \* (escaping the character)….then to double whammy things, \ is also a special character, so in this case, when looking for “ > /”, we need to store “ > /\\*”, which properly translates at the Java level from config file to program.

replace: > /\\*

replace is just string we want it to replace the string it found with, remembering that we need to escape ‘special’ RegEx characters. The above regEx and replace transform

<FLTR> <TID: 0000035508> <RPC ID: 0057089889> <Queue: Admin     > <Client-RPC: 390680   > <USER: Remedy Application Service > /* Tue Jul 30 2013 08:18:04.1380

Into

<FLTR> /* Tue Jul 30 2013 08:18:04.1380

This utility suite was presented @ WWRUG13.  I hope you all find it as useful as I have over the years.

Version History

2.0 – Feb 18 2020

• Updated the code in the Split program, works about twice as fast as before
• Added syntax to all of the programs
• Made all programs jar files for easier execution

1.1 – October 30 2013

• Removed trim from ‘RemoveThreadInfo’ because it was messing up 8.1 Web Logging indentation
• Updated ‘RemoveFailedDisabled’ to handle Failed/Disabled Active Links in 8.1 Web Logging

1.0 – October 1 2013

• Initial Release of 6 tools
• ParseServerSideLog
• ParseString
• RemoveFailedDisabled
• RemoveThreadInfo
• SplitServerSideLog
• TransformRegEx