Wrapping up lines between pattern (date/timestamp) in log files – Parse

this is definitely not the best way to do this, but it works

Example log file before parse, the lines in bold are those that are going to be wrapped up to one line

2019-04-10 23:46:31.397 DEBUG  HOOKS_CALLOUT_CALLED hooks
2019-04-11 00:25:59.191 DEBUG [hwtype=1 02:04:61:20:20:2c]
type=001, len=004: 4294967040 (uint32)
type=003, len=004: 185.88.229.1
2019-04-11 00:26:00.192 DEBUG HOOKS_CALLOUTS_BEGIN
2019-04-11 00:26:00.192 DEBUG HOOKS_CALLOUT_CALLED

example after parse script

2019-04-10 23:46:31.397 DEBUG  HOOKS_CALLOUT_CALLED hooks
2019-04-11 00:25:59.191 DEBUG [hwtype=1 02:04:61:20:20:2c] type=001, len=004: 4294967040 (uint32) type=003, len=004: 185.88.229.1
2019-04-11 00:26:00.192 DEBUG HOOKS_CALLOUTS_BEGIN
2019-04-11 00:26:00.192 DEBUG HOOKS_CALLOUT_CALLED

Regex used in this script matches logfiles that have date in the beginning of lines formated in YYYY-MM-DD, this can be changed, just change the regex variable

### Regex to match ####
regex='^20[1-2][0-9]-[0-1][1-9]-[0-3][0-9] '

### Input/Output Files ####
input_file='test.txt'
output_file='output.txt'

### Script Start ###

echo "" > "$output_file"
while read p; do
          if [[ ! $p =~ $regex ]]; then
                  a="$a $p"
          elif [[ $p =~ $regex ]] && [[ ! $a =~ "aaaaa" ]]; then
                  a="aaaaa - $p"
          elif [[ $p =~ $regex ]] && [[ $a =~ "aaaaa" ]]; then
                  echo "$a" >> $output_file
                  a="aaaaa - $p"
          fi
done <$input_file 
echo $a >> $output_file
sed 's/aaaaa - //' -i $output_file
cat $output_file

Leave a Reply

Your email address will not be published. Required fields are marked *