Ygtjki

We can do this using the paragraph mode of Perl wherein we read in paragraph sized chunk of files as records and build up a hash keyed on IDs and values as time taken. At the end of file, we reverse sort numerically on the times taken and then print the desired message.

$ perl -ln -00 -e '
 %h = (%h, /^Req_id=(d+)n.*^Time_taken=(d+)/ms){
 print "The below IDs took more than expected:" if scalar keys %h;
 print join " ", "Req ID:", "$_," , "Time taken", $h$_, "sec"
 for (sort $h$b <=> $h$a keys %h)[0..2];
' input.file

Output:

The below IDs took more than expected:
Req ID: 3456, Time taken 200 sec
Req ID: 3001, Time taken 15 sec
Req ID: 1234, Time taken 10 sec

you need to remember request number.

I would use (this could be one lined)

awk -F= '$1 == "Req_id" r=$2 ; 
 $1 == "Time_taken" printf "Request id %s %s %sn",r,$1,$2 ; ' file |
sort -r -n -k5 |
head -3

which give

Request id 3456 Time_taken 200 sec
Request id 3001 Time_taken 15 sec
Request id 1234 Time_taken 10 sec

where

• 
  -F= use = as separator


• 
  $1 == "Req_id" r=$2 ; get last request id


• 
  $1 == "Time_taken" if line is "time taken"


• 
  printf "Request id %s %s %sn",r,$1,$2 ; print request id and seconds


• 
  | sort pipe to sort


• 
  -r reverse order


• 
  -n numeric sort (e.g. 200 greater than 15)


• 
  -k5 on 5th field


• 
  | head -3 get first 3 lines

I would prefer Python over awk for anything except one-liners. Using a Python script will allow you to more easily handle poorly formatted input and to perform logging and error-handling.

Here is a basic Python script that will produce the desired output for your example input, and which has a structure which is suggestive of how you might build it out further if needed:

#!/usr/bin/env python3
# -*- encoding: utf-8 -*-
"""parse_log.py"""

import sys
from collections import OrderedDict

logfilepath = sys.argv[1]

# Define a function to parse a single block/entry in the log file
def parse_block(block):
 parsed_block = dict()
 lines = block.split("n")
 for line in lines:
 if line.startswith("Company="):
 parsed_block["Company"] = line[8:]
 elif line.startswith("Req_id="):
 parsed_block["Required_ID"] = line[7:]
 elif line.startswith("Time_taken="):
 parsed_block["Time_Taken"] = line[11:]
 elif line.startswith("Status="):
 parsed_block["Status"] = line[7:]
 else:
 pass
 return parsed_block

# Initialize a list to store the processed entries
parsed_blocks = list()

# Populate the list
with open(logfilepath, "r") as logfile:
 blocks = logfile.read().split("nn")
 for block in blocks:
 parsed_block = parse_block(block)
 parsed_blocks.append(parsed_block)

# Print the results
print("Below Request Id took more then expected")
for parsed_block in parsed_blocks:
 print("Request id , Time Taken: ".format(parsed_block["Required_ID"], parsed_block["Time_Taken"]))

You could run it like this:

python parse_log.py data.log

On your example input, it produces the following output (as requested):

Below Request Id took more then expected
Request id 1234, Time Taken: 10 sec
Request id 3456, Time Taken: 200 sec
Request id 3001, Time Taken: 15 sec