Ygtjki

Hostile work environment after whistle-blowing on coworker and our boss. What do I do?

Efficiently merge handle parallel feature branches in SFDX

Is there any reason not to eat food that's been dropped on the surface of the moon?

What would be the benefits of having both a state and local currencies?

At which point does a character regain all their Hit Dice?

How can I get through very long and very dry, but also very useful technical documents when learning a new tool?

Where in the Bible does the greeting ("Dominus Vobiscum") used at Mass come from?

Is it okay / does it make sense for another player to join a running game of Munchkin?

Can a monster with multiattack use this ability if they are missing a limb?

Is exact Kanji stroke length important?

Will it be accepted, if there is no ''Main Character" stereotype?

Mapping a list into a phase plot

Can somebody explain Brexit in a few child-proof sentences?

Applicability of Single Responsibility Principle

Why "be dealt cards" rather than "be dealing cards"?

Trouble understanding overseas colleagues

Is HostGator storing my password in plaintext?

I'm in charge of equipment buying but no one's ever happy with what I choose. How to fix this?

Products and sum of cubes in Fibonacci

Is there a good way to store credentials outside of a password manager?

Understanding "audieritis" in Psalm 94

Hide Select Output from T-SQL

The plural of 'stomach"

What is the oldest known work of fiction?

Find Consecutive error in file and list file names

Why *not* parse `ls` (and what do to instead)?Arg list too long error while using findshell script to do some text manipulation of text file data structure and slight content changesCount and merge consecutive patternshow to select specific range of line & count the specific occurence of the first charcter w.r.t. each unique second charcter?Process a file list with space in file namesList names of aliases, functions and variables in zshfinding a folder names based on list contained in .txt fileFile check for multiple file namesFind size of file names with space and hyphen | pass file names containing space and hyphen to “du”find files with similar names and then narrow down further

0

I got this project to analysis my logs where i need to find any consecutive errors occured more then thrice and list logs name . As i logs were huge , i wrote below script to help myself.

for i in `ls` 
 do 
 count=`uniq -c $i | grep 'FATAL ERROR' |sed 's/^ *//g'| sed 's/ /,/g' | awk -F',' 'if($1 >1) print $1'` 
 if [[ $count -ge 2 ]] 
 then 
 echo $i 
 fi 
done

This Helps if you just need to know number of occurrence in which all file

grep -o -c Source * | awk -F: 'if ($2 > 2)print $1' 

What could have been the better version of my script? What if log is really huge , my piece of code will slowdown.

shell-script shell

share|improve this question

edited yesterday

RoVo

3,407317

asked yesterday

MachineMachine

335

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  Do not parse ls and use $() instead of backticks.
  
  – RoVo
  yesterday
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @RoVo, Thanks a lot for your suggestion .
  
  – Machine
  18 hours ago
  

  
  
  
  

add a comment | 

0

I got this project to analysis my logs where i need to find any consecutive errors occured more then thrice and list logs name . As i logs were huge , i wrote below script to help myself.

for i in `ls` 
 do 
 count=`uniq -c $i | grep 'FATAL ERROR' |sed 's/^ *//g'| sed 's/ /,/g' | awk -F',' 'if($1 >1) print $1'` 
 if [[ $count -ge 2 ]] 
 then 
 echo $i 
 fi 
done

This Helps if you just need to know number of occurrence in which all file

grep -o -c Source * | awk -F: 'if ($2 > 2)print $1' 

What could have been the better version of my script? What if log is really huge , my piece of code will slowdown.

shell-script shell

share|improve this question

edited yesterday

RoVo

3,407317

asked yesterday

MachineMachine

335

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  
  Do not parse ls and use $() instead of backticks.
  
  – RoVo
  yesterday
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @RoVo, Thanks a lot for your suggestion .
  
  – Machine
  18 hours ago
  

  
  
  
  

add a comment | 

I got this project to analysis my logs where i need to find any consecutive errors occured more then thrice and list logs name . As i logs were huge , i wrote below script to help myself.

for i in `ls` 
 do 
 count=`uniq -c $i | grep 'FATAL ERROR' |sed 's/^ *//g'| sed 's/ /,/g' | awk -F',' 'if($1 >1) print $1'` 
 if [[ $count -ge 2 ]] 
 then 
 echo $i 
 fi 
done

This Helps if you just need to know number of occurrence in which all file

grep -o -c Source * | awk -F: 'if ($2 > 2)print $1' 

What could have been the better version of my script? What if log is really huge , my piece of code will slowdown.

shell-script shell

share|improve this question

edited yesterday

RoVo

3,407317

asked yesterday

MachineMachine

335

for i in `ls` 
 do 
 count=`uniq -c $i | grep 'FATAL ERROR' |sed 's/^ *//g'| sed 's/ /,/g' | awk -F',' 'if($1 >1) print $1'` 
 if [[ $count -ge 2 ]] 
 then 
 echo $i 
 fi 
done

grep -o -c Source * | awk -F: 'if ($2 > 2)print $1' 

share|improve this question

edited yesterday

RoVo

3,407317

asked yesterday

MachineMachine

335

share|improve this question

edited yesterday

RoVo

3,407317

asked yesterday

MachineMachine

335

edited yesterday

RoVo

3,407317

edited yesterday

RoVo

3,407317

asked yesterday

MachineMachine

335

asked yesterday

MachineMachine

335

1 Answer
1

active

oldest

votes

2

Pipelines should generally be kept short, and most of the things that you do to get the value for count could be done in a single awk program, including the loop and the if statement.

awk 'FNR == 1 || !/FATAL ERROR/ count = 0 
 /FATAL ERROR/ ++count 
 count == 2 print FILENAME ' ./*

This runs a single awk program across all non-hidden files in the current directory.

The program will reset its count variable to zero if we're at the first line of a new file or if the current line does not match the pattern FATAL ERROR.

If the line matches the pattern FATAL ERROR, the count variable is incremented.

If the count variable reaches a value of 2, the name of the current file is printed.

The code will print the name of the file each time it finds two consecutive lines that matches the pattern, even if this happens multiple times in the same file. If this in not wanted, you could expand the code slightly:

awk 'FNR == 1 count = 0; do_print = 1 
 !/FATAL ERROR/ count = 0 
 /FATAL ERROR/ ++count 
 count == 2 && do_print print FILENAME; do_print = 0 ' ./*

or with GNU awk (using nextfile to skip to the next file):

awk 'FNR == 1 || !/FATAL ERROR/ count = 0 
 /FATAL ERROR/ ++count 
 count == 2 && do_print print FILENAME; nextfile ' ./*

Related to your shell loop:

• Why *not* parse `ls` (and what do to instead)?

share|improve this answer

edited 17 hours ago

answered yesterday

Kusalananda♦Kusalananda

138k17258426

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Finding it hard to understand , but still trying :)
  
  – Machine
  18 hours ago
  
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f508516%2ffind-consecutive-error-in-file-and-list-file-names%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

2

Pipelines should generally be kept short, and most of the things that you do to get the value for count could be done in a single awk program, including the loop and the if statement.

awk 'FNR == 1 || !/FATAL ERROR/ count = 0 
 /FATAL ERROR/ ++count 
 count == 2 print FILENAME ' ./*

This runs a single awk program across all non-hidden files in the current directory.

The program will reset its count variable to zero if we're at the first line of a new file or if the current line does not match the pattern FATAL ERROR.

If the line matches the pattern FATAL ERROR, the count variable is incremented.

If the count variable reaches a value of 2, the name of the current file is printed.

The code will print the name of the file each time it finds two consecutive lines that matches the pattern, even if this happens multiple times in the same file. If this in not wanted, you could expand the code slightly:

awk 'FNR == 1 count = 0; do_print = 1 
 !/FATAL ERROR/ count = 0 
 /FATAL ERROR/ ++count 
 count == 2 && do_print print FILENAME; do_print = 0 ' ./*

or with GNU awk (using nextfile to skip to the next file):

awk 'FNR == 1 || !/FATAL ERROR/ count = 0 
 /FATAL ERROR/ ++count 
 count == 2 && do_print print FILENAME; nextfile ' ./*

Related to your shell loop:

• Why *not* parse `ls` (and what do to instead)?

share|improve this answer

edited 17 hours ago

answered yesterday

Kusalananda♦Kusalananda

138k17258426

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Finding it hard to understand , but still trying :)
  
  – Machine
  18 hours ago
  
  

  
  
  
  

add a comment | 

2

Pipelines should generally be kept short, and most of the things that you do to get the value for count could be done in a single awk program, including the loop and the if statement.

awk 'FNR == 1 || !/FATAL ERROR/ count = 0 
 /FATAL ERROR/ ++count 
 count == 2 print FILENAME ' ./*

This runs a single awk program across all non-hidden files in the current directory.

The program will reset its count variable to zero if we're at the first line of a new file or if the current line does not match the pattern FATAL ERROR.

If the line matches the pattern FATAL ERROR, the count variable is incremented.

If the count variable reaches a value of 2, the name of the current file is printed.

The code will print the name of the file each time it finds two consecutive lines that matches the pattern, even if this happens multiple times in the same file. If this in not wanted, you could expand the code slightly:

awk 'FNR == 1 count = 0; do_print = 1 
 !/FATAL ERROR/ count = 0 
 /FATAL ERROR/ ++count 
 count == 2 && do_print print FILENAME; do_print = 0 ' ./*

or with GNU awk (using nextfile to skip to the next file):

awk 'FNR == 1 || !/FATAL ERROR/ count = 0 
 /FATAL ERROR/ ++count 
 count == 2 && do_print print FILENAME; nextfile ' ./*

Related to your shell loop:

• Why *not* parse `ls` (and what do to instead)?

share|improve this answer

edited 17 hours ago

answered yesterday

Kusalananda♦Kusalananda

138k17258426

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Finding it hard to understand , but still trying :)
  
  – Machine
  18 hours ago
  
  

  
  
  
  

add a comment | 

Pipelines should generally be kept short, and most of the things that you do to get the value for count could be done in a single awk program, including the loop and the if statement.

awk 'FNR == 1 || !/FATAL ERROR/ count = 0 
 /FATAL ERROR/ ++count 
 count == 2 print FILENAME ' ./*

This runs a single awk program across all non-hidden files in the current directory.

The program will reset its count variable to zero if we're at the first line of a new file or if the current line does not match the pattern FATAL ERROR.

If the line matches the pattern FATAL ERROR, the count variable is incremented.

If the count variable reaches a value of 2, the name of the current file is printed.

The code will print the name of the file each time it finds two consecutive lines that matches the pattern, even if this happens multiple times in the same file. If this in not wanted, you could expand the code slightly:

awk 'FNR == 1 count = 0; do_print = 1 
 !/FATAL ERROR/ count = 0 
 /FATAL ERROR/ ++count 
 count == 2 && do_print print FILENAME; do_print = 0 ' ./*

or with GNU awk (using nextfile to skip to the next file):

awk 'FNR == 1 || !/FATAL ERROR/ count = 0 
 /FATAL ERROR/ ++count 
 count == 2 && do_print print FILENAME; nextfile ' ./*

Related to your shell loop:

• Why *not* parse `ls` (and what do to instead)?

share|improve this answer

edited 17 hours ago

answered yesterday

Kusalananda♦Kusalananda

138k17258426

awk 'FNR == 1 || !/FATAL ERROR/ count = 0 
 /FATAL ERROR/ ++count 
 count == 2 print FILENAME ' ./*

awk 'FNR == 1 count = 0; do_print = 1 
 !/FATAL ERROR/ count = 0 
 /FATAL ERROR/ ++count 
 count == 2 && do_print print FILENAME; do_print = 0 ' ./*

awk 'FNR == 1 || !/FATAL ERROR/ count = 0 
 /FATAL ERROR/ ++count 
 count == 2 && do_print print FILENAME; nextfile ' ./*

share|improve this answer

edited 17 hours ago

answered yesterday

Kusalananda♦Kusalananda

138k17258426

answered yesterday

Kusalananda♦Kusalananda

138k17258426

answered yesterday

Kusalananda♦Kusalananda

138k17258426