Pass shell variable as a /pattern/ to awk2019 Community Moderator ElectionHow to pass shell variable to an awk search pattern?How to use a variable inside an awk statement?make awk print the line that match a variable and the next n lines and use a variable in awkAWK and Bash Scriptingusing awk with variablesPass Shell variable to awkawk search for string containing forward slash in variablePass a variable in another variable within an awkhow to use variable with awkHow to make variable be taken into accountGawk: Passing arrays to functionsHow can I convert this bash function to the fish shellBEGIN and END with the awk commandBest way to iterate through the following awk commandHow to pass shell variable to an awk search pattern?(Shell Script) Variable not pass correctly when fetching value from file?Use awk interactively through a pipePass Shell variable to awkusing awk with variablesDeclaring a variable that does not have input
Does Doodling or Improvising on the Piano Have Any Benefits?
What does "Scientists rise up against statistical significance" mean? (Comment in Nature)
Picking the different solutions to the time independent Schrodinger eqaution
Redundant comparison & "if" before assignment
What should you do when eye contact makes your subordinate uncomfortable?
How does the math work for Perception checks?
15% tax on $7.5k earnings. Is that right?
Has any country ever had 2 former presidents in jail simultaneously?
Hero deduces identity of a killer
Fear of getting stuck on one programming language / technology that is not used in my country
Why is the "ls" command showing permissions of files in a FAT32 partition?
What is Cash Advance APR?
Mixing PEX brands
Why does the Sun have different day lengths, but not the gas giants?
Store Credit Card Information in Password Manager?
putting logo on same line but after title, latex
Why is so much work done on numerical verification of the Riemann Hypothesis?
Multiplicative persistence
Using substitution ciphers to generate new alphabets in a novel
Does the Linux kernel need a file system to run?
Yosemite Fire Rings - What to Expect?
Electoral considerations aside, what are potential benefits, for the US, of policy changes proposed by the tweet recognizing Golan annexation?
Why is this estimator biased?
Does IPv6 have similar concept of network mask?
Pass shell variable as a /pattern/ to awk
2019 Community Moderator ElectionHow to pass shell variable to an awk search pattern?How to use a variable inside an awk statement?make awk print the line that match a variable and the next n lines and use a variable in awkAWK and Bash Scriptingusing awk with variablesPass Shell variable to awkawk search for string containing forward slash in variablePass a variable in another variable within an awkhow to use variable with awkHow to make variable be taken into accountGawk: Passing arrays to functionsHow can I convert this bash function to the fish shellBEGIN and END with the awk commandBest way to iterate through the following awk commandHow to pass shell variable to an awk search pattern?(Shell Script) Variable not pass correctly when fetching value from file?Use awk interactively through a pipePass Shell variable to awkusing awk with variablesDeclaring a variable that does not have input
Having the following in one of my shell functions:
function _process ()
awk -v l="$line" '
BEGIN p=0
/'"$1"'/ p=1
END if(p) print l >> "outfile.txt"
'
, so when called as _process $arg
, $arg
gets passed as $1
, and used as a search pattern. It works this way, because shell expands $1
in place of awk pattern! Also l
can be used inside awk program, being declared with -v l="$line"
. All fine.
Is it possible in same manner give pattern to search as a variable?
Following will not work,
awk -v l="$line" -v search="$pattern" '
BEGIN p=0
/search/ p=1
END if(p) print l >> "outfile.txt"
'
,as awk will not interpret /search/
as a variable, but instead literally.
shell awk quoting variable
add a comment |
Having the following in one of my shell functions:
function _process ()
awk -v l="$line" '
BEGIN p=0
/'"$1"'/ p=1
END if(p) print l >> "outfile.txt"
'
, so when called as _process $arg
, $arg
gets passed as $1
, and used as a search pattern. It works this way, because shell expands $1
in place of awk pattern! Also l
can be used inside awk program, being declared with -v l="$line"
. All fine.
Is it possible in same manner give pattern to search as a variable?
Following will not work,
awk -v l="$line" -v search="$pattern" '
BEGIN p=0
/search/ p=1
END if(p) print l >> "outfile.txt"
'
,as awk will not interpret /search/
as a variable, but instead literally.
shell awk quoting variable
add a comment |
Having the following in one of my shell functions:
function _process ()
awk -v l="$line" '
BEGIN p=0
/'"$1"'/ p=1
END if(p) print l >> "outfile.txt"
'
, so when called as _process $arg
, $arg
gets passed as $1
, and used as a search pattern. It works this way, because shell expands $1
in place of awk pattern! Also l
can be used inside awk program, being declared with -v l="$line"
. All fine.
Is it possible in same manner give pattern to search as a variable?
Following will not work,
awk -v l="$line" -v search="$pattern" '
BEGIN p=0
/search/ p=1
END if(p) print l >> "outfile.txt"
'
,as awk will not interpret /search/
as a variable, but instead literally.
shell awk quoting variable
Having the following in one of my shell functions:
function _process ()
awk -v l="$line" '
BEGIN p=0
/'"$1"'/ p=1
END if(p) print l >> "outfile.txt"
'
, so when called as _process $arg
, $arg
gets passed as $1
, and used as a search pattern. It works this way, because shell expands $1
in place of awk pattern! Also l
can be used inside awk program, being declared with -v l="$line"
. All fine.
Is it possible in same manner give pattern to search as a variable?
Following will not work,
awk -v l="$line" -v search="$pattern" '
BEGIN p=0
/search/ p=1
END if(p) print l >> "outfile.txt"
'
,as awk will not interpret /search/
as a variable, but instead literally.
shell awk quoting variable
shell awk quoting variable
edited Mar 21 '14 at 23:04
Gilles
543k12811011618
543k12811011618
asked Mar 21 '14 at 15:03
branquitobranquito
4451616
4451616
add a comment |
add a comment |
5 Answers
5
active
oldest
votes
Use awk's ~
operator, and you don't need to provide a literal regex on the right-hand side:
function _process ()
awk -v l="$line" -v pattern="$1" '
$0 ~ pattern p=1
END if(p) print l >> "outfile.txt"
'
Although this would be more efficient (don't have to read the whole file)
function _process ()
grep -q "$1" && echo "$line"
Depending on the pattern, may want grep -Eq "$1"
This is exactly what solves this in a way I wanted (1st example), because it keeps the semantics, which was my goal. Thanks.
– branquito
Mar 21 '14 at 15:30
1
I didn't note the removal of the BEGIN block: an unassigned variable is treated as 0 in a numeric context or the empty string otherwise. So, an unassigned variable will be false inif (p) ...
– glenn jackman
Mar 21 '14 at 15:35
yes I noticed, it needs to be set on BEGIN block to zero each time, as it serves as a switch. But interestingly I tried now script using$0 ~ pattern
, and it does not work, however with/'"$1"'/
it does work!? :O
– branquito
Mar 21 '14 at 15:42
maybe it has something to do with the way$line
is retrieved, pattern search is done on the output ofwhois $line
,$line
coming from file in a WHILE DO block.
– branquito
Mar 21 '14 at 15:53
Please show the contents of$line
-- do it in your question for proper formatting.
– glenn jackman
Mar 21 '14 at 15:57
|
show 3 more comments
awk -v pattern="$1" '$0 ~ pattern'
Has an issue in that awk
expands the ANSI C escape sequences (like n
for newline, f
for form feed, \
for backslash and so on) in $1
. So it becomes an issue if $1
contains backslash characters which is common in regular expressions. Another approach that doesn't suffer from that issue is to write it:
PATTERN=$1 awk '$0 ~ ENVIRON["PATTERN"]'
How bad it's going to be will depend on the awk
implementation.
$ nawk -v 'a=.' 'BEGIN print a'
.
$ mawk -v 'a=.' 'BEGIN print a'
.
$ gawk -v 'a=.' 'BEGIN print a'
gawk: warning: escape sequence `.' treated as plain `.'
.
All awk
s work the same for valid escape sequences though:
$ a='\-b' awk 'BEGIN print ENVIRON["a"]' | od -tc
0000000 - b n
0000006
(content of $a
passed as-is)
$ awk -v a='\-b' 'BEGIN print a' | od -tc
0000000 - b n
0000004
(\
changed to and
b
changed to a backspace character).
So you are saying that if pattern was for exampled3
to find three digits, that wouldn't work as expected, if I understood you well?
– branquito
Mar 21 '14 at 16:24
2
ford
which is not a valid C escape sequence, that depends on yourawk
implementation (runawk -v 'a=d3' 'BEGINprint a'
to check). But for` or
b, yes definitely. (BTW, I don't know of any awk implementations that understands
d` as meaning a digit).
– Stéphane Chazelas
Mar 21 '14 at 16:30
it says: awk warning - escape sequenced' treated as plain
d' d3, so I guess I would have a problem in this case?
– branquito
Mar 21 '14 at 16:34
1
Sorry, my bad, I had a typo in my answer. The name of then environment variable has to matchENVIRON["PATTERN"]
for thePATTERN
environment variable. If you want to use a shell variable, you need to export it first (export variable
) or use theENV=VALUE awk '...ENVIRON["ENV"]'
env-var passing syntax as in my answer.
– Stéphane Chazelas
Mar 21 '14 at 17:03
1
Because you need to export a shell variable for it to be passed in the environment to a command.
– Stéphane Chazelas
Mar 21 '14 at 17:20
|
show 5 more comments
Try something like:
awk -v l="$line" -v search="$pattern" 'BEGIN p=0; if ( match( $0, search )) p=1; END if(p) print l >> "outfile.txt" '
If this behaves same as/regex/
in terms of finding pattern, this could be a nice solution. I will try.
– branquito
Mar 21 '14 at 15:22
1
The quick tests I ran seemed to work the same, but I won't even begin to guarantee it... :)
– Hunter Eidson
Mar 21 '14 at 15:24
add a comment |
No, but you can simply interpolate the pattern into the double-quoted string you pass to awk:
awk -v l="$line" "BEGIN p=0; /$pattern/ p=1; END if(p) print l >> "outfile.txt" "
Note that you now have to escape the double-quoted awk literal, but it is still the simplest way of accomplishing this.
Is this way safe if$pattern
contains spaces, my example from above will work as $1 is protected with "$1" double quotes, however not shure what happens in your case.
– branquito
Mar 21 '14 at 15:14
2
Your original example ends the single-quoted string at the second'
, then protects the$1
via double quotes and then tacks another single-quoted string for the second half of the awk program. If I understand correctly, this should have exactly the same effect as protecting the$1
via the outer single quotes - awk never sees the double quotes that you put around it.
– Kilian Foth
Mar 21 '14 at 15:26
4
But if$pattern
contains^/ system("rm -rf /");
, then you're in big trouble.
– Stéphane Chazelas
Mar 21 '14 at 16:17
is that downside of this approach only, having all wrapped in "" ?
– branquito
Mar 21 '14 at 16:27
add a comment |
You could use the eval function which resolves in this example the nets variable before the awk is run.
nets="searchtext"
eval "awk '/"$nets"/'" file.txt
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f120788%2fpass-shell-variable-as-a-pattern-to-awk%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
5 Answers
5
active
oldest
votes
5 Answers
5
active
oldest
votes
active
oldest
votes
active
oldest
votes
Use awk's ~
operator, and you don't need to provide a literal regex on the right-hand side:
function _process ()
awk -v l="$line" -v pattern="$1" '
$0 ~ pattern p=1
END if(p) print l >> "outfile.txt"
'
Although this would be more efficient (don't have to read the whole file)
function _process ()
grep -q "$1" && echo "$line"
Depending on the pattern, may want grep -Eq "$1"
This is exactly what solves this in a way I wanted (1st example), because it keeps the semantics, which was my goal. Thanks.
– branquito
Mar 21 '14 at 15:30
1
I didn't note the removal of the BEGIN block: an unassigned variable is treated as 0 in a numeric context or the empty string otherwise. So, an unassigned variable will be false inif (p) ...
– glenn jackman
Mar 21 '14 at 15:35
yes I noticed, it needs to be set on BEGIN block to zero each time, as it serves as a switch. But interestingly I tried now script using$0 ~ pattern
, and it does not work, however with/'"$1"'/
it does work!? :O
– branquito
Mar 21 '14 at 15:42
maybe it has something to do with the way$line
is retrieved, pattern search is done on the output ofwhois $line
,$line
coming from file in a WHILE DO block.
– branquito
Mar 21 '14 at 15:53
Please show the contents of$line
-- do it in your question for proper formatting.
– glenn jackman
Mar 21 '14 at 15:57
|
show 3 more comments
Use awk's ~
operator, and you don't need to provide a literal regex on the right-hand side:
function _process ()
awk -v l="$line" -v pattern="$1" '
$0 ~ pattern p=1
END if(p) print l >> "outfile.txt"
'
Although this would be more efficient (don't have to read the whole file)
function _process ()
grep -q "$1" && echo "$line"
Depending on the pattern, may want grep -Eq "$1"
This is exactly what solves this in a way I wanted (1st example), because it keeps the semantics, which was my goal. Thanks.
– branquito
Mar 21 '14 at 15:30
1
I didn't note the removal of the BEGIN block: an unassigned variable is treated as 0 in a numeric context or the empty string otherwise. So, an unassigned variable will be false inif (p) ...
– glenn jackman
Mar 21 '14 at 15:35
yes I noticed, it needs to be set on BEGIN block to zero each time, as it serves as a switch. But interestingly I tried now script using$0 ~ pattern
, and it does not work, however with/'"$1"'/
it does work!? :O
– branquito
Mar 21 '14 at 15:42
maybe it has something to do with the way$line
is retrieved, pattern search is done on the output ofwhois $line
,$line
coming from file in a WHILE DO block.
– branquito
Mar 21 '14 at 15:53
Please show the contents of$line
-- do it in your question for proper formatting.
– glenn jackman
Mar 21 '14 at 15:57
|
show 3 more comments
Use awk's ~
operator, and you don't need to provide a literal regex on the right-hand side:
function _process ()
awk -v l="$line" -v pattern="$1" '
$0 ~ pattern p=1
END if(p) print l >> "outfile.txt"
'
Although this would be more efficient (don't have to read the whole file)
function _process ()
grep -q "$1" && echo "$line"
Depending on the pattern, may want grep -Eq "$1"
Use awk's ~
operator, and you don't need to provide a literal regex on the right-hand side:
function _process ()
awk -v l="$line" -v pattern="$1" '
$0 ~ pattern p=1
END if(p) print l >> "outfile.txt"
'
Although this would be more efficient (don't have to read the whole file)
function _process ()
grep -q "$1" && echo "$line"
Depending on the pattern, may want grep -Eq "$1"
answered Mar 21 '14 at 15:16
glenn jackmanglenn jackman
52.6k573114
52.6k573114
This is exactly what solves this in a way I wanted (1st example), because it keeps the semantics, which was my goal. Thanks.
– branquito
Mar 21 '14 at 15:30
1
I didn't note the removal of the BEGIN block: an unassigned variable is treated as 0 in a numeric context or the empty string otherwise. So, an unassigned variable will be false inif (p) ...
– glenn jackman
Mar 21 '14 at 15:35
yes I noticed, it needs to be set on BEGIN block to zero each time, as it serves as a switch. But interestingly I tried now script using$0 ~ pattern
, and it does not work, however with/'"$1"'/
it does work!? :O
– branquito
Mar 21 '14 at 15:42
maybe it has something to do with the way$line
is retrieved, pattern search is done on the output ofwhois $line
,$line
coming from file in a WHILE DO block.
– branquito
Mar 21 '14 at 15:53
Please show the contents of$line
-- do it in your question for proper formatting.
– glenn jackman
Mar 21 '14 at 15:57
|
show 3 more comments
This is exactly what solves this in a way I wanted (1st example), because it keeps the semantics, which was my goal. Thanks.
– branquito
Mar 21 '14 at 15:30
1
I didn't note the removal of the BEGIN block: an unassigned variable is treated as 0 in a numeric context or the empty string otherwise. So, an unassigned variable will be false inif (p) ...
– glenn jackman
Mar 21 '14 at 15:35
yes I noticed, it needs to be set on BEGIN block to zero each time, as it serves as a switch. But interestingly I tried now script using$0 ~ pattern
, and it does not work, however with/'"$1"'/
it does work!? :O
– branquito
Mar 21 '14 at 15:42
maybe it has something to do with the way$line
is retrieved, pattern search is done on the output ofwhois $line
,$line
coming from file in a WHILE DO block.
– branquito
Mar 21 '14 at 15:53
Please show the contents of$line
-- do it in your question for proper formatting.
– glenn jackman
Mar 21 '14 at 15:57
This is exactly what solves this in a way I wanted (1st example), because it keeps the semantics, which was my goal. Thanks.
– branquito
Mar 21 '14 at 15:30
This is exactly what solves this in a way I wanted (1st example), because it keeps the semantics, which was my goal. Thanks.
– branquito
Mar 21 '14 at 15:30
1
1
I didn't note the removal of the BEGIN block: an unassigned variable is treated as 0 in a numeric context or the empty string otherwise. So, an unassigned variable will be false in
if (p) ...
– glenn jackman
Mar 21 '14 at 15:35
I didn't note the removal of the BEGIN block: an unassigned variable is treated as 0 in a numeric context or the empty string otherwise. So, an unassigned variable will be false in
if (p) ...
– glenn jackman
Mar 21 '14 at 15:35
yes I noticed, it needs to be set on BEGIN block to zero each time, as it serves as a switch. But interestingly I tried now script using
$0 ~ pattern
, and it does not work, however with /'"$1"'/
it does work!? :O– branquito
Mar 21 '14 at 15:42
yes I noticed, it needs to be set on BEGIN block to zero each time, as it serves as a switch. But interestingly I tried now script using
$0 ~ pattern
, and it does not work, however with /'"$1"'/
it does work!? :O– branquito
Mar 21 '14 at 15:42
maybe it has something to do with the way
$line
is retrieved, pattern search is done on the output of whois $line
, $line
coming from file in a WHILE DO block.– branquito
Mar 21 '14 at 15:53
maybe it has something to do with the way
$line
is retrieved, pattern search is done on the output of whois $line
, $line
coming from file in a WHILE DO block.– branquito
Mar 21 '14 at 15:53
Please show the contents of
$line
-- do it in your question for proper formatting.– glenn jackman
Mar 21 '14 at 15:57
Please show the contents of
$line
-- do it in your question for proper formatting.– glenn jackman
Mar 21 '14 at 15:57
|
show 3 more comments
awk -v pattern="$1" '$0 ~ pattern'
Has an issue in that awk
expands the ANSI C escape sequences (like n
for newline, f
for form feed, \
for backslash and so on) in $1
. So it becomes an issue if $1
contains backslash characters which is common in regular expressions. Another approach that doesn't suffer from that issue is to write it:
PATTERN=$1 awk '$0 ~ ENVIRON["PATTERN"]'
How bad it's going to be will depend on the awk
implementation.
$ nawk -v 'a=.' 'BEGIN print a'
.
$ mawk -v 'a=.' 'BEGIN print a'
.
$ gawk -v 'a=.' 'BEGIN print a'
gawk: warning: escape sequence `.' treated as plain `.'
.
All awk
s work the same for valid escape sequences though:
$ a='\-b' awk 'BEGIN print ENVIRON["a"]' | od -tc
0000000 - b n
0000006
(content of $a
passed as-is)
$ awk -v a='\-b' 'BEGIN print a' | od -tc
0000000 - b n
0000004
(\
changed to and
b
changed to a backspace character).
So you are saying that if pattern was for exampled3
to find three digits, that wouldn't work as expected, if I understood you well?
– branquito
Mar 21 '14 at 16:24
2
ford
which is not a valid C escape sequence, that depends on yourawk
implementation (runawk -v 'a=d3' 'BEGINprint a'
to check). But for` or
b, yes definitely. (BTW, I don't know of any awk implementations that understands
d` as meaning a digit).
– Stéphane Chazelas
Mar 21 '14 at 16:30
it says: awk warning - escape sequenced' treated as plain
d' d3, so I guess I would have a problem in this case?
– branquito
Mar 21 '14 at 16:34
1
Sorry, my bad, I had a typo in my answer. The name of then environment variable has to matchENVIRON["PATTERN"]
for thePATTERN
environment variable. If you want to use a shell variable, you need to export it first (export variable
) or use theENV=VALUE awk '...ENVIRON["ENV"]'
env-var passing syntax as in my answer.
– Stéphane Chazelas
Mar 21 '14 at 17:03
1
Because you need to export a shell variable for it to be passed in the environment to a command.
– Stéphane Chazelas
Mar 21 '14 at 17:20
|
show 5 more comments
awk -v pattern="$1" '$0 ~ pattern'
Has an issue in that awk
expands the ANSI C escape sequences (like n
for newline, f
for form feed, \
for backslash and so on) in $1
. So it becomes an issue if $1
contains backslash characters which is common in regular expressions. Another approach that doesn't suffer from that issue is to write it:
PATTERN=$1 awk '$0 ~ ENVIRON["PATTERN"]'
How bad it's going to be will depend on the awk
implementation.
$ nawk -v 'a=.' 'BEGIN print a'
.
$ mawk -v 'a=.' 'BEGIN print a'
.
$ gawk -v 'a=.' 'BEGIN print a'
gawk: warning: escape sequence `.' treated as plain `.'
.
All awk
s work the same for valid escape sequences though:
$ a='\-b' awk 'BEGIN print ENVIRON["a"]' | od -tc
0000000 - b n
0000006
(content of $a
passed as-is)
$ awk -v a='\-b' 'BEGIN print a' | od -tc
0000000 - b n
0000004
(\
changed to and
b
changed to a backspace character).
So you are saying that if pattern was for exampled3
to find three digits, that wouldn't work as expected, if I understood you well?
– branquito
Mar 21 '14 at 16:24
2
ford
which is not a valid C escape sequence, that depends on yourawk
implementation (runawk -v 'a=d3' 'BEGINprint a'
to check). But for` or
b, yes definitely. (BTW, I don't know of any awk implementations that understands
d` as meaning a digit).
– Stéphane Chazelas
Mar 21 '14 at 16:30
it says: awk warning - escape sequenced' treated as plain
d' d3, so I guess I would have a problem in this case?
– branquito
Mar 21 '14 at 16:34
1
Sorry, my bad, I had a typo in my answer. The name of then environment variable has to matchENVIRON["PATTERN"]
for thePATTERN
environment variable. If you want to use a shell variable, you need to export it first (export variable
) or use theENV=VALUE awk '...ENVIRON["ENV"]'
env-var passing syntax as in my answer.
– Stéphane Chazelas
Mar 21 '14 at 17:03
1
Because you need to export a shell variable for it to be passed in the environment to a command.
– Stéphane Chazelas
Mar 21 '14 at 17:20
|
show 5 more comments
awk -v pattern="$1" '$0 ~ pattern'
Has an issue in that awk
expands the ANSI C escape sequences (like n
for newline, f
for form feed, \
for backslash and so on) in $1
. So it becomes an issue if $1
contains backslash characters which is common in regular expressions. Another approach that doesn't suffer from that issue is to write it:
PATTERN=$1 awk '$0 ~ ENVIRON["PATTERN"]'
How bad it's going to be will depend on the awk
implementation.
$ nawk -v 'a=.' 'BEGIN print a'
.
$ mawk -v 'a=.' 'BEGIN print a'
.
$ gawk -v 'a=.' 'BEGIN print a'
gawk: warning: escape sequence `.' treated as plain `.'
.
All awk
s work the same for valid escape sequences though:
$ a='\-b' awk 'BEGIN print ENVIRON["a"]' | od -tc
0000000 - b n
0000006
(content of $a
passed as-is)
$ awk -v a='\-b' 'BEGIN print a' | od -tc
0000000 - b n
0000004
(\
changed to and
b
changed to a backspace character).
awk -v pattern="$1" '$0 ~ pattern'
Has an issue in that awk
expands the ANSI C escape sequences (like n
for newline, f
for form feed, \
for backslash and so on) in $1
. So it becomes an issue if $1
contains backslash characters which is common in regular expressions. Another approach that doesn't suffer from that issue is to write it:
PATTERN=$1 awk '$0 ~ ENVIRON["PATTERN"]'
How bad it's going to be will depend on the awk
implementation.
$ nawk -v 'a=.' 'BEGIN print a'
.
$ mawk -v 'a=.' 'BEGIN print a'
.
$ gawk -v 'a=.' 'BEGIN print a'
gawk: warning: escape sequence `.' treated as plain `.'
.
All awk
s work the same for valid escape sequences though:
$ a='\-b' awk 'BEGIN print ENVIRON["a"]' | od -tc
0000000 - b n
0000006
(content of $a
passed as-is)
$ awk -v a='\-b' 'BEGIN print a' | od -tc
0000000 - b n
0000004
(\
changed to and
b
changed to a backspace character).
edited yesterday
answered Mar 21 '14 at 16:16
Stéphane ChazelasStéphane Chazelas
311k57586945
311k57586945
So you are saying that if pattern was for exampled3
to find three digits, that wouldn't work as expected, if I understood you well?
– branquito
Mar 21 '14 at 16:24
2
ford
which is not a valid C escape sequence, that depends on yourawk
implementation (runawk -v 'a=d3' 'BEGINprint a'
to check). But for` or
b, yes definitely. (BTW, I don't know of any awk implementations that understands
d` as meaning a digit).
– Stéphane Chazelas
Mar 21 '14 at 16:30
it says: awk warning - escape sequenced' treated as plain
d' d3, so I guess I would have a problem in this case?
– branquito
Mar 21 '14 at 16:34
1
Sorry, my bad, I had a typo in my answer. The name of then environment variable has to matchENVIRON["PATTERN"]
for thePATTERN
environment variable. If you want to use a shell variable, you need to export it first (export variable
) or use theENV=VALUE awk '...ENVIRON["ENV"]'
env-var passing syntax as in my answer.
– Stéphane Chazelas
Mar 21 '14 at 17:03
1
Because you need to export a shell variable for it to be passed in the environment to a command.
– Stéphane Chazelas
Mar 21 '14 at 17:20
|
show 5 more comments
So you are saying that if pattern was for exampled3
to find three digits, that wouldn't work as expected, if I understood you well?
– branquito
Mar 21 '14 at 16:24
2
ford
which is not a valid C escape sequence, that depends on yourawk
implementation (runawk -v 'a=d3' 'BEGINprint a'
to check). But for` or
b, yes definitely. (BTW, I don't know of any awk implementations that understands
d` as meaning a digit).
– Stéphane Chazelas
Mar 21 '14 at 16:30
it says: awk warning - escape sequenced' treated as plain
d' d3, so I guess I would have a problem in this case?
– branquito
Mar 21 '14 at 16:34
1
Sorry, my bad, I had a typo in my answer. The name of then environment variable has to matchENVIRON["PATTERN"]
for thePATTERN
environment variable. If you want to use a shell variable, you need to export it first (export variable
) or use theENV=VALUE awk '...ENVIRON["ENV"]'
env-var passing syntax as in my answer.
– Stéphane Chazelas
Mar 21 '14 at 17:03
1
Because you need to export a shell variable for it to be passed in the environment to a command.
– Stéphane Chazelas
Mar 21 '14 at 17:20
So you are saying that if pattern was for example
d3
to find three digits, that wouldn't work as expected, if I understood you well?– branquito
Mar 21 '14 at 16:24
So you are saying that if pattern was for example
d3
to find three digits, that wouldn't work as expected, if I understood you well?– branquito
Mar 21 '14 at 16:24
2
2
for
d
which is not a valid C escape sequence, that depends on your awk
implementation (run awk -v 'a=d3' 'BEGINprint a'
to check). But for ` or
b, yes definitely. (BTW, I don't know of any awk implementations that understands
d` as meaning a digit).– Stéphane Chazelas
Mar 21 '14 at 16:30
for
d
which is not a valid C escape sequence, that depends on your awk
implementation (run awk -v 'a=d3' 'BEGINprint a'
to check). But for ` or
b, yes definitely. (BTW, I don't know of any awk implementations that understands
d` as meaning a digit).– Stéphane Chazelas
Mar 21 '14 at 16:30
it says: awk warning - escape sequence
d' treated as plain
d' d3, so I guess I would have a problem in this case?– branquito
Mar 21 '14 at 16:34
it says: awk warning - escape sequence
d' treated as plain
d' d3, so I guess I would have a problem in this case?– branquito
Mar 21 '14 at 16:34
1
1
Sorry, my bad, I had a typo in my answer. The name of then environment variable has to match
ENVIRON["PATTERN"]
for the PATTERN
environment variable. If you want to use a shell variable, you need to export it first (export variable
) or use the ENV=VALUE awk '...ENVIRON["ENV"]'
env-var passing syntax as in my answer.– Stéphane Chazelas
Mar 21 '14 at 17:03
Sorry, my bad, I had a typo in my answer. The name of then environment variable has to match
ENVIRON["PATTERN"]
for the PATTERN
environment variable. If you want to use a shell variable, you need to export it first (export variable
) or use the ENV=VALUE awk '...ENVIRON["ENV"]'
env-var passing syntax as in my answer.– Stéphane Chazelas
Mar 21 '14 at 17:03
1
1
Because you need to export a shell variable for it to be passed in the environment to a command.
– Stéphane Chazelas
Mar 21 '14 at 17:20
Because you need to export a shell variable for it to be passed in the environment to a command.
– Stéphane Chazelas
Mar 21 '14 at 17:20
|
show 5 more comments
Try something like:
awk -v l="$line" -v search="$pattern" 'BEGIN p=0; if ( match( $0, search )) p=1; END if(p) print l >> "outfile.txt" '
If this behaves same as/regex/
in terms of finding pattern, this could be a nice solution. I will try.
– branquito
Mar 21 '14 at 15:22
1
The quick tests I ran seemed to work the same, but I won't even begin to guarantee it... :)
– Hunter Eidson
Mar 21 '14 at 15:24
add a comment |
Try something like:
awk -v l="$line" -v search="$pattern" 'BEGIN p=0; if ( match( $0, search )) p=1; END if(p) print l >> "outfile.txt" '
If this behaves same as/regex/
in terms of finding pattern, this could be a nice solution. I will try.
– branquito
Mar 21 '14 at 15:22
1
The quick tests I ran seemed to work the same, but I won't even begin to guarantee it... :)
– Hunter Eidson
Mar 21 '14 at 15:24
add a comment |
Try something like:
awk -v l="$line" -v search="$pattern" 'BEGIN p=0; if ( match( $0, search )) p=1; END if(p) print l >> "outfile.txt" '
Try something like:
awk -v l="$line" -v search="$pattern" 'BEGIN p=0; if ( match( $0, search )) p=1; END if(p) print l >> "outfile.txt" '
answered Mar 21 '14 at 15:15
Hunter EidsonHunter Eidson
17116
17116
If this behaves same as/regex/
in terms of finding pattern, this could be a nice solution. I will try.
– branquito
Mar 21 '14 at 15:22
1
The quick tests I ran seemed to work the same, but I won't even begin to guarantee it... :)
– Hunter Eidson
Mar 21 '14 at 15:24
add a comment |
If this behaves same as/regex/
in terms of finding pattern, this could be a nice solution. I will try.
– branquito
Mar 21 '14 at 15:22
1
The quick tests I ran seemed to work the same, but I won't even begin to guarantee it... :)
– Hunter Eidson
Mar 21 '14 at 15:24
If this behaves same as
/regex/
in terms of finding pattern, this could be a nice solution. I will try.– branquito
Mar 21 '14 at 15:22
If this behaves same as
/regex/
in terms of finding pattern, this could be a nice solution. I will try.– branquito
Mar 21 '14 at 15:22
1
1
The quick tests I ran seemed to work the same, but I won't even begin to guarantee it... :)
– Hunter Eidson
Mar 21 '14 at 15:24
The quick tests I ran seemed to work the same, but I won't even begin to guarantee it... :)
– Hunter Eidson
Mar 21 '14 at 15:24
add a comment |
No, but you can simply interpolate the pattern into the double-quoted string you pass to awk:
awk -v l="$line" "BEGIN p=0; /$pattern/ p=1; END if(p) print l >> "outfile.txt" "
Note that you now have to escape the double-quoted awk literal, but it is still the simplest way of accomplishing this.
Is this way safe if$pattern
contains spaces, my example from above will work as $1 is protected with "$1" double quotes, however not shure what happens in your case.
– branquito
Mar 21 '14 at 15:14
2
Your original example ends the single-quoted string at the second'
, then protects the$1
via double quotes and then tacks another single-quoted string for the second half of the awk program. If I understand correctly, this should have exactly the same effect as protecting the$1
via the outer single quotes - awk never sees the double quotes that you put around it.
– Kilian Foth
Mar 21 '14 at 15:26
4
But if$pattern
contains^/ system("rm -rf /");
, then you're in big trouble.
– Stéphane Chazelas
Mar 21 '14 at 16:17
is that downside of this approach only, having all wrapped in "" ?
– branquito
Mar 21 '14 at 16:27
add a comment |
No, but you can simply interpolate the pattern into the double-quoted string you pass to awk:
awk -v l="$line" "BEGIN p=0; /$pattern/ p=1; END if(p) print l >> "outfile.txt" "
Note that you now have to escape the double-quoted awk literal, but it is still the simplest way of accomplishing this.
Is this way safe if$pattern
contains spaces, my example from above will work as $1 is protected with "$1" double quotes, however not shure what happens in your case.
– branquito
Mar 21 '14 at 15:14
2
Your original example ends the single-quoted string at the second'
, then protects the$1
via double quotes and then tacks another single-quoted string for the second half of the awk program. If I understand correctly, this should have exactly the same effect as protecting the$1
via the outer single quotes - awk never sees the double quotes that you put around it.
– Kilian Foth
Mar 21 '14 at 15:26
4
But if$pattern
contains^/ system("rm -rf /");
, then you're in big trouble.
– Stéphane Chazelas
Mar 21 '14 at 16:17
is that downside of this approach only, having all wrapped in "" ?
– branquito
Mar 21 '14 at 16:27
add a comment |
No, but you can simply interpolate the pattern into the double-quoted string you pass to awk:
awk -v l="$line" "BEGIN p=0; /$pattern/ p=1; END if(p) print l >> "outfile.txt" "
Note that you now have to escape the double-quoted awk literal, but it is still the simplest way of accomplishing this.
No, but you can simply interpolate the pattern into the double-quoted string you pass to awk:
awk -v l="$line" "BEGIN p=0; /$pattern/ p=1; END if(p) print l >> "outfile.txt" "
Note that you now have to escape the double-quoted awk literal, but it is still the simplest way of accomplishing this.
answered Mar 21 '14 at 15:11
Kilian FothKilian Foth
680315
680315
Is this way safe if$pattern
contains spaces, my example from above will work as $1 is protected with "$1" double quotes, however not shure what happens in your case.
– branquito
Mar 21 '14 at 15:14
2
Your original example ends the single-quoted string at the second'
, then protects the$1
via double quotes and then tacks another single-quoted string for the second half of the awk program. If I understand correctly, this should have exactly the same effect as protecting the$1
via the outer single quotes - awk never sees the double quotes that you put around it.
– Kilian Foth
Mar 21 '14 at 15:26
4
But if$pattern
contains^/ system("rm -rf /");
, then you're in big trouble.
– Stéphane Chazelas
Mar 21 '14 at 16:17
is that downside of this approach only, having all wrapped in "" ?
– branquito
Mar 21 '14 at 16:27
add a comment |
Is this way safe if$pattern
contains spaces, my example from above will work as $1 is protected with "$1" double quotes, however not shure what happens in your case.
– branquito
Mar 21 '14 at 15:14
2
Your original example ends the single-quoted string at the second'
, then protects the$1
via double quotes and then tacks another single-quoted string for the second half of the awk program. If I understand correctly, this should have exactly the same effect as protecting the$1
via the outer single quotes - awk never sees the double quotes that you put around it.
– Kilian Foth
Mar 21 '14 at 15:26
4
But if$pattern
contains^/ system("rm -rf /");
, then you're in big trouble.
– Stéphane Chazelas
Mar 21 '14 at 16:17
is that downside of this approach only, having all wrapped in "" ?
– branquito
Mar 21 '14 at 16:27
Is this way safe if
$pattern
contains spaces, my example from above will work as $1 is protected with "$1" double quotes, however not shure what happens in your case.– branquito
Mar 21 '14 at 15:14
Is this way safe if
$pattern
contains spaces, my example from above will work as $1 is protected with "$1" double quotes, however not shure what happens in your case.– branquito
Mar 21 '14 at 15:14
2
2
Your original example ends the single-quoted string at the second
'
, then protects the $1
via double quotes and then tacks another single-quoted string for the second half of the awk program. If I understand correctly, this should have exactly the same effect as protecting the $1
via the outer single quotes - awk never sees the double quotes that you put around it.– Kilian Foth
Mar 21 '14 at 15:26
Your original example ends the single-quoted string at the second
'
, then protects the $1
via double quotes and then tacks another single-quoted string for the second half of the awk program. If I understand correctly, this should have exactly the same effect as protecting the $1
via the outer single quotes - awk never sees the double quotes that you put around it.– Kilian Foth
Mar 21 '14 at 15:26
4
4
But if
$pattern
contains ^/ system("rm -rf /");
, then you're in big trouble.– Stéphane Chazelas
Mar 21 '14 at 16:17
But if
$pattern
contains ^/ system("rm -rf /");
, then you're in big trouble.– Stéphane Chazelas
Mar 21 '14 at 16:17
is that downside of this approach only, having all wrapped in "" ?
– branquito
Mar 21 '14 at 16:27
is that downside of this approach only, having all wrapped in "" ?
– branquito
Mar 21 '14 at 16:27
add a comment |
You could use the eval function which resolves in this example the nets variable before the awk is run.
nets="searchtext"
eval "awk '/"$nets"/'" file.txt
add a comment |
You could use the eval function which resolves in this example the nets variable before the awk is run.
nets="searchtext"
eval "awk '/"$nets"/'" file.txt
add a comment |
You could use the eval function which resolves in this example the nets variable before the awk is run.
nets="searchtext"
eval "awk '/"$nets"/'" file.txt
You could use the eval function which resolves in this example the nets variable before the awk is run.
nets="searchtext"
eval "awk '/"$nets"/'" file.txt
edited Feb 21 '17 at 11:51
Noxy
32
32
answered Feb 21 '17 at 11:19
NoxyNoxy
1
1
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f120788%2fpass-shell-variable-as-a-pattern-to-awk%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown