Ygtjki

I came across a code in script which add ',' after every 3 digits from backwards. the code only considers numeric data.

Following is the code.

sed 's/(^|[^0-9.])([0-9]+)([0-9]3)/12,3/g' number.txt

number.txt

1234
12345
123456

output

1,234
12,345
123,456
1234,567

Can anyone explain the code flow..

sed (the stream editor) can operate in search and replace mode using regular expressions. There is a bit of sed-specific escaping going on, but for the regex itself, you can confer the explanation tool from regexr:

( Capturing group #1. Groups multiple tokens
together and creates a capture group for extracting a substring or using
a backreference.

^ Beginning. Matches the beginning of the
string.

| Alternation. Acts like a boolean OR. Matches
the expression before or after the |.

[^ Negated set. Match any character that is not
in the set.

0-9 Range. Matches a character in the range "0"
to "9". Case sensitive.

. Character. Matches a "." character.

]

)

( Capturing group #2. Groups multiple tokens
together and creates a capture group for extracting a substring or using
a backreference.

[ Character set. Match any character in the set.

0-9 Range. Matches a character in the range "0"
to "9". Case sensitive.

]

+ Quantifier. Match 1 or more of the preceding
token.

)

( Capturing group #3. Groups multiple tokens
together and creates a capture group for extracting a substring or using
a backreference.

[ Character set. Match any character in the set.

0-9 Range. Matches a character in the range "0"
to "9". Case sensitive.

]

3 Quantifier. Match 3 of the preceding token.

)

The pattern captures (1) the start of line or something that is not a digit nor a dot, followed by (2) any number of digits, at least one, followed by (3) exactly three digits.
It then puts them back with a comma between (2) and (3), in effect adding one thousands separator. The first group is only required to avoid touching fractional parts after a decimal point, since we don't want 1.2345 to turn into 1.2,345.

Note that the pattern is written in basic regular expressions (BRE), requiring backslashes in front of each () and to make them special. Moreover, it requires GNU sed, where + and | also have special meanings in BRE as an extension. The command would be better written as an extended regular expression (sed -E is supported by many sed implementations):

sed -E 's/(^|[^0-9.])([0-9]+)([0-9]3)/12,3/g'

Also, the pattern only does one replacement, it does not add multiple thousand separators to the same number. The /g at the end will match multiple times on the same line, but it doesn't process already replaced data. 1234567 will become 1234,567, not 1,234,567. To fix that, we'll need to add a loop:

sed -E -e :a -e 's/(^|[^0-9.])([0-9]+)([0-9]3)/12,3/g' -e ta

Here, :a is just a label, and the final ta tests for a successful replacement, and jumps back to a if a replacement was done, in effect repeating the process as many times as it does something. Hence 1234567 will turn into 1,234,567.

• 
  sed 's/foo/bar/g' number.txt: This reads the file number.txt, and replaces the regex pattern foo with bar. This will occur for all matches on each line (/g).


• 
  (^|[^0-9.])([0-9]+)([0-9]3): This is the pattern to replace. Each part in the escaped parentheses (…) is a "capturing group". The pattern inside is "captured" for later use.
  
  
  • 
    (^|[^0-9.]): Find the beginning of the line ^ or | a character that is not a numeral or period [^0-9.]. This essentially finds the character preceding a number.
  
  
  • 
    ([0-9]+): Find one or more numerals [0-9]+.
  
  
  • 
    ([0-9]3): Find 3 numerals [0-9]3.
  
  


• 
  12,3: replace the above matches with the first two capturing groups followed by , then the last capturing group. In other words, insert , between the second and third patterns.

Because sed is "greedy", it will try to maximise the length of the match. Hence the final capturing group will be the last three numerals in the number.

N.B. many of the "special" characters are escaped with , e.g. (…) and 3. If your sed supported "extended regular expressions" with -E or -r, then you would not have to escape these. This would improve readability.