Need shell script to transform a CSV into Apache httpd format The Next CEO of Stack OverflowShell Script to capture file name and file size in CSV formatScripting to split a single CSV row into multipleTransform .csv into 3 columns and a rowNeed output in a good format using shell scriptHow to transform row to collumn in csv file?Merging contents of multiple .csv files into single .csv fileNeed to remove a specific column in a CSVNeed to format output in csv formatScript for changing CSV into Key Value (KV) formatshell script read from csv column and search files
Is HostGator storing my password in plaintext?
Whats the best way to handle refactoring a big file?
I believe this to be a fraud - hired, then asked to cash check and send cash as Bitcoin
India just shot down a satellite from the ground. At what altitude range is the resulting debris field?
How to safely derail a train during transit?
How do I get the green key off the shelf in the Dobby level of Lego Harry Potter 2?
Would this house-rule that treats advantage as a +1 to the roll instead (and disadvantage as -1) and allows them to stack be balanced?
Apart from "berlinern", do any other German dialects have a corresponding verb?
Why doesn't a table tennis ball float on the surface? How do we calculate buoyancy here?
Anatomically Correct Strange Women In Ponds Distributing Swords
Why does standard notation not preserve intervals (visually)
Does the Brexit deal have to be agreed by both Houses?
Customer Requests (Sometimes) Drive Me Bonkers!
Why do professional authors make "consistency" mistakes? And how to avoid them?
Why do remote companies require working in the US?
What does "Its cash flow is deeply negative" mean?
Go Pregnant or Go Home
Is it my responsibility to learn a new technology in my own time my employer wants to implement?
What is the purpose of the Evocation wizard's Potent Cantrip feature?
How easy is it to start Magic from scratch?
When airplanes disconnect from a tanker during air to air refueling, why do they bank so sharply to the right?
Increase performance creating Mandelbrot set in python
How to get regions to plot as graphics
% symbol leads to superlong (forever?) compilations
Need shell script to transform a CSV into Apache httpd format
The Next CEO of Stack OverflowShell Script to capture file name and file size in CSV formatScripting to split a single CSV row into multipleTransform .csv into 3 columns and a rowNeed output in a good format using shell scriptHow to transform row to collumn in csv file?Merging contents of multiple .csv files into single .csv fileNeed to remove a specific column in a CSVNeed to format output in csv formatScript for changing CSV into Key Value (KV) formatshell script read from csv column and search files
Need some pointing in right direction on script to fetch and regex or sed.
Site24x7 provides a URL with a CSV list of their source IP's used for monitoring. (they also provide other formats, CSV seems the least messed up as their structure leaves a lot to be desired. https://www.site24x7.com/multi-location-web-site-monitoring.html )
Like so:
Country,City,IP Address External
Australia,Sydney,"101.0.67.53"
Australia,Melbourne,"125.214.65.59"
Belgium,Brussels,"87.238.165.164"
Brazil,São Paulo,"200.170.83.170"
Brazil,Rio de Janeiro,"201.20.20.237"
Canada,Toronto,"208.69.56.166,
208.69.56.171,
208.69.56.172 "
Canada,Montreal,"199.204.45.153,
199.204.45.154,
199.204.45.155,
199.204.45.156"
I need to save it as an Allow include file in apache. Like so:
Allow from
72.5.230.111
72.5.230.65
72.5.230.84
shell-script text-processing csv
asked Feb 9 '12 at 18:14
8None18None1
1185
add a comment |
Need some pointing in right direction on script to fetch and regex or sed.
Site24x7 provides a URL with a CSV list of their source IP's used for monitoring. (they also provide other formats, CSV seems the least messed up as their structure leaves a lot to be desired. https://www.site24x7.com/multi-location-web-site-monitoring.html )
Like so:
Country,City,IP Address External
Australia,Sydney,"101.0.67.53"
Australia,Melbourne,"125.214.65.59"
Belgium,Brussels,"87.238.165.164"
Brazil,São Paulo,"200.170.83.170"
Brazil,Rio de Janeiro,"201.20.20.237"
Canada,Toronto,"208.69.56.166,
208.69.56.171,
208.69.56.172 "
Canada,Montreal,"199.204.45.153,
199.204.45.154,
199.204.45.155,
199.204.45.156"
I need to save it as an Allow include file in apache. Like so:
Allow from
72.5.230.111
72.5.230.65
72.5.230.84
shell-script text-processing csv
asked Feb 9 '12 at 18:14
8None18None1
1185
add a comment |
Need some pointing in right direction on script to fetch and regex or sed.
Site24x7 provides a URL with a CSV list of their source IP's used for monitoring. (they also provide other formats, CSV seems the least messed up as their structure leaves a lot to be desired. https://www.site24x7.com/multi-location-web-site-monitoring.html )
Like so:
Country,City,IP Address External
Australia,Sydney,"101.0.67.53"
Australia,Melbourne,"125.214.65.59"
Belgium,Brussels,"87.238.165.164"
Brazil,São Paulo,"200.170.83.170"
Brazil,Rio de Janeiro,"201.20.20.237"
Canada,Toronto,"208.69.56.166,
208.69.56.171,
208.69.56.172 "
Canada,Montreal,"199.204.45.153,
199.204.45.154,
199.204.45.155,
199.204.45.156"
I need to save it as an Allow include file in apache. Like so:
Allow from
72.5.230.111
72.5.230.65
72.5.230.84
shell-script text-processing csv
asked Feb 9 '12 at 18:14
8None18None1
1185
Need some pointing in right direction on script to fetch and regex or sed.
Site24x7 provides a URL with a CSV list of their source IP's used for monitoring. (they also provide other formats, CSV seems the least messed up as their structure leaves a lot to be desired. https://www.site24x7.com/multi-location-web-site-monitoring.html )
Like so:
Country,City,IP Address External
Australia,Sydney,"101.0.67.53"
Australia,Melbourne,"125.214.65.59"
Belgium,Brussels,"87.238.165.164"
Brazil,São Paulo,"200.170.83.170"
Brazil,Rio de Janeiro,"201.20.20.237"
Canada,Toronto,"208.69.56.166,
208.69.56.171,
208.69.56.172 "
Canada,Montreal,"199.204.45.153,
199.204.45.154,
199.204.45.155,
199.204.45.156"
I need to save it as an Allow include file in apache. Like so:
Allow from
72.5.230.111
72.5.230.65
72.5.230.84
shell-script text-processing csv
shell-script text-processing csv
asked Feb 9 '12 at 18:14
8None18None1
1185
asked Feb 9 '12 at 18:14
8None18None1
1185
edited Aug 24 '15 at 20:17
8None1
asked Feb 9 '12 at 18:14
8None18None1
1185
asked Feb 9 '12 at 18:14
8None18None1
1185
asked Feb 9 '12 at 18:14
8None18None1
1185
1185
add a comment |
add a comment |
3 Answers
3
active
oldest
votes
#!/bin/bash
webpage="https://www.site24x7.com/multi-location-web-site-monitoring.html"
csv=$(curl -s "$webpage" | grep 'title="CSV"' |
sed 's/^.*href="(http[^ ]*)".*$/1/')
echo -e "Allow from \"
curl -s "$csv" | egrep -o '[0-9.]7,15' | paste -s | sed 's/t/ \n/g'
answered Feb 9 '12 at 21:19
forcefsckforcefsck
5,7862131
Looks Jedi. I'll test in the AM. Thanks, more than expected.
– 8None1
Feb 10 '12 at 5:15
add a comment |
Quick and dirty and not very robust sed(1) one-liner:
echo 'Country,City,IP Address External
Australia,Sydney,"101.0.67.53"
Australia,Melbourne,"125.214.65.59"
Belgium,Brussels,"87.238.165.164"
Brazil,São Paulo,"200.170.83.170"
Brazil,Rio de Janeiro,"201.20.20.237"
Canada,Toronto,"208.69.56.166,
208.69.56.171,
208.69.56.172 "
Canada,Montreal,"199.204.45.153,
199.204.45.154,
199.204.45.155,
199.204.45.156"' |
sed -e 1d -e 's/"$//' -e 's/^[^"]*"/Allow from /' -e 's/,$/ \/'
Output:
Allow from 101.0.67.53
Allow from 125.214.65.59
Allow from 87.238.165.164
Allow from 200.170.83.170
Allow from 201.20.20.237
Allow from 208.69.56.166
208.69.56.171
208.69.56.172
Allow from 199.204.45.153
199.204.45.154
199.204.45.155
199.204.45.156
Assumptions:
- Every IP address group is surrounded with the double quote (
") character - The
"character cannot occur anywhere else except to delimit IP groups. - A comma (
,) on the end of a line means multiple IP addresses are continued on the next line.
answered Feb 9 '12 at 18:39
jw013jw013
36.8k7101125
add a comment |
My solution using dns instead:
echo -n 'Allow from ';
host -4 -T -t A site24x7.enduserexp.com |
cut -d ' ' -f 4 |
tr 'n' ' '
edited yesterday
tripleee
5,29811930
answered yesterday
Karl DaneKarl Dane
111
New contributor
Karl Dane is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f31312%2fneed-shell-script-to-transform-a-csv-into-apache-httpd-format%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
#!/bin/bash
webpage="https://www.site24x7.com/multi-location-web-site-monitoring.html"
csv=$(curl -s "$webpage" | grep 'title="CSV"' |
sed 's/^.*href="(http[^ ]*)".*$/1/')
echo -e "Allow from \"
curl -s "$csv" | egrep -o '[0-9.]7,15' | paste -s | sed 's/t/ \n/g'
answered Feb 9 '12 at 21:19
forcefsckforcefsck
5,7862131
Looks Jedi. I'll test in the AM. Thanks, more than expected.
– 8None1
Feb 10 '12 at 5:15
add a comment |
#!/bin/bash
webpage="https://www.site24x7.com/multi-location-web-site-monitoring.html"
csv=$(curl -s "$webpage" | grep 'title="CSV"' |
sed 's/^.*href="(http[^ ]*)".*$/1/')
echo -e "Allow from \"
curl -s "$csv" | egrep -o '[0-9.]7,15' | paste -s | sed 's/t/ \n/g'
answered Feb 9 '12 at 21:19
forcefsckforcefsck
5,7862131
Looks Jedi. I'll test in the AM. Thanks, more than expected.
– 8None1
Feb 10 '12 at 5:15
add a comment |
#!/bin/bash
webpage="https://www.site24x7.com/multi-location-web-site-monitoring.html"
csv=$(curl -s "$webpage" | grep 'title="CSV"' |
sed 's/^.*href="(http[^ ]*)".*$/1/')
echo -e "Allow from \"
curl -s "$csv" | egrep -o '[0-9.]7,15' | paste -s | sed 's/t/ \n/g'
answered Feb 9 '12 at 21:19
forcefsckforcefsck
5,7862131
#!/bin/bash
webpage="https://www.site24x7.com/multi-location-web-site-monitoring.html"
csv=$(curl -s "$webpage" | grep 'title="CSV"' |
sed 's/^.*href="(http[^ ]*)".*$/1/')
echo -e "Allow from \"
curl -s "$csv" | egrep -o '[0-9.]7,15' | paste -s | sed 's/t/ \n/g'
answered Feb 9 '12 at 21:19
forcefsckforcefsck
5,7862131
answered Feb 9 '12 at 21:19
forcefsckforcefsck
5,7862131
answered Feb 9 '12 at 21:19
forcefsckforcefsck
5,7862131
answered Feb 9 '12 at 21:19
forcefsckforcefsck
5,7862131
5,7862131
Looks Jedi. I'll test in the AM. Thanks, more than expected.
– 8None1
Feb 10 '12 at 5:15
add a comment |
Looks Jedi. I'll test in the AM. Thanks, more than expected.
– 8None1
Feb 10 '12 at 5:15
Looks Jedi. I'll test in the AM. Thanks, more than expected.
– 8None1
Feb 10 '12 at 5:15
Looks Jedi. I'll test in the AM. Thanks, more than expected.
– 8None1
Feb 10 '12 at 5:15
add a comment |
Quick and dirty and not very robust sed(1) one-liner:
echo 'Country,City,IP Address External
Australia,Sydney,"101.0.67.53"
Australia,Melbourne,"125.214.65.59"
Belgium,Brussels,"87.238.165.164"
Brazil,São Paulo,"200.170.83.170"
Brazil,Rio de Janeiro,"201.20.20.237"
Canada,Toronto,"208.69.56.166,
208.69.56.171,
208.69.56.172 "
Canada,Montreal,"199.204.45.153,
199.204.45.154,
199.204.45.155,
199.204.45.156"' |
sed -e 1d -e 's/"$//' -e 's/^[^"]*"/Allow from /' -e 's/,$/ \/'
Output:
Allow from 101.0.67.53
Allow from 125.214.65.59
Allow from 87.238.165.164
Allow from 200.170.83.170
Allow from 201.20.20.237
Allow from 208.69.56.166
208.69.56.171
208.69.56.172
Allow from 199.204.45.153
199.204.45.154
199.204.45.155
199.204.45.156
Assumptions:
- Every IP address group is surrounded with the double quote (
") character - The
"character cannot occur anywhere else except to delimit IP groups. - A comma (
,) on the end of a line means multiple IP addresses are continued on the next line.
answered Feb 9 '12 at 18:39
jw013jw013
36.8k7101125
add a comment |
Quick and dirty and not very robust sed(1) one-liner:
echo 'Country,City,IP Address External
Australia,Sydney,"101.0.67.53"
Australia,Melbourne,"125.214.65.59"
Belgium,Brussels,"87.238.165.164"
Brazil,São Paulo,"200.170.83.170"
Brazil,Rio de Janeiro,"201.20.20.237"
Canada,Toronto,"208.69.56.166,
208.69.56.171,
208.69.56.172 "
Canada,Montreal,"199.204.45.153,
199.204.45.154,
199.204.45.155,
199.204.45.156"' |
sed -e 1d -e 's/"$//' -e 's/^[^"]*"/Allow from /' -e 's/,$/ \/'
Output:
Allow from 101.0.67.53
Allow from 125.214.65.59
Allow from 87.238.165.164
Allow from 200.170.83.170
Allow from 201.20.20.237
Allow from 208.69.56.166
208.69.56.171
208.69.56.172
Allow from 199.204.45.153
199.204.45.154
199.204.45.155
199.204.45.156
Assumptions:
- Every IP address group is surrounded with the double quote (
") character - The
"character cannot occur anywhere else except to delimit IP groups. - A comma (
,) on the end of a line means multiple IP addresses are continued on the next line.
answered Feb 9 '12 at 18:39
jw013jw013
36.8k7101125
add a comment |
Quick and dirty and not very robust sed(1) one-liner:
echo 'Country,City,IP Address External
Australia,Sydney,"101.0.67.53"
Australia,Melbourne,"125.214.65.59"
Belgium,Brussels,"87.238.165.164"
Brazil,São Paulo,"200.170.83.170"
Brazil,Rio de Janeiro,"201.20.20.237"
Canada,Toronto,"208.69.56.166,
208.69.56.171,
208.69.56.172 "
Canada,Montreal,"199.204.45.153,
199.204.45.154,
199.204.45.155,
199.204.45.156"' |
sed -e 1d -e 's/"$//' -e 's/^[^"]*"/Allow from /' -e 's/,$/ \/'
Output:
Allow from 101.0.67.53
Allow from 125.214.65.59
Allow from 87.238.165.164
Allow from 200.170.83.170
Allow from 201.20.20.237
Allow from 208.69.56.166
208.69.56.171
208.69.56.172
Allow from 199.204.45.153
199.204.45.154
199.204.45.155
199.204.45.156
Assumptions:
- Every IP address group is surrounded with the double quote (
") character - The
"character cannot occur anywhere else except to delimit IP groups. - A comma (
,) on the end of a line means multiple IP addresses are continued on the next line.
answered Feb 9 '12 at 18:39
jw013jw013
36.8k7101125
Quick and dirty and not very robust sed(1) one-liner:
echo 'Country,City,IP Address External
Australia,Sydney,"101.0.67.53"
Australia,Melbourne,"125.214.65.59"
Belgium,Brussels,"87.238.165.164"
Brazil,São Paulo,"200.170.83.170"
Brazil,Rio de Janeiro,"201.20.20.237"
Canada,Toronto,"208.69.56.166,
208.69.56.171,
208.69.56.172 "
Canada,Montreal,"199.204.45.153,
199.204.45.154,
199.204.45.155,
199.204.45.156"' |
sed -e 1d -e 's/"$//' -e 's/^[^"]*"/Allow from /' -e 's/,$/ \/'
Output:
Allow from 101.0.67.53
Allow from 125.214.65.59
Allow from 87.238.165.164
Allow from 200.170.83.170
Allow from 201.20.20.237
Allow from 208.69.56.166
208.69.56.171
208.69.56.172
Allow from 199.204.45.153
199.204.45.154
199.204.45.155
199.204.45.156
Assumptions:
- Every IP address group is surrounded with the double quote (
") character - The
"character cannot occur anywhere else except to delimit IP groups. - A comma (
,) on the end of a line means multiple IP addresses are continued on the next line.
answered Feb 9 '12 at 18:39
jw013jw013
36.8k7101125
edited Feb 9 '12 at 18:50
answered Feb 9 '12 at 18:39
jw013jw013
36.8k7101125
answered Feb 9 '12 at 18:39
jw013jw013
36.8k7101125
answered Feb 9 '12 at 18:39
jw013jw013
36.8k7101125
36.8k7101125
add a comment |
add a comment |
My solution using dns instead:
echo -n 'Allow from ';
host -4 -T -t A site24x7.enduserexp.com |
cut -d ' ' -f 4 |
tr 'n' ' '
edited yesterday
tripleee
5,29811930
answered yesterday
Karl DaneKarl Dane
111
New contributor
Karl Dane is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
My solution using dns instead:
echo -n 'Allow from ';
host -4 -T -t A site24x7.enduserexp.com |
cut -d ' ' -f 4 |
tr 'n' ' '
edited yesterday
tripleee
5,29811930
answered yesterday
Karl DaneKarl Dane
111
New contributor
Karl Dane is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
My solution using dns instead:
echo -n 'Allow from ';
host -4 -T -t A site24x7.enduserexp.com |
cut -d ' ' -f 4 |
tr 'n' ' '
edited yesterday
tripleee
5,29811930
answered yesterday
Karl DaneKarl Dane
111
New contributor
Karl Dane is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
My solution using dns instead:
echo -n 'Allow from ';
host -4 -T -t A site24x7.enduserexp.com |
cut -d ' ' -f 4 |
tr 'n' ' '
edited yesterday
tripleee
5,29811930
answered yesterday
Karl DaneKarl Dane
111
New contributor
Karl Dane is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited yesterday
tripleee
5,29811930
edited yesterday
tripleee
5,29811930
edited yesterday
tripleee
5,29811930
5,29811930
answered yesterday
Karl DaneKarl Dane
111
New contributor
Karl Dane is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered yesterday
Karl DaneKarl Dane
111
answered yesterday
Karl DaneKarl Dane
111
111
New contributor
Karl Dane is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Karl Dane is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Karl Dane is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f31312%2fneed-shell-script-to-transform-a-csv-into-apache-httpd-format%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
