How do I use wget or CURL in php to grab a snapshot of a web page with an asterick * in its URL? (Wayback Machine archive.org question) The Next CEO of Stack OverflowHow to make wget save the web page contents to a file?How to use wget or other tool to download a file, for which the link is hidden(activates on-click only)make install - how to use wget, not curlDownload documents referenced by a web page with wgetwhy would curl and wget result in a 403 forbidden?Set timeout for web page response with curlcURL and Iceweasel disagree about TLS certificate validity, despite same CAwget and curl saving web page as gibberish (encrypted?)downloading a file behind a linkHow to curl full web page content?
What happens if you break a law in another country outside of that country?
What day is it again?
Why did early computer designers eschew integers?
Salesforce opportunity stages
How do I keep Mac Emacs from trapping M-`?
Calculate the Mean mean of two numbers
Cannot restore registry to default in Windows 10?
Does the Idaho Potato Commission associate potato skins with healthy eating?
Creating a script with console commands
Can this transistor (2n2222) take 6V on emitter-base? Am I reading datasheet incorrectly?
Is a distribution that is normal, but highly skewed, considered Gaussian?
Free fall ellipse or parabola?
What difference does it make matching a word with/without a trailing whitespace?
Is it okay to majorly distort historical facts while writing a fiction story?
How to pronounce fünf in 45
How exploitable/balanced is this homebrew spell: Spell Permanency?
Can a PhD from a non-TU9 German university become a professor in a TU9 university?
Horror film about a man brought out of cryogenic suspension without a soul, around 1990
Does Germany produce more waste than the US?
Create custom note boxes
Strange use of "whether ... than ..." in official text
Could you use a laser beam as a modulated carrier wave for radio signal?
Direct Implications Between USA and UK in Event of No-Deal Brexit
Why does freezing point matter when picking cooler ice packs?
How do I use wget or CURL in php to grab a snapshot of a web page with an asterick * in its URL? (Wayback Machine archive.org question)
The Next CEO of Stack OverflowHow to make wget save the web page contents to a file?How to use wget or other tool to download a file, for which the link is hidden(activates on-click only)make install - how to use wget, not curlDownload documents referenced by a web page with wgetwhy would curl and wget result in a 403 forbidden?Set timeout for web page response with curlcURL and Iceweasel disagree about TLS certificate validity, despite same CAwget and curl saving web page as gibberish (encrypted?)downloading a file behind a linkHow to curl full web page content?
The following web page on the Wayback Machine (archive.org) has an asterick * in its URL:
https://web.archive.org/web/*/https://www.nasa.gov/
Neither CURL nor wget will allow me to grab the page. I am using the latest version of Linux Mint (2018).
How do I use wget or CURL in php to grab a snapshot of a web page with an asterick * in its URL, specifically https://web.archive.org/web/*/https://www.nasa.gov/?
Thanks! Might httrack work?
EDIT: Open the site web.archive.org/web/%2A/https://www.nasa.gov in a browser. Then compare it with what you see with the wget command. Are they the same? For me the wget command just returns a generic page.
php wget curl special-characters httrack
asked Feb 5 '18 at 20:39
gamedesignergamedesigner
62
bumped to the homepage by Community♦ 2 days ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
The following web page on the Wayback Machine (archive.org) has an asterick * in its URL:
https://web.archive.org/web/*/https://www.nasa.gov/
Neither CURL nor wget will allow me to grab the page. I am using the latest version of Linux Mint (2018).
How do I use wget or CURL in php to grab a snapshot of a web page with an asterick * in its URL, specifically https://web.archive.org/web/*/https://www.nasa.gov/?
Thanks! Might httrack work?
EDIT: Open the site web.archive.org/web/%2A/https://www.nasa.gov in a browser. Then compare it with what you see with the wget command. Are they the same? For me the wget command just returns a generic page.
php wget curl special-characters httrack
asked Feb 5 '18 at 20:39
gamedesignergamedesigner
62
bumped to the homepage by Community♦ 2 days ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
Works for me. What are you trying to run, and what happens when you do? You can edit your question.
– Michael Homer
Feb 5 '18 at 21:12
@MichaelHomer Open the site web.archive.org/web/%2A/nasa.gov in a browser. Then compare it with what you see with the wget command you used above. Are they the same? For me the wget just returns a generic page.
– gamedesigner
Feb 5 '18 at 21:26
I don't think that's the asterisk's doing, just that they don't want you crawling it.
– Michael Homer
Feb 5 '18 at 21:28
@MichaelHomer I don't want to crawl it. I just want a single snapshot of that page.
– gamedesigner
Feb 5 '18 at 21:33
add a comment |
The following web page on the Wayback Machine (archive.org) has an asterick * in its URL:
https://web.archive.org/web/*/https://www.nasa.gov/
Neither CURL nor wget will allow me to grab the page. I am using the latest version of Linux Mint (2018).
How do I use wget or CURL in php to grab a snapshot of a web page with an asterick * in its URL, specifically https://web.archive.org/web/*/https://www.nasa.gov/?
Thanks! Might httrack work?
EDIT: Open the site web.archive.org/web/%2A/https://www.nasa.gov in a browser. Then compare it with what you see with the wget command. Are they the same? For me the wget command just returns a generic page.
php wget curl special-characters httrack
asked Feb 5 '18 at 20:39
gamedesignergamedesigner
62
The following web page on the Wayback Machine (archive.org) has an asterick * in its URL:
https://web.archive.org/web/*/https://www.nasa.gov/
Neither CURL nor wget will allow me to grab the page. I am using the latest version of Linux Mint (2018).
How do I use wget or CURL in php to grab a snapshot of a web page with an asterick * in its URL, specifically https://web.archive.org/web/*/https://www.nasa.gov/?
Thanks! Might httrack work?
EDIT: Open the site web.archive.org/web/%2A/https://www.nasa.gov in a browser. Then compare it with what you see with the wget command. Are they the same? For me the wget command just returns a generic page.
php wget curl special-characters httrack
php wget curl special-characters httrack
asked Feb 5 '18 at 20:39
gamedesignergamedesigner
62
asked Feb 5 '18 at 20:39
gamedesignergamedesigner
62
edited Feb 5 '18 at 21:26
gamedesigner
asked Feb 5 '18 at 20:39
gamedesignergamedesigner
62
asked Feb 5 '18 at 20:39
gamedesignergamedesigner
62
asked Feb 5 '18 at 20:39
gamedesignergamedesigner
62
62
bumped to the homepage by Community♦ 2 days ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ 2 days ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
Works for me. What are you trying to run, and what happens when you do? You can edit your question.
– Michael Homer
Feb 5 '18 at 21:12
@MichaelHomer Open the site web.archive.org/web/%2A/nasa.gov in a browser. Then compare it with what you see with the wget command you used above. Are they the same? For me the wget just returns a generic page.
– gamedesigner
Feb 5 '18 at 21:26
I don't think that's the asterisk's doing, just that they don't want you crawling it.
– Michael Homer
Feb 5 '18 at 21:28
@MichaelHomer I don't want to crawl it. I just want a single snapshot of that page.
– gamedesigner
Feb 5 '18 at 21:33
add a comment |
Works for me. What are you trying to run, and what happens when you do? You can edit your question.
– Michael Homer
Feb 5 '18 at 21:12
@MichaelHomer Open the site web.archive.org/web/%2A/nasa.gov in a browser. Then compare it with what you see with the wget command you used above. Are they the same? For me the wget just returns a generic page.
– gamedesigner
Feb 5 '18 at 21:26
I don't think that's the asterisk's doing, just that they don't want you crawling it.
– Michael Homer
Feb 5 '18 at 21:28
@MichaelHomer I don't want to crawl it. I just want a single snapshot of that page.
– gamedesigner
Feb 5 '18 at 21:33
Works for me. What are you trying to run, and what happens when you do? You can edit your question.
– Michael Homer
Feb 5 '18 at 21:12
Works for me. What are you trying to run, and what happens when you do? You can edit your question.
– Michael Homer
Feb 5 '18 at 21:12
@MichaelHomer Open the site web.archive.org/web/%2A/nasa.gov in a browser. Then compare it with what you see with the wget command you used above. Are they the same? For me the wget just returns a generic page.
– gamedesigner
Feb 5 '18 at 21:26
@MichaelHomer Open the site web.archive.org/web/%2A/nasa.gov in a browser. Then compare it with what you see with the wget command you used above. Are they the same? For me the wget just returns a generic page.
– gamedesigner
Feb 5 '18 at 21:26
I don't think that's the asterisk's doing, just that they don't want you crawling it.
– Michael Homer
Feb 5 '18 at 21:28
I don't think that's the asterisk's doing, just that they don't want you crawling it.
– Michael Homer
Feb 5 '18 at 21:28
@MichaelHomer I don't want to crawl it. I just want a single snapshot of that page.
– gamedesigner
Feb 5 '18 at 21:33
@MichaelHomer I don't want to crawl it. I just want a single snapshot of that page.
– gamedesigner
Feb 5 '18 at 21:33
add a comment |
1 Answer
1
active
oldest
votes
It seems to work fine for me, as long as the URL is enclosed in quotation marks.
If you want to avoid the message "Warning: wildcards not supported in HTTP", the easiest thing to do is to replace the asterisk with its percent-encoded equivalent:
wget "https://web.archive.org/web/%2A/https://www.nasa.gov"
answered Feb 5 '18 at 21:16
ErikFErikF
3,0161513
Open the site web.archive.org/web/%2A/https://www.nasa.gov in a browser. Then compare it with what you see with the wget command you used above. Are they the same? For me the wget just returns a generic page.
– gamedesigner
Feb 5 '18 at 21:25
add a comment |
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f422119%2fhow-do-i-use-wget-or-curl-in-php-to-grab-a-snapshot-of-a-web-page-with-an-asteri%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
It seems to work fine for me, as long as the URL is enclosed in quotation marks.
If you want to avoid the message "Warning: wildcards not supported in HTTP", the easiest thing to do is to replace the asterisk with its percent-encoded equivalent:
wget "https://web.archive.org/web/%2A/https://www.nasa.gov"
answered Feb 5 '18 at 21:16
ErikFErikF
3,0161513
Open the site web.archive.org/web/%2A/https://www.nasa.gov in a browser. Then compare it with what you see with the wget command you used above. Are they the same? For me the wget just returns a generic page.
– gamedesigner
Feb 5 '18 at 21:25
add a comment |
It seems to work fine for me, as long as the URL is enclosed in quotation marks.
If you want to avoid the message "Warning: wildcards not supported in HTTP", the easiest thing to do is to replace the asterisk with its percent-encoded equivalent:
wget "https://web.archive.org/web/%2A/https://www.nasa.gov"
answered Feb 5 '18 at 21:16
ErikFErikF
3,0161513
Open the site web.archive.org/web/%2A/https://www.nasa.gov in a browser. Then compare it with what you see with the wget command you used above. Are they the same? For me the wget just returns a generic page.
– gamedesigner
Feb 5 '18 at 21:25
add a comment |
It seems to work fine for me, as long as the URL is enclosed in quotation marks.
If you want to avoid the message "Warning: wildcards not supported in HTTP", the easiest thing to do is to replace the asterisk with its percent-encoded equivalent:
wget "https://web.archive.org/web/%2A/https://www.nasa.gov"
answered Feb 5 '18 at 21:16
ErikFErikF
3,0161513
It seems to work fine for me, as long as the URL is enclosed in quotation marks.
If you want to avoid the message "Warning: wildcards not supported in HTTP", the easiest thing to do is to replace the asterisk with its percent-encoded equivalent:
wget "https://web.archive.org/web/%2A/https://www.nasa.gov"
answered Feb 5 '18 at 21:16
ErikFErikF
3,0161513
answered Feb 5 '18 at 21:16
ErikFErikF
3,0161513
answered Feb 5 '18 at 21:16
ErikFErikF
3,0161513
answered Feb 5 '18 at 21:16
ErikFErikF
3,0161513
3,0161513
Open the site web.archive.org/web/%2A/https://www.nasa.gov in a browser. Then compare it with what you see with the wget command you used above. Are they the same? For me the wget just returns a generic page.
– gamedesigner
Feb 5 '18 at 21:25
add a comment |
Open the site web.archive.org/web/%2A/https://www.nasa.gov in a browser. Then compare it with what you see with the wget command you used above. Are they the same? For me the wget just returns a generic page.
– gamedesigner
Feb 5 '18 at 21:25
Open the site web.archive.org/web/%2A/https://www.nasa.gov in a browser. Then compare it with what you see with the wget command you used above. Are they the same? For me the wget just returns a generic page.
– gamedesigner
Feb 5 '18 at 21:25
Open the site web.archive.org/web/%2A/https://www.nasa.gov in a browser. Then compare it with what you see with the wget command you used above. Are they the same? For me the wget just returns a generic page.
– gamedesigner
Feb 5 '18 at 21:25
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f422119%2fhow-do-i-use-wget-or-curl-in-php-to-grab-a-snapshot-of-a-web-page-with-an-asteri%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Works for me. What are you trying to run, and what happens when you do? You can edit your question.
– Michael Homer
Feb 5 '18 at 21:12
@MichaelHomer Open the site web.archive.org/web/%2A/nasa.gov in a browser. Then compare it with what you see with the wget command you used above. Are they the same? For me the wget just returns a generic page.
– gamedesigner
Feb 5 '18 at 21:26
I don't think that's the asterisk's doing, just that they don't want you crawling it.
– Michael Homer
Feb 5 '18 at 21:28
@MichaelHomer I don't want to crawl it. I just want a single snapshot of that page.
– gamedesigner
Feb 5 '18 at 21:33