I am trying to mirror a site which I manage for backup purposes.
Here is my wget command:
wget \
--mirror \ # Download the whole site, updating local files as needed.
--page-requisites \ # Get all assets/elements (CSS/JS/images).
--adjust-extension \ # Save files with .html on the end.
--span-hosts \ # Include necessary assets from offsite as well.
--convert-links \ # Update links to still work in the static version.
--backup-converted \ # Backup original HTML files before converting links
--restrict-file-names=windows \ # Modify filenames to work in Windows as well.
--domains=********.*** \ # Do not follow links outside this domain.
--no-parent \ # Don't follow links outside the directory you pass in.
--append-output=wget.log \ # Send output to log file
--rejected-log=wget-rejected.log \ # separate log file for rejected requests
--reject=SwitchToAdmin,SignOut
--show-progress \ # Show progress bar
--random-wait \ # Roandomize wait time (0.5 - 1.5 * wait)
--wait=2 \ # median wait time in seconds
# The URL to download
and here are the results:
--2021-05-03 14:14:20--
Resolving ( )... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘ ’
--2021-05-03 14:14:20--
Resolving download (download)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘download’
--2021-05-03 14:14:20--
Resolving the (the)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘the’
--2021-05-03 14:14:20--
Resolving whole (whole)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘whole’
--2021-05-03 14:14:20--
Resolving site, (site,)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘site,’
--2021-05-03 14:14:20--
Resolving updating (updating)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘updating’
--2021-05-03 14:14:20--
Resolving local (local)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘local’
--2021-05-03 14:14:20--
Resolving files (files)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘files’
--2021-05-03 14:14:20--
Resolving as (as)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘as’
--2021-05-03 14:14:20--
Resolving needed. (needed.)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘needed.’
wget.sh: line 9: syntax error near unexpected token `('
wget.sh: line 9: ` --page-requisites \ # Get all assets/elements (CSS/JS/images).'
Why is wget trying to access when I asked it to access
1 Answer
In bash, \ is used to escape the following character.
It is often used (and probably this is what you want) to escape the newline character to enable a multi-line command.
wget \
-arg1 \
-arg2
If you want the \ to escape the newline character, it must be placed directly in front of it, otherwise it won't work:
wget \ # some comment
-arg1 \
-arg2
... will escape the space and the following will be executed (check set -x):
+ wget ' #' some comment
Further, as your next line(s) is/are not connected anymore, you will likely get an error saying something like:
`-arg1`: command not found.
