How can I force wget to stop after X seconds?
I have a script that downloads images and from time to time it gets stuck and refuses to "timeout".
What I've tried:
--tries=3 --connect-timeout=30
From ps aux:
root 26543 0.0 0.0 38636 1656 ? S 20:40 0:00 wget -nc --tries=3 --connect-timeout=30 --restrict-file-names=nocontrol -O 18112012/image.jpg
5 Answers
You can run the wget command as a background process and send a SIGKILL to forcibly kill it after sleeping for a certain amount of time.
wget ... &
wget_pid=$!
counter=0
timeout=60
while [[ -n $(ps -e) | grep "$wget_pid") && "$counter" -lt "$timeout" ]]
do
sleep 1
counter=$(($counter+1))
done
if [[ -n $(ps -e) | grep "$wget_pid") ]]; then
kill -s SIGKILL "$wget_pid"
fi
Explanation:
wget ... &- the¬ation at the end runs the command in the background as opposed to the foregroundwget_pid=$!-$!is a special shell variable that contains the process id of the most recently executed command. Here we save it to a variable calledwget_pid.while [[ -n $(ps -e) | grep "$wget_pid") && "$counter" -lt "$timeout" ]]- Look for the process every one second, if it's still there, keep waiting until a timeout limit.kill -s SIGKILL "$wget_pid"- We usekillto forcibly kill the wget process running in the background by sending it a SIGKILL signal.
Easiest way is to use the timeout(1) command, part of GNU coreutils, so available pretty much anywhere bash is installed:
timeout 60 wget ..various wget args..
or if you want to hard-kill wget if its running too long:
timeout -s KILL 60 wget ..various wget args..
Recent wget versions (1.19+, at least) allow to set a timeout:
-T, --timeout=SECONDS set all timeout values to SECONDS
--dns-timeout=SECS set the DNS lookup timeout to SECS
--connect-timeout=SECS set the connect timeout to SECS
--read-timeout=SECS set the read timeout to SECS
--waitretry=SECONDS wait 1..SECONDS between retries of a retrieval
-t, --tries=NUMBER set number of retries to NUMBER (0 unlimits)
Example
wget --timeout 4 --tries 1 ""
This example will try once to connect and get URL . After 4 seconds it will timeout.
Community wiki because this is mostly a copy of sampson-chen's answer, but I wanted to point out a couple of things:
wget ... &
# Strictly speaking you can just use the job number,
# which is probably %1, but saving the pid is also fine.
wget_pid=$!
counter=0
timeout=60
# use kill -0 to check if a pid is still running
while kill -0 "$wget_pid" && (( counter < timeout )); do
sleep 1
(( counter++ ))
done
# if killing nothing is distasteful, use kill -0 one more time.
# also think a SIGKILL is overkill since the question doesn't imply wget needs it.
kill -0 "$wget_pid" && kill "$wget_pid"
I recently noticed that wget 1.14 was silently ignoring the --timeout option, it worked ok when I updated it to 1.19
