wget: Time-Stamping Usage
5.1 Time-Stamping Usage
=======================
The usage of time-stamping is simple. Say you would like to download a
file so that it keeps its date of modification.
wget -S http://www.gnu.ai.mit.edu/
A simple ‘ls -l’ shows that the time stamp on the local file equals
the state of the ‘Last-Modified’ header, as returned by the server. As
you can see, the time-stamping info is preserved locally, even without
‘-N’ (at least for HTTP).
Several days later, you would like Wget to check if the remote file
has changed, and download it if it has.
wget -N http://www.gnu.ai.mit.edu/
Wget will ask the server for the last-modified date. If the local
file has the same timestamp as the server, or a newer one, the remote
file will not be re-fetched. However, if the remote file is more
recent, Wget will proceed to fetch it.
The same goes for FTP. For example:
wget "ftp://ftp.ifi.uio.no/pub/emacs/gnus/*"
(The quotes around that URL are to prevent the shell from trying to
interpret the ‘*’.)
After download, a local directory listing will show that the
timestamps match those on the remote server. Reissuing the command with
‘-N’ will make Wget re-fetch _only_ the files that have been modified
since the last download.
If you wished to mirror the GNU archive every week, you would use a
command like the following, weekly:
wget --timestamping -r ftp://ftp.gnu.org/pub/gnu/
Note that time-stamping will only work for files for which the server
gives a timestamp. For HTTP, this depends on getting a ‘Last-Modified’
header. For FTP, this depends on getting a directory listing with dates
in a format that Wget can parse (⇒FTP Time-Stamping Internals).
Info Catalog
wget: Time-Stamping
wget: Time-Stamping
wget: HTTP Time-Stamping Internals