wget: Time-Stamping
5 Time-Stamping
***************
One of the most important aspects of mirroring information from the
Internet is updating your archives.
Downloading the whole archive again and again, just to replace a few
changed files is expensive, both in terms of wasted bandwidth and money,
and the time to do the update. This is why all the mirroring tools
offer the option of incremental updating.
Such an updating mechanism means that the remote server is scanned in
search of “new” files. Only those new files will be downloaded in the
place of the old ones.
A file is considered new if one of these two conditions are met:
1. A file of that name does not already exist locally.
2. A file of that name does exist, but the remote file was modified
more recently than the local file.
To implement this, the program needs to be aware of the time of last
modification of both local and remote files. We call this information
the “time-stamp” of a file.
The time-stamping in GNU Wget is turned on using ‘--timestamping’
(‘-N’) option, or through ‘timestamping = on’ directive in ‘.wgetrc’.
With this option, for each file it intends to download, Wget will check
whether a local file of the same name exists. If it does, and the
remote file is not newer, Wget will not download it.
If the local file does not exist, or the sizes of the files do not
match, Wget will download the remote file no matter what the time-stamps
say.
Menu
Info Catalog
wget: Following Links
wget: Top
wget: Startup File