Spider / Download Website with Wget on Windows

Wget has been around forever it seems, but is still get great tool for spidering or downloading content (all content) from a website to your local machine. Below are some simple steps to install Wget and to run it.

To install Wget:

  1. Visit http://gnuwin32.sourceforge.net/packages/wget.htm and download the setup file, labeled “Complete package, except sources”.
  2. Run the setup .exe and leave everything as defaults.
  3. In a command prompt (cmd.exe) change to the C:\Program Files (x86)\GnuWin32\bin> directory (or C:\Program Files\GnuWin32\bin if your on a 32 bit OS)
  4. To test, if you just run wget.exe it should return wget: missing URL

 

To download a website:

  1. In a command prompt (cmd.exe) change to the C:\Program Files (x86)\GnuWin32\bin> directory (or C:\Program Files\GnuWin32\bin if your on a 32 bit OS)
  2. Now run:
    wget -e robots=off --no-check-certificate --recursive --level=0 -P "c:\\somefolder" http://chrisbitting.com

    (obviously replace my website with your own)

In that folder, you should soon files from the site Wget copied.

I’ve only had issues with a site that I had forced to use SSL. Wget didn’t like that.

Spider / Download Website with Wget on Windows

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s