download

Creating a copy of your website using GNU Wget for Windows or OS X

chrisbitting, , , Leave a comment

There are times when you want to have a copy of your site (the frontend / user side). GNU Wget has been around a long time, but in my opinion, it’s still a great tool to backup / mirror websites.

Wget has many options and parameters, of which I won’t even scratch the surface, but below are the simple steps to get Wget setup and running on Windows and OSX machines. Wget is a command line utility, so it might appear overwhelming, but don’t worry, it’s cake!

Windows steps:

Step 1. Download / install Wget. Visit http://gnuwin32.sourceforge.net/packages/wget.htm and choose to download the Setup labeled “Complete package, except sources.”

Step 2. After installation is finished, open a command prompt (cmd.exe).

Step 3. Go to your GNU application folder (on 64 bit it’s in C:\Program Files (x86)\GnuWin32\bin, on 32 bit, it’s probably in C:\Program Files\GnuWin32\bin).

Step 4. To test if wget is installed correctly, run “wget -V“. It should return the current version and some credit. If not, revisit previous steps.

Step 5. To download / mirror a site, run  wget -e robots=off -r -l 0 -P “c:\\temp” http://www.chrisbitting.com – replacing “c:\temp” with the folder you want the site files to download to and “chrisbitting.com” with your site address.

wget_pc

 

You should now see the command prompt update with the progress – depending on the size of your site – it may take some time to download everything. After it’s finished, your directory should contact a mirror of your site, including html, css, images, etc.

 

Apple OS X steps:

Step 1. Open a blank terminal.

Step 2. Install homebrew by running:

ruby -e “$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)”

Step 3. After installing brew (and entering your password), run:

brew doctor

Step 4. Now install Wget using:

brew install wget

Step 5. When installation finishes, run “wget -V” to ensure Wget installed correctly. It should return the current version and some credit. If not, revisit previous steps.

Step 6. To download / mirror a site, run  wget -e robots=off -r -l 0 -P “/temp” http://www.chrisbitting.com – replacing “/temp” with the folder you want the site files to download to and “chrisbitting.com” with your site address.

wget_osx

You should now see the terminal update with the progress – depending on the size of your site – it may take some time to download everything. After it’s finished, your folder should contact a mirror of your site, including html, css, images, etc.

To see the multitude of options Wget provides, run “wget –help“. Happy downloading!

 

Creating a copy of your website using GNU Wget for Windows or OS X

Downloading a remote file with vb.net / c# using WebClient

chrisbitting, , , , , Leave a comment

Can you believe it’s 2013 already? Me either. Anyhow, just wanted to provide a simple solution for a very popularly asked question: “how can I download a remote file?.” Well, below is a basic method using the WebClient class. It’s pretty cut and dry, just pass a remote file (ie: http://www.chrisbitting.com/image.jpg) and a local file (c:\temp\file.jpg) and it downloads. If you’re interested how to do this with cookies enabled for authentication, let me know!

vb.net

Private Sub dloadhttpFile(fileToDownload As String, localFile As String)
        'Examples:
        'fileToDownload = http://www.chrisbitting.com/image.jpg
        'localFile = c:\files\someimage.jpg

        Try
            'create an instance of WebClient - the heart of downloading a file
            Dim fileReader As New WebClient()

            'check if the file already exists locally
            If Not (System.IO.File.Exists(localFile)) Then
                fileReader.DownloadFile(fileToDownload, localFile)
            End If
        Catch ex As HttpListenerException
            Console.WriteLine(("Error Downloading: " & fileToDownload & " - ") + ex.Message)
        Catch ex As Exception
            Console.WriteLine(("Error Downloading: " & fileToDownload & " - ") + ex.Message)
        End Try
    End Sub

c#

   private void dloadhttpFile(string fileToDownload, string localFile)
        {
             //Examples:
             //fileToDownload = http://www.chrisbitting.com/image.jpg
             //localFile = c:\files\someimage.jpg

            try
            {
                //create an instance of WebClient - the heart of downloading a file
                WebClient fileReader = new WebClient();

                //check if the file already exists locally
                if (!(System.IO.File.Exists(localFile)))
                {
                    fileReader.DownloadFile(fileToDownload, localFile);
                }
            }
            catch (HttpListenerException ex)
            {
                Console.WriteLine("Error Downloading: " + fileToDownload + " - " + ex.Message);
            }
            catch (Exception ex)
            {
                Console.WriteLine("Error Downloading: " + fileToDownload + " - " + ex.Message);
            }
        }
Downloading a remote file with vb.net / c# using WebClient