Spider / Download Website with Wget on Windows

Wget has been around forever it seems, but is still get great tool for spidering or downloading content (all content) from a website to your local machine. Below are some simple steps to install Wget and to run it.

To install Wget:

  1. Visit http://gnuwin32.sourceforge.net/packages/wget.htm and download the setup file, labeled “Complete package, except sources”.
  2. Run the setup .exe and leave everything as defaults.
  3. In a command prompt (cmd.exe) change to the C:\Program Files (x86)\GnuWin32\bin> directory (or C:\Program Files\GnuWin32\bin if your on a 32 bit OS)
  4. To test, if you just run wget.exe it should return wget: missing URL

 

To download a website:

  1. In a command prompt (cmd.exe) change to the C:\Program Files (x86)\GnuWin32\bin> directory (or C:\Program Files\GnuWin32\bin if your on a 32 bit OS)
  2. Now run: wget -e robots=off –no-check-certificate –recursive –level=0 -P “c:\\somefolder” http://chrisbitting.com (obviously replace my website with your own)

In that folder, you should soon files from the site Wget copied.

I’ve only had issues with a site that I had forced to use SSL. Wget didn’t like that.

Spider / Download Website with Wget on Windows

SQL Server Date Convert Samples

Many times I want a particular date format but can’t remember which style it is. Below is a list of the styles and examples of how it will display. I don’t know if there are any additional Styles, if you know of any, comment and I can add them.

Usage:

CONVERT(VARCHAR, GETDATE(), 1) AS [Date]
Style Date
1 02/24/17
2 17.02.24
3 24/02/17
4 24.02.17
5 24-02-17
6 24 Feb 17
7 Feb 24, 17
8 13:10:36
9 Feb 24 2017 1:10:36:037PM
10 02-24-17
11 17/02/24
12 170224
13 24 Feb 2017 13:10:36:037
14 13:10:36:037
20 2017-02-24 13:10:36
21 2017-02-24 13:10:36.037
22 02/24/17 1:10:36 PM
23 2017-02-24
24 13:10:36
25 2017-02-24 13:10:36.037
100 Feb 24 2017 1:10PM
101 02/24/2017
102 2017.02.24
103 24/02/2017
104 24.02.2017
105 24-02-2017
106 24 Feb 2017
107 Feb 24, 2017
108 13:10:36
109 Feb 24 2017 1:10:36:037PM
110 02-24-2017
111 2017/02/24
112 20170224
113 24 Feb 2017 13:10:36:037
114 13:10:36:037
120 2017-02-24 13:10:36
121 2017-02-24 13:10:36.037
126 2017-02-24T13:10:36.037
127 2017-02-24T13:10:36.037
131 28/05/1438 1:10:36:037PM
SQL Server Date Convert Samples

C# – Updating a ListView from a new thread (multi-thread)

If you’re using ListViews in c# (Winforms or WPF – like a ListBox – but better), you’ve probably wanted to update the ListView data (ItemsSource – I also hope you’re using binding) without blocking the UI. Below is a simple method I’ve used that seems to work well, and allows you to pass parameters if needed. You can also download the entire project at: https://github.com/cbitting/ListViewUpdateMultiThread

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Documents;

namespace wpfMultiThreadListViewUpdate
{
    public class Person
    {
        public string Name { get; set; }
    }

    public partial class MainWindow : Window
    {
        private Thread _thread;

        public MainWindow()
        {
            InitializeComponent();
        }

        private void btnUpdate_Click(object sender, RoutedEventArgs e)
        {
            _thread = new Thread(() => showSomePeople(new Random().Next(4, 15), lstvwPeople));
            _thread.Start();
        }

        private void showSomePeople(int numberToGet, ListView listvw)
        {
            List<Person> somePeople = getSomePeople(numberToGet);

            Dispatcher.BeginInvoke(new Action(delegate()
            {
                listvw.ItemsSource = somePeople;
            }));
        }

        private List<Person> getSomePeople(int numberToGet)
        {
            List<Person> peeps = new List<Person>();

            for (int i = 0; i < numberToGet; i++)
            {
                peeps.Add(new Person() { Name = randomName() });
                Thread.Sleep(200);
            }

            return peeps;
        }

        private string randomName()
        {
            const string chrs = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
            Random rdm = new Random();
            return new string(Enumerable.Repeat(chrs, 6)
              .Select(s => s[rdm.Next(s.Length)]).ToArray());
        }
    }
}
C# – Updating a ListView from a new thread (multi-thread)

Using SortedSet as a Sorted List with Custom Comparer – c#

Sometimes you have a list of objects you want to stay sorted. You can usually use an IList and then linq: orderby to sort the list – but other times it helps to have the list always sorted. In those times, SortedSet works very well. In my below example, a sorted set contains some people – and whenever you look at the list in the SortedSet – they are already sorted (by age, name & location).

I was concerned about the performance of the SortedSet vs. a List, so after running some tests, as expected – the SortedSet needs some initial time when adding items (to compare) vs. the List which will is fast to write to – but needs a little time when sorting the list after. Below is some really fast numbers (data in milliseconds) I’ve compiled based on adding / reading 25000 records:

sortedSet-performance

Below is the sample using SortedSet and an IComparer:

using System;
using System.Collections.Generic;

namespace sortedSet
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            //create the SortedSet
            SortedSet<Person> people = new SortedSet<Person>(new PersonComparer());

            //add some random folks:
            Random rnd = new Random();
            for (int i = 1; i <= 25; i++)
            {
                //new person
                Person person = new Person();
                person.name = "Bob " + i.ToString();
                person.location = "Miami" + i.ToString();

                //random age
                int r = rnd.Next(1, 99);
                person.age = r;

                //add person to set
                people.Add(person);
            }

            //show our list:
            foreach (Person person in people)
            {
                Console.WriteLine(person.name + " - Age: " + person.age.ToString());
            }

            Console.ReadLine();
        }
    }

    //create comparer
    internal class PersonComparer : IComparer<Person>
    {
        public int Compare(Person x, Person y)
        {
            //first by age
            int result = x.age.CompareTo(y.age);

            //then name
            if (result == 0)
                result = x.name.CompareTo(y.name);

            //a third sort
            if (result == 0)
                result = x.location.CompareTo(y.location);

            return result;
        }
    }

    internal class Person
    {
        public string name { get; set; }

        public string location { get; set; }

        public int age { get; set; }
    }
}

-If you don’t have a need to sort your lists – don’t forget about HashSet (very fast).

Using SortedSet as a Sorted List with Custom Comparer – c#

Local web server for testing / development using Node.js and http-server

localhost8080If you’re developing html / javascript applications and want to test locally, many times you will go beyond what local file access (file:///C:/…) in browsers will allow (like XMLHttpRequests, json calls, cross domain access and Access-Control-Allow-Origin restrictions).

A simple solution instead of deploying your code to apache or IIS is to install a local http server. http-server for Node.js is a fast, easy install and app that will allow you to use any directory as a http://localhost.

Installing this simple http server only takes a few steps:

  1. Install node.js if you don’t already have installed (from http://nodejs.org)
  2. In a command prompt / terminal, now run:
    npm install http-server -g
    

    (this installs http-server globally so you can access from any folder or directory)

  3. Now using command prompt or terminal, browser to a folder with some html you want to serve as http. (ie: c:\someproject\).
  4. Run:
    http-server
    
  5. Open your browser and visit http://localhost:8080.

 

You can change port 8080 (the default) to anything using “-p”, so http-server -p 8088 would change your local site to http://localhost:8088

Run http-server –help to see the other options available for running.

Local web server for testing / development using Node.js and http-server

Microsoft’s Social “so.cl” – Not off to a good start!

Being a historic Microsoft fan, I have long waited for the day when Microsoft releases a social network. However, currently, there is an error just getting on the list for so.cl:

Microsoft so.cl error

“Error binding ‘function(){}’ view. The view named ‘Anonymous’ does not derive from the ViewBase class. http://www.so.cl/#/error/waitinglist

And yes, this is in IE9.

But I still have hope and I’m eager to see the network when it’s finished!

Microsoft’s Social “so.cl” – Not off to a good start!