New Search Engine

Many many years ago I had created a regional search engine and sold it for a very small amount of money.  Recently, while scraping the content from a website (because we were moving CMS providers), the idea surfaced again.  The scraping gave me great insight into why to use markup (

Fast forward to January 2015.  My son was working on a website and had some questions about the coding.  While helping him as well as providing him with a book to learn more, I decided once more to see about building a little search engine.  After a little research a decent domain was located that had some SEO juice and just got put back out for sale –


Going back to the old-school search engine way of doing things, WebzLinks only picks up the root domain (on interior pages), site title, site description, and some of the body.  I figured the body would serve for information if the site description didn’t exist.

After crawling my first 1,000 domains I looked at the database.  It was amazing the number of websites that have no description.  Just over 40% of the website home pages had no meta description tag.  This is especially surprising given the fact that this is SEO 101.

The other issue was so surprising.  When scraping the beginning of the body of a page I was getting menu text and on many pages getting some javascript.  This issue wasn’t so surprising because html5 is still fairly new and a lot of these website were old or not updated in quite a while.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s