If you’re like most people, you will use Google to search for something at least 5 times today. Me? I’m sitting around 10 or 15 at midday. 80-85% of all web searches are done using Google. There are 63,000 searches per second being done on their servers. 15-20% of searches done each day have never previously been searched for, yet somehow Google finds a way to stack up relevant results. But how?
How Google Basically Works
Before we get to that, let’s go back in time. Its 1994 and you come home to find out you have a flood in your basement. You do what everyone does in that situation – you cuss, and eventually get to a yellow book full of businesses (if you were born before or near this time, this is probably that big yellow book that you still receive occasionally and recycle, or if you are crafty you use it as a monitor stand on your desk). You look up plumbers, find some ads for experts that can help you out and randomly pick one. They come out and stop the flood from getting worse but charge you an arm and a leg.
The same thing happens to you in 1999 (you should investigate getting a new house). This time you have learned your lesson (and the world of tomorrow is here!) and you get on your awesome, new Window 98 desktop computer. You get online using your 56k modem you just bought from the Best Buy nearby and after 30 seconds of hissing and clicking noises, you’re ready. You go to AOL.com and look for plumbers. After about 10 minutes you find an ad online that has a plumber, in St. Louis, which is 4 hours away from you, and you give up. You go back to the big yellow book and pick a different plumber. They get the job done but take forever to get to your house, increasing the damage due to your poor, ceaselessly perspiring pipes.
But Wait, My Increasingly More Recent Examples Continue
It’s 2004 and you are a decade older, but you uniquely give people pause regarding the cliché about being wiser, as you still live in the same cursed place – a house that has shown itself to be always one beer run away from you coming back to a flooded basement. You come in all ready to watch the newest Lost episode (rumor is now that the island is just purgatory, but you know the writers are better than that) only to find out your basement flooded again. This time you are ready with a much faster DSL internet connection and go to your favorite place in mere moments. You pull up Google.com, look for an ‘Indianapolis-based plumber’ and find one. You call the emergency number on their website and get ahold of someone. They come out and tell you that it looks like over the last 10 years you must have had some poor plumbers do work because the pipes are being held together by chewing gum and mouse droppings. The good news is they fixed things up for you, the bad news is, they complained about the gum and mouse droppings a lot…like more than any plumber should complain about something considering their line of work.
How Did We Get to a Working Search Engine?
In 1998, Sergey Brin and Larry Page were two PHD candidates at Stanford. And like the stereotype of Stanford attendees, they were both smart and had a startup on their minds. They came up with the idea to create a search engine that worked like your dad used to work to find a good plumber (i.e. not the yellow book). That guy would call some of his friends that had used plumbers previously, and ask them who they used and if they had a good experience. Of course, he had the benefit of taking the time to do that, unlike your stupid 2004 self, because he was fancy and knew how to shut off the water, so it didn’t keep flooding the basement.
The Way Other Search Engines Worked at That Time
Search engines at that time included Excite, Lycos, and surprising failure Dogpile, a search engine that I assume was literally named based on how poor its results were. The engines were manually created or curated by volunteers, a poor way to find new websites, or would send out spiders (little web-based robots) to find all the websites they could, which allowed just about anything to be shown. Typically, the websites that used the words people were searching for the most times within their copy came up first or even worse, the websites simply paid to be showed near the top. Common websites using the copy trick to show up towards the top may have had critical information such as, “Plumbers like plumber Joe plumb plumbing with the best of plumbers.” And when companies competed to show ads based on static ad rates, the company spending the most money would always win regardless of utility to the searcher. Voilà– garbage results.
Google was different. They used a system called PageRank (maybe co-founder Larry Page arm wrestled Brin and won the naming rights) that would use web-based spiders but instead of simply counting the number of times words were used on a website, they looked at links to other websites using those words. For example, if a website linked the phrase ‘This is my favorite plumber in Indianapolis’ (coincidentally most websites at that time were just lists of people’s favorite things) to bobsplumbing.com, Google’s PageRank algorithm would figure that bobsplumbing.com should come up when people search for ‘best Indianapolis plumber’. These links acted like your father’s friends did when they told him he should hire Bob’s Plumbing because they used Bob last year and Bob never once complained when he found gum holding their pipes together or the KFC napkins mysteriously flushed down the toilet.
What Google ended up with was a much better search result. That idea – using a link as a true word of mouth referral was the killer idea of Google and it still a critical part of the results you see today. As the company grew, many people tried cheating Google’s PageRank and so Google developed many more things they looked at (currently around 200 indicators) that determine search results. They also found a way to have a completely silent auction within the time someone makes a search, and they provide a result (these silent auctions which occur within a blink of an eye helped them generate 25 billion dollars last quarter in ad revenue).
If you have ever wondered how Google makes money or how those shoe ads from Zappos always follow you around the net, you won’t want to miss the next discussion.
