So I’m old-school… wedgie me!
As any beard-sporting, elbow-pad-wearing history teacher will tell you; in order to truly understand the present and drive the future, you first need to understand the past. Having been around before Google, having used HotBot and the Yahoo! Directory to find pages on the web you get a certain sense of context. One thing that I find consistently frustrating and surprising whenever I talk to some site owners and some fellow search professionals is this kind of victim attitude in response to Google updates “they keep changing the rules”…
I don’t think they do, to be honest. I think the thing that’s changed is that Google have got better at detecting your shite linkbuilding tactics. You know the type…read a few blogs in 2007 that talked about having lots of links to help you rank, and then something about free online directories and well, it all went wrong from there. Following instructions will only ever take anyone so far, for a limited period of time. To truly understand and be able to lead, innovate and market a product or concept online; you need to get it from first principles. One of the things that we do at theMediaFlow in our approach to staff training is start at the beginning, so that our people understand the basic origins of information retrieval (as it applies to the web). We find that this gets people up to speed faster and able to make decisions much earlier than would be, with a more prescribed approach to training. I thought it might be useful to share our “history lesson” reading list.
An extremely thorough resource, that starts right back at the roots of the idea for the creation of the web. On of the most important points to take from all of the information here (and I’ll admit it’s some years since I last read through in detail) is that the web emerged before there was really a solution to navigate the information. Previously records, resources and content have been collated in hierarchical classification systems (think of a library) but navigating the web pre search engines required either knowledge of the existence of a web page or the serendipitous discovery of a page linked to from another page. With a thorough examination of different types of emerging solutions, from directories to meta-search to what we know today as a web search engine; Search Engine History is a thorough record of what was once a very diverse landscape.
An amazing and recent post, written as a reflective and personal account of how Inktomi worked (TF-IDF), what they achieved and the limitations of that search experience that allowed Google to blast past Inktomi and all the others. (Yahoo! aquired Inktomi in early 2004). Basch was a senior engineer at Inktomi and an early employee of what was for a time the number 1 search engine. This post has particular resonance and sentimental value for me personally; having joined Yahoo! not long after the acquisition and using some legacy systems driven by Inktomi.
Patents confer authority or right (if granted) to an individual or company for sole use of what it is that is patented. With search engines and other fields of computer science patent documents are particularly fascinating in that a patent has to apply to invention rather than something innate. i.e. I can’t patent grass. Maths is arguably innate and it can be at times quite difficult to patent (for example) a a formula, therefore search patents often go to some lengths to demonstrate the invention by expanding on the likely application of that formula (or sometimes technology). A patent document in our field can therefore often be quite a complex document to process and understand; however Bill Slawski is notoriously adept at examination and interpretation of patent documentation. In this series of posts Slawski explains the most critical search patents of the past decade and more.
For organisations like AOL, MSN and Yahoo! being the starting destination for web users became the number one mission. Whilst from different roots (Yahoo! evolving from a directory, MSN from Microsoft, AOL from online connectivity and ISP services) the portals were fighting for hearts and minds of internet users. Collating content, web services (like email and instant messaging), directories and search functionality these giants shared the bulk of all web traffic like old Fleet Street editors controlled the press. Search was not yet habitual and search volume was largely driven by whichever portal your search engine was partnered with. Make no mistake these portals played a pivotal role in the emergence of search and the development of individual search engine businesses.
One of the most critical documents to facilitate understanding of where we are today The Anatomy is the research paper in which Brin and Page present Backrub (the Google prototype name) but the most crucial part of this document is PageRank. PageRank and what the algorithm was designed to determine (i.e. link based relevancy) was the revolutinary difference between Google and al other search engines at the time.
A familiar tale on the web, monetization of search activity became the primary focus. Just as today we have Facebook buying Instagram for $700 Million despite no commercial model; portals and search engines needed to monetize fast. In February 1998 GoTo.com began to offer pay-fpr-performance (cost-per-click) based search advertising. GoTo.com later became Overture Search Services; powering paid search on MSN, Yahoo! and Google. Yes Google.
Google launched their own paid search advertising product and were promptly sued by Overture for infringement of the above patent. Overture was bought by Yahoo! who settled the patent infringement suit out of court; leaving Google free to develop their Adwords product in-house. In my opinion this was the final ingredient in the recipe for search domination. Google had the product, the growing audience and now an uncontested revenue model…the rest as they say, is history!
A detailed look at all of the history and development of search engines, with particular focus on revenue models and the emergence of search as-business-facilitator (for discovered and advertised sites).
And finally one from our very own Paddy Moogan, a starters must read!