The most popular search engines.

Search engines are automated tools that send out 'crawlers' or 'spiders' to scan the web and index every website that they find. The submission process for a website is simply to provide the URL for a page and the search engine will visit the site and index the text and HTML tags.

In contrast to directories, that are compiled manually by humans, the search engines are automated so tend to have a much greater coverage of the web and therefore provide better opportunities for a site to achieve a ranking across more search terms - although optimising a website for each search engine is crucial in order to get these results.

The following is a summary of the most important search engines on the web. They all operate differently and will use a variety of 'algorithms' to retrieve websites once a search has been made on them. More information is provided by links to pages on the history of the main sites and information about how to get a site listed and to achieve a high ranking on each tool.

A late entrant in search engine terms, Google only launched in 1998 but has rapidly become the most popular search engine favoured by savvy web users due to the speed and accuracy of results served. It is now the undisputed leader in terms of search traffic and the most important search tool to achieve a top placement on, with Google results also now being used by Yahoo, AOL and Netscape Search, as well as Freeserve and BBCi in the UK.

Google is also one of the main search engine to use 'link popularity' as a key element impacting search rankings, whilst Google's PageRank measure is a core indicator of how a site may perform in the search results, based on both internal and external factors on a website.

A brief history of Google.
Getting a site indexed and ranked by Google.

Developed in Norway and launched in Spring 1999, Alltheweb/FAST remains relatively unknown as a search engine but it has become established as one of the big players in the field by providing it's results to the Lycos network. It offers a powerful range of search options and an accuracy that is second only to Google.

A brief history of FAST.
Getting a site indexed and ranked by FAST.

Inktomi is not a search engine in its own right, but it spiders the web and provides an index to many key partners such as HotBot and MSN. During 2002 it lost a significant share of the search market by key partners such as AOL and Netscape moving to Google results. At the end of the year Inktomi was acquired by Yahoo and it remains to be seen what changes this will bring to the ever-shrinking group of search tools. The best way to get a site listed on Inktomi is to use the paid submission option through one of their partners (see the section on paying for search engine placement).

A brief history of Inktomi.
Getting a site indexed and ranked by Inktomi.

Since it was launched at the end of 1995, AltaVista was one of the largest and most popular search engines, providing a good collection of search tools and advanced search functions. However, it has suffered financial and operational setbacks during 2001, consequently losing a large number of users to the ascendant Google. It relaunched its interface - again - and provided new search functionality at the end of 2002.

A brief history of AltaVista.
Getting a site indexed and ranked by AltaVista.

One of the original search engines, Lycos still spiders the web but now places more emphasis on search results from FAST and on its directory format, taking data from the Open Directory Project (see the top Directories). Investment in this search tool has reduced its usage over recent years and more prominence is now given to its portal services.

A brief history of Lycos.
Getting a site indexed and ranked by Lycos.

Launched in Spring 1996, HotBot is now owned by Lycos and is a large index with a good range of advanced searching features. Having taken its search results from the Inktomi and Direct Hit databases for some years, it was another search tool to relaunch at the end of 2002 with a new, cleaner look. It now offers a more powerful search functionality with multiple search options allowing users to search through the databases from Google, FAST, Inktomi or Teoma.

A brief history of HotBot.
Getting a site indexed and ranked by HotBot.

Teoma is a relatively new search engine founded in 2000 by a team from Rutgers University in the US. Using a highly regarded combination of criteria to rank relevant websites, this has been seen as a potential new competitor to the dominance established by Google. During 2002 Teoma was purchased by Ask Jeeves and used as the main search results on this tool, as well as retaining its presence as a standalone search tool. It has yet to gain widespread usage in this format however.

A brief history of Teoma.
Getting a site indexed and ranked by Teoma.

Wisenut was a new arrival on the search engine scene during 2001 and initially attracted much attention as a potential new challenger to Google as a fast and accurate search tool. Acquired by Looksmart in 2002, only time will tell whether this search tool is developed further and generates the quality of results that will live up to the early expectations.

And finally, in memory of...

One of the original search engines, Excite became a popular general search tool and portal-based service. It acquired two competitors - Magellan and Webcrawler - in 1996, but sadly succumbed to financial losses at the end of 2001 by closing its own search database and many regional search sites. Excite.com currently provides metasearch results, including from Overture and Inktomi.

Infoseek was another of the original big search engines and was acquired by the Disney-owned Go portal in 1999. Sadly, Disney effectively closed Go and the related 'Go Guides' directory at the end of February 2001 and Go now provides its search results from the Overture 'pay-for-placement' search tool.

Direct Hit was a unique search tool that measured 'click-throughs' and the time spent on websites, to improve the rankings on its search results. It worked closely with key partners, such as Ask Jeeves and HotBot, to refine and develop these results. Owned by Ask Jeeves, this search tool was superceded by Teoma (above) towards the end of 2002.

Northern Light was a popular search tool for academic researchers as it maintains a large index and combines this with a special collection of articles and other documents from more than 7,000 sources that are not normally accessible on the web. At the start of 2002 the free search facility was disbanded although a paid service is still offered to corporate customers.

To find out how your website can achieve a top ranking on these search engines by contacting the web search workshop for more information or to request your FREE site assessment.