The Ask search engine has taken on a new look in the past month with some interesting features in its search results, as it makes a further attempt to attract more user traffic away from the ‘big 3′ of Google, Yahoo! and MSN.
There was a good piece of news announced in April by Google and the other main search engines - Yahoo! and MSN - concerning further enhancements of the recent Sitemap protocol. This standardised method of supplying these search engines with a recognised format to help with the indexing of a website has now been extended to the robots.txt file as well.
We reported in December about the significant step forward made by the ‘big 3′ search engines in agreeing a standardised protocol for submitting a Sitemap document - in a number of different formats - which would enable these search tools to index pages from a website more efficiently and more often, if required.
Now Google and the others (who have also been joined by Ask) have taken this another step forward by announcing a simple addition that can be made to a robots.txt file, which will enable these Sitemap files to be identified and indexed by the indexing ’spiders’. Although not essential, the robots.txt file is something that the automated ‘spiders’ sent out by the main search engines will look for when they first arrive on a website’s server, as it should indicate which sections of the site can, or can’t, be indexed. This can help to streamline the indexing process and is something we recommend that every website should include – you can find out more about this file here.
Now, the search engines that use the standardised Sitemap protocol have extended the service to allow website managers to insert a command in the robots.txt file that will direct the spiders to the Sitemap file. This means that you don’t necessarily need to register to use Google’s Webmaster tools, for example, (although this can still be the best option) but you can just edit your robots file to include the location of the Sitemap file and then let the search engines do the rest.
There’s more information about this on the official Sitemaps protocol website.
There are plently of searches you can make on the main search engines to discover more about your own or competitor’s websites. We’ve discussed some of these previously:
Recently we’ve come accross some more advanced searches you can make, including one to check the pages within a website that are included within Google’s supplemental results.
Google have recently introduced some changes to their UK listings which allow websites to obtain better visibility at the top of the search results.
What do we see happening this year? Here are our 5 predictions for 2007.
1) A growing emphasis on site conversion: since search engine marketing has now gained a level of popularity that is making it much more competitive, both in terms of achieving ‘natural’ rankings and the average cost per click from pay-per-click advertising rising, there will be greater importance placed on converting visitors to a site into an enquiry or sale, so that the ROI (return on investment) for business websites can be maintained or improved. This will encompass many elements, from the site’s usability, to the effectiveness of copy and images, the checkout process for e-commerce sites, plus the response time and quality once contact is made.
2) Search marketing becomes a core discipline: the other main aspect from the growth of search marketing as a significant area of advertising spend is that it is no longer a niche sector but a significant consideration for inclusion in any website marketing plan. This will be reflected in new jobs and departments being established in mainstream agencies and large companies who want to develop this area, but will also be reflected in a growing skills shortage, leading to more training courses and demand for skilled staff who have a broad experience in this market.
3) Local search gains prominence: the advances in local search opportunities by Google, Yahoo! and MSN will offer small businesses the chance to use search marketing effectively to reach their specific catchment areas. It will create an alternative to more traditional forms of local advertising as more people also use local search on the web to find businesses or other suppliers for their immediate needs. The technology for targeting local search traffic will improve and give smaller companies better alternatives to reach their audience.
4) The question of click fraud is tackled: one of the biggest issues for the pay-per-click providers and advertisers is the question of click fraud, whereby advertisers are maliciously targeted to increase their spend levels from irrelevant click activity. There will be increasing clamour to resolve this issue and although it can’t be stamped out entirely, the main players like Google and Yahoo! will introduce tighter controls to identify and combat this practice, despite remaining secretive about their methods.
5) Social networks become a new marketing challenge: the dramatic growth in social networks like MySpace and Bebo in 2006 was a significant new trend, with MySpace reportedly becoming the most visited US website by the end of the year, overtaking Yahoo!’s long-established top position. Other ‘virtual world’ sites like Second Life indicate the trend towards the web being used as a social interactive tool and one that marketers will begin to target more aggressively. However, the nature of these sites mean that marketing techniques will need to be more creative to reach these target audiences as the wrong strategy can quickly turn into negative publicity.
This time last year we made 5 predictions for 2006 and so here’s a quick summary of our list and what actually happened:
1) Internet growth continues: OK, so this wasn’t difficult to get wrong, but the rapid uptake in broadband usage, both in the UK, Australia and other countries, has demonstrated that the web is now a significant trading zone for both consumer and business spend. Improved access speeds have led to new creativity in advertising and the use of interactive sites, although search marketing continues to grow at a much faster rate than other forms of online advertising as more and more companies realise the potential of this form of cost-effective online advertising. It is becoming harder for new entrants to establish a successful business on the web, although better marketing planning and campaign implementation can help to ensure the best opportunities against established online businesses.
2) Search marketing gains recognition: as noted above, search marketing continues to boom and the introduction of Microsoft’s new PPC tool last year added another option to this section of the market in the US and UK. Both the optimisation and PPC market is now becoming more crowded and increasingly competitive in many business sectors, yet there are still good results to be achieved. Many large marketing agencies are now developing their in-house search marketing teams, whereas the older established specialists continue to do well as new entrants find it harder to gain a foothold. Just before the end of the year, the merger between one of the UK’s largest SEM agencies and a German search firm was indicative of the way this business sector has developed over the past 2-3 years.
3) The PPC market gains a new channel: Microsoft’s AdCenter tool launched in the US at the start of the year and then in the UK during the late summer, marking a significant change to the established PPC monopoly between Google and Yahoo! Although it has yet to demonstrate any significant volumes of traffic, the Microsoft service has introduced new levels of PPC management tools that have been replicated by Google, to some extent, and will also feature in Yahoo!’s new system to be launched in 2007.
4) Another big year for Google: yes, Google continued to grab the headlines and make lots of money over the past year, with new developments in its online ‘office’ tools, including a calendar system and spreadsheet. Although Google didn’t get to launch its own web browser yet, it did pull off a major purchase of the highly popular YouTube website and we will therefore expect more developments in this area soon.
5) Online viral attacks: thankfully our prediction didn’t happen in any significant form, although spam remains a major issue for business with reports that it grew by 30% last year and accounted for around 90% of all email traffic. The technological and legal developments have so far failed to reduce the volumes reaching most inboxes – if anything, December saw a particularly big increase in trashy automated messages. Microsoft did launch an updated version of the Internet Explorer browser, although this had to fix some potential loopholes within the first few weeks.
During November, the ‘big 3’ search engines – Google, Yahoo! and Microsoft – have got together to agree a standardised system to allow website owners to submit their site details to the search indexes by using a unified format site map file. This is a positive move for all parties and may lead to further cooperation between the search engines in the future.
During October new versions of both Microsoft Internet Explorer and Firefox were released, marking the latest stage in the competition between these two browsers.
During August Microsoft launched its pay per click advertising service adCenter, in the UK, replacing Yahoo’s provision of adverts to the MSN search engine.
We have been busy setting up new accounts on this tool, which offers a good level of functionality, and the ability to target adverts by time of day, and demographic, through analysing details of Microsoft’s Hotmail, and Passport users.
Following the success of the pay per click advertising model, the race is on to use auction based, pay for performance models beyond text adverts, with Google recently announcing the introduction of video commercials online.
Google plans to provide video adverts through its AdSense content network initially (other websites which syndicate Google adverts), targeted to the content of the pages that the user is viewing. Payment is made when a user clicks to view the advert.
Signup to our newsletter about search engine marketing & online marketing:
