Problems with the default Drupal robotstxt file

There are several problems with the default Drupal robots.txt file. If you use Google Webmaster Tool's robots.txt testing utility (detailed instructions on this utility later in this chapter) to test each line of the file, you'll find that a lot of paths which look like they're being blocked will actually be crawled. The reason is that Drupal does not require the trailing slash ( ) after the path to show you the content. Because of the way robots.txt files are parsed, Googlebot will avoid the...

WC markup validation

Drupal is a well-written piece of software that produces well-formed web sites. However, don't assume that it will still be that way when you're done with it. Not all of the modules, themes, or content on your site will pass muster. This is especially true if your site is open to users to create their own content. You should run a comprehensive scan of the site to check for improperly formed code, broken links, and other oversights that could hinder your search engine positioning. Obviously,...

Meta tags

Meta tags are pieces of text in the header of your web site that tell search engine spiders about your site. They are not visible to your site visitors, which make them handy places to communicate details about your site that visitors just don't care about. The problem is that in the stone age of search engines 1997 many people abused the meta tags by stuffing them full of keywords. This was invisible to their visitors but the search engines gave a lot of credence to the meta tags, so it was a...

Good body content

Base pages are the anchor content on any site and are always visible. They're considered the main pages and they don't change or move around much. These are the pages that you'll build links to and will probably show up in the search engines for the most difficult terms. The following are some good base pages on your site that you should consider creating. They're easy to create and are a positive addition to any site Calendar of Upcoming Events Supplemental pages are the pages on your site...

Using taxonomy to organize around the main topics

For the main content of your site, you should create a taxonomy category with a handful of keyword-rich terms that all of your content will fit under. Do not allow free tagging in this main content category as it creates too many categories and becomes a nightmare to maintain. If you want to have free tagging for other types of content, like blogs or products, and it's a good idea in many cases then create a separate taxonomy for it. Don't go crazy with deep hierarchies. It makes sense in some...

Submitting your XML sitemap to Google

Carry out the following steps in order to submit your XML sitemap to Google 1. If you have not already done so, you need to verify your web site with Google Webmaster Tools. Refer to Chapter 1, The Tools You'll Need, for details. 2. Now point your browser to Google's Webmaster Tools at http www.google.com Webmasters . Click on Sign in to Webmaster Tools, as shown in the following screenshot Improve traffic with Google Webmaster Tools Welcome to your one-stop shop for webmaster resources that...

The page title module

The page title module gives you full control of your page titles throughout your site. It lets you Write your own page titles anywhere you can create content Define a pattern that will create search engine optimized titles automatically as you create new content Two active Drupal developers, Nicolas Thompson and John Albin, maintain the Page Title module. We should all doff our hats to them for keeping this module up-to-date. Thanks, guys The page title module uses the token module. Tokens are...

Installing the Meta tags module

Carry out the following steps to install the Meta tags module 1. Download the Meta tags module from the following link 2. Install as per normal Drupal modules. See Chapter 1, The Tools You'll Need, for step-by-step module installation instructions. 3. Be sure to turn on the module on the Modules admin page 4. Adjust the permissions, as needed, at the following link Is it the Meta tags module or the Nodewords module They're one and the same, actually. The early versions of the module were...

Setting up a Google Website Optimizer account

Carry out the following steps in order to set up a Google Website Optimizer account 1. Go to www.google.com websiteoptimizer. 2. Log in using your Google account refer to Chapter 1, The Tools You'll Need, if you don't already have a Google account . The following screenshot shows the Google Website Optimizer sign in screen 3. Select your time zone and click on Continue. 4. Click on the Getting Started button and agree with the terms of service.

Setting up the Google News sitemap generator module

Download the Google News sitemap module from http drupal.org project googlenews and install it just like a normal Drupal module. See Chapter 1, The Tools You'll Need, for step-by-step module installation instructions. 2. Go to or go to your admin screen and click the Administer Site Configuration Google News sitemap feed link. You'll see a screen similar to the following screenshot Home Administer gt Site configuration Choose the categories that are generated by the Google news feed Select...

Robotstxt htaccess and WC Validation

Much of the SEO that we've accomplished so far is visible to your visitors for example, titles, headings, body text, and even a sitemap or two . In this chapter, we're going to address some of the more technical aspects of on-page SEO. Over the last ten years, many elements have been added to the HTML specification. The search engines themselves have developed other elements to help you communicate better with them. Since our ultimate goal is to do well by the search engines and our visitors,...

Bold strong and emphasized text

Many search engines take into account text that is set apart on the page. You can set apart a word or phrase using a couple of methods. Bold and italics will do just that, bold or italicize the text. lt strong gt and lt em gt are terms that can be styled to look like anything you'd like using a style sheet in your theme. Typically, strong and emphasis tend to look like bold or italics. All are good methods for pulling a word out of a block of text and making it stand out. This text is bold....

Setting up a visitorfacing sitemap

Carry out the following steps to set up a visitor-facing sitemap 1. Download the Sitemap module from http drupal.org project site_map and install it just like a normal Drupal module. See Chapter 1, The Tools You'll Need, for step-by-step module installation instructions. 2. Go to or go to your admin screen and click the Administer Site configuration Site Map link. You'll see the Site map, as shown on the following screenshot Home gt Administer gt Site configuration Define a message to be...

Short catchy headlines

Start your content with a short, catchy headline. The headline is the most obvious part of any piece of content that you create. It's what reaches out and grabs the reader and keeps them interested long enough to read the first couple of sentences. Remember, most Internet readers don't read, they skim. Therefore, it's important that your headline grabs the eye and the imagination. If your headlines are weak, and your content can't carry the team, you have a major problem. Great headlines will...

Structure your site hierarchically

There's a reason you learned the outlining format in grade school. It's easier to organize related ideas when they're structured hierarchically. It turns out that it's easier for search engines to figure your site out when it's structured that way as well. So, send a long-overdue thank you note to your fifth grade language arts teacher and let's get organized. Organized content is useful content. Keep similar ideas together and your site will be more useful to your visitors. It's also easy to...

Scanning your site with the Wc Html Validator

Carry out the following steps to scan your site with the W3C HTML Validator 1. Go to http validator.w3.org . You will see a screen similar to the following screenshot 1. Go to http validator.w3.org . You will see a screen similar to the following screenshot 2. Paste your URL in the Address box and click on the Check button. 3. If your site passes validation, you should see a success message, as shown in the following screenshot Chsok this markup HTML. XHTML. of Wet documents Chsok this markup...

How to pick the best keywords

By now, you know the goals of your SEO campaign branding, lead generation, sales transactions, and so on. Now, it's time to dig into the data. There are infinite number of ways to go about doing keyword research. I'm going to take you step-by-step through one of them. It's not necessarily the right or the best way but it's a good, solid technique that I've used many times to produce excellent results. You need one more thing a spreadsheet. Whether you use Excel, Numbers, or Google Docs, the...

Submitting your Google News sitemap to Google News

Once you've assembled your new articles for a single publication label, submit them to Google News sitemaps by carrying out the following steps 1. Check Google News to see if your site is already included. If not, you can request inclusion by visiting the following link, http www.google.com The inclusion process may take up to a few weeks, and you'll only be able to submit a News sitemap once this process is complete. 2. If your site is already showing up in Google News then proceed. If not,...

Around

Two years ago, you might have created a path that looked like this But now, you realize that it would be better to have a path looking like this In Drupal, this is easy. The actual content sits at a node reference location for example, node 123. So you just have to create a new alias, right Not so fast Although it is possible to assign multiple aliases to a node, this is probably not a good idea from an SEO perspective. If Google sees two identical pages on your web site, they're not going to...

Additional changes to the robotstxt file

Using directives and pattern matching commands, the robots.txt file can exclude entire sections of the site from the crawlers like the admin pages, certain individual files like cron.php, and some directories like scripts and modules. In many cases, though, you should tweak your robots.txt file for optimal SEO results. Here are several changes you can make to the file to meet your needs in certain situations You are developing a new site and you don't want it to show up in any search engine...

Table of Contents

Installing 99 of Drupal modules 9 Installing the remaining 1 Drupal modules 10 Creating a Google Analytics account and installing it on your Drupal site 22 Verify your site with Google 26 Google Webmaster Tools settings 28 Understanding search engine crawlers 30 Keywords aggregate searchers into organized groups 38 Why keyword research is important 39 Page impression or ad impression 42 How to set up the Top Searches module 43 How to scrape your competitors' web site for keywords 45 How to pick...

Paid tools

Here are a handful of useful tools that are not quite free. They are useful and have a place in every good SEO bag of tricks. Eye-tracking studies show you where people are looking while they're on your web site. Dries Buytart, the founder of Drupal, showed one during his 2008 State of Drupal address at the Boston DrupalCon. The heat map shows where the users look in the first five seconds after landing on Drupal's main administration page. The red X's show where the users clicked, as shown in...

Setting up the Syndication module

Carry out the following steps to set up the Syndication module 1. Download the Syndication module from the following link, and install it just like a normal Drupal module. Refer to Chapter 1, The Tools You'll Need, for step-by-step module installation instructions. 2. Go to or go to your admin screen and click on the Administer Content Management RSS feed syndication link. You'll see the RSS feed syndication screen, as shown in the following screenshot The configuration options have been saved...

Path to conversion

Once you know what you want visitors to do, it's time to define clear steps you want them to take to do those things. The fantastic usability book, titled Don't Make Me Think, espouses the idea that it's not how many clicks someone has to go through it's how easy it is to make the click decision. If clicking is easy and provides value then there's a high chance the user will take the next step towards becoming your customer. Key to understanding what your customers are thinking is to understand...

The alt and title attributes

The alt attribute specifies alternative text to display if the image, movie, or other media can't be displayed. Suppose someone has images turned off in their browser settings common on dial-up connections and text-based browsers or the images get moved. The alt text would be displayed instead. For search engines, the alt text can be another indicator of what that element of the page is about and thence, what the entire page or site is about. Unfortunately, many black-hat SEOs have used alt and...

Turning on Drupals builtin caching

Thanks to browsers, most people know what caching is. If you go to a web site once, Firefox and other modern browsers will keep a snapshot of that page for a while so that if you hit the Back button, it instantly loads the page. Servers can do that as well. When someone visits a page, the server will cache and compress it. When a second person visits the same page, the server can send the cached page instead of going back to the database. It's much faster and saves bandwidth. Drupal has its own...

How to gather a keywords list

Carry out the following steps to gather a keywords list 1. Visit You'll start with the most basic keyword that you can think of, for your industry, like pc, and click Get keyword ideas as shown in the following screenshot 1. Visit You'll start with the most basic keyword that you can think of, for your industry, like pc, and click Get keyword ideas as shown in the following screenshot How would you like to generate keyword ideas Enter ore keyword or phrase per line Descriptive words or phrases...

Using Googles Webmaster Tools to evaluate your robotstxt file

Warning The robots.txt file is easy to mess up It's not written for humans so it's easy for site owners and webmasters to misunderstand exactly how to use it. Take care not to break your SEO campaign simply because a poorly written robots, txt file is excluding your site from Google. Fortunately, Google's Webmaster Tools provides a helpful utility that shows you exactly which pages are being excluded and included by your robots.txt file. Carry out the following steps to evaluate your robots.txt...

Adding your XML Sitemap to the robotstxt file

Another way that that the robots.txt file helps you search engine optimize your Drupal site is by allowing you to specify where your sitemaps are located. While you probably want to submit your sitemap directly to Google, Yahoo , and MSN, it's a good idea to put a reference to it in the robots.txt file for all of those other search engines. You can do this by carrying out the following steps 1. Open the robots.txt file for editing. 2. The sitemap directive is independent of the User-agent line,...

Keeping content compliant with the HTML Purifier module

Suppose you've run the W3C check and there are some problems with user-created content You'd never make mistakes, would you . You can easily fix it by editing the node, but how do you keep more issues from cropping up over time The answer is the HTML Purifier module. This module utilizes a program called HTML Purifier that does several things to keep your code clean. It will Remove malicious code with a secure yet permissive whitelist Make sure your documents are standards compliant The HTML...

Integrating Google Website Optimizer with Drupal

Drupal Plugin Website Optimiser

Carry out the following steps in order to integrate Google Website Optimizer with Drupal 1. Download the Google Website Optimizer module from http drupal. org project google_website_optimizer and install it just like a normal Drupal module. Refer to Chapter 1,The Tools You'll Need, for step-by-step module installation instructions. 2. Visit the following link, http www.yourDrupalsite.com admin settings google_website_optimizer, or go to your admin screen and click on Administer Site...

Automating paths with Pathauto and Path Redirect

One of the great things about Drupal is its ability to automate things that you've been doing manually up until now. With the PathAuto module, you can tell Drupal to automatically give a path to any new content you create based on a pattern that you define in the admin screens. Path Redirect gives some excellent SEO options that you should use when using PathAuto. We'll look at both. PathAuto is co-maintained by Greg Knaddison, Mike Ryan, and Frederik Freso S. Olesen. Great job, guys Path...

Specifying the XML sitemap priority for nodes

Now that you have the XML sitemap module properly installed and configured, you can start defining the priority of the content on your site by default, the priority is .5. However, there are times when you may want Google to visit some content more often and other times when you may not want your content in the sitemap at all like the comment or contact us submission forms . Each node now has an XML sitemap section that looks like the following screenshot You can adjust the priority on a...

Mastering the htaccess file

There is a server configuration file at the root level of your Drupal 6 site called the .htaccess file. This file is a list of instructions to your web server software, usually Apache. These instructions are very helpful for cleaning up some redirects and otherwise making your site function a bit better for the search engines. In Chapter 1, The Tools You'll Need, we told Google Webmaster Tools that we wanted our site to show up in Google with or without the www in the URL. The .htaccess file...

Using Open Calais bulk processing to tag existing content

Maybe you've got a site with a lot of content already. Consider using OpenCalais' Bulk Processing to go through your site and create tags for each piece of content. This is a very quick way to organize your content, you just need to carry out the following steps 1. After you have OpenCalais installed on your Drupal site, point your browser to bulk-process, or navigate to Administer Site configuration Calais Configuration Calais Bulk Processing. You'll see a screen similar to the following...

Using Open Calais to offer More Like This blocks

One of the ways that search engines determines the relevance of a piece of content to a particular topic is where it links to and which pages link to it. For example, real estate and mortgages are related to Google because a lot of realtors link to mortgage companies and vice versa. A More Like This block shows your visitors more content that is similar to what is being displayed on the page. This is not only good to increase the amount of time visitors spend on your site, but it's also a great...

Modules

A module is a community-created plugin that enhances Drupal's core functionality. From XML sitemaps to better page titles, modules are crucial to the search engine optimization of any Drupal site. Installing modules is easy and once you know how to install one, you probably know how to install them all. _ For a complete explanation of installing modules, check out the following links http drupal.org node 12 0 641 and http drupal.org node 120642. In the upcoming chapters, you'll come across a...

AB testing with Google Website Optimizer

A great conversion rate is not a destination it's a journey. To get the most out of that journey, you need to try different things to increase the usability and conversion rate of your web site. One tool to help you do that testing is Google's Website Optimizer. Google Website Optimizer allows you to test and optimize site content and design by simultaneously deploying two different versions of your web site. It's not as hard as it sounds For example, say you wanted to test two different calls...

Setting up the XML Sitemap module

The XML Sitemap module creates a sitemap that conforms to the sitemap.org specification. Which XML Sitemap module should you use There are two versions of the XML Sitemap module for Drupal 6. The 1.x version is, as of this writing, considered the stable release and should be used for production sites. However, if you have a site with more than about 2000 nodes, you should probably consider using the 2.x version. From www.drupal.org 'The 6.x-2.x branch is a complete refactoring with...

Installing Open Calais on your Drupal site

It takes two different modules and an extra download to get OpenCalais working. It's not difficult, though. Just carry out the following steps 1. Firstly, install the modules by carrying out the following steps Download the Resource Description Framework RDF module from http www.drupal.org project rdf and install it just like a normal Drupal module. See Chapter 1 for step-by-step module installation instructions. However, do not enable it until you've finished Step 2 and have installed ARC 2...

Using Open Calais to tag and organize your content

Suppose you've got hundreds of pages of content on your site and you haven't been doing a very good job so far of categorizing everything. Or, perhaps you have a lot of users creating posts every day, and it's too much trouble to train everyone or go back through and tag everything that's created. Well, there's an automated solution called OpenCalais also known as Calais that will do the heavy lifting for you. OpenCalais reads in the content of your nodes, as you create them, or in bulk...

Setting up RSS feeds

Ever seen an icon, similar to the following screenshot, on the web, before These icons are indicators that the site's owner wants to share his content with you. If you ever subscribe to a podcast, or read a blog in a newsreader, then you've used an RSS feed. RSS is a standard way of sharing your Drupal web site without visitors needing to come back to your site each day or hour, or minute to check if you have new content. RSS stands for Really Simple Syndication or Rich Site Summary, depending...

Configuring the Meta tags module

Drupal Screenshot Meta Tags

Carry out the following steps to configure the Meta tags module 1. Go to or click on Administer Content Management Meta tags. You'll see something like this Home gt Administer gt Content management Enter a short copyright statement one line that will be used on all pages unless specifically set. Global GeoURL _ Enter a GeoURL latitude, longitude that will be used on all pages unless specifically set Global keywords Enter a comma separated list of global keywords. These global keywords will be...

Google Webmaster Tools

If you've got a site that shows up in Google then you need a Google Webmaster Tools account. The Google Webmaster Tools provide you with detailed reports about your pages' visibility on Google. It's one of the most direct ways that you can communicate with Google about your site. It allows you to upload an XML sitemap, see if there are any problems with your site and fix them. It even lets you control the Google spider so that it doesn't drag your site down with constant visits. To use the...