Learn about search engines and Web directories

You can direct attention to your Web site by registering it with major search engines and Web directories. Here's how.

The fastest and easiest way to start getting your site attention is to register it with the major search engines and Web directories.

Search engines use automated software (known as robots or spiders) to follow Web hyperlinks, harvesting information about sites as they go. When someone submits a query to a search engine, the engine returns a list of sites, ranking them on their relevance to the keywords used in the search. How search engines assess your site and determine the relevance of the words is difficult to predict, as it often depends on how the specific engine works. Some engines, such as Excite, use artificial intelligence to recognize concepts that frequently appear together. If you search for "animal health," for instance, Excite also returns sites devoted to "veterinary medicine." Other search engines, such as WebCrawler, first list more popular sites—those its database shows are more frequently linked.

Web directories have the modest task of being the yellow pages for the World Wide Web. Unlike search engines, which can find an unregistered site if other sites link to it, directories list your site only after you submit certain pieces of information: the title, the URL, a description, a few keywords, and sometimes a contact email address. (Although it's not necessary, it's also a good idea to register your site with search engines to ensure that your site comes up in queries.) Typically, a directory's reviewers decide whether a site meets their standards and where the site should be placed in the directory's hierarchy of categories.

There's no way to guarantee that your site will come up on top in a search or stand out in a directory (though it helps if your subject matter is obscure and uses a unique vocabulary). But try these tricks to make sure your site pops up in as many searches as possible.

Title your pages effectively

Make sure each of your pages has a descriptive title. Search engines usually give the most weight to the words between a page's title tags. You can use this to your advantage by adding a short, descriptive phrase in the title of each of your pages, which will make sure certain keywords get the attention they deserve. For example, the KillerGardens home page might include the following description in its title to emphasize its specialties:

<TITLE>KillerGardens - The online source for man-eating plants and landscaping</TITLE>

Search engines return their results in the form of titles linked to each site, so descriptive titles draw people to your site. A page with just the name of the site in its title is less compelling than one with a description. And if you don't include title tags at all, your site will be listed in search results as "No Title" or something similarly uninteresting.

Make the most of tags

You can control how search engines catalog your site with two types of <META> tags. <META> tags are part of the HTML code that some search engines, such as AltaVista and Infoseek, look for but most visitors to your pages never see.

<META> description tags let you specify a short summary that appears below the page's title on a search response. (If a page doesn't have a <META> description, search engines usually list the page's first dozen or so words instead.)

<META> keyword tags let you specify the keywords that a search robot should give precedence to when cataloging the page. <META> keywords are typically given less importance than words in the title, but more importance than words found in the page's body.

<META> tags typically go after the title and between the head tags of an HTML page:

<TITLE>KillerGardens - The online source for man-eating plants and landscaping</TITLE>
<META NAME="description" CONTENT="KillerGardens is located in Berkeley, California, and specializes in man-eating plants and landscaping.">
<META NAME="keywords" CONTENT="gardens, gardening, plants, landscaping, dangerous, man-eating, Venus flytraps, fertilizers">

Make sure your keyword list includes both general and specific words related to your site. It's best to make them plural where appropriate and to include derivatives, since you can't rely on all search engines to account for such instances. Having trouble thinking of keywords? Check out eyescream interactive's list of top 200 search words (smuggled from Yahoo).

To ensure that you're making the most of your <META> tags, go to the Meta Medic site and submit your site's URL. This free Perl script checks your page's <META> tags and suggests ways to improve the descriptions or keywords.

Don't repeat keywords

Repeating keywords—whether in the title tag, in the <META> keyword tag, or hidden against a colored background—has long been a popular ploy to convince search engines to list a site high on keyword searches. The tactic worked when search engines were unsophisticated and judged a word's relevance only by the number of times it appeared on a page. Now, most search engines are hip to the trick and count only the first few occurrences of a keyword or phrase. Thus, hundreds of repetitions of "Pamela Anderson" on your celebrity models Web page will probably have no greater effect on the search engines than would ten repetitions. Search engines also rely more on word density (frequency relative to the total size of the page) or distribution (how well the word is spread throughout the page) than on the number of occurrences when they judge relevance. Some search engines, such as Lycos, even penalize your page (by placing it further down the list or not listing it at all) if they suspect you of repeating words to improve its ranking.

Put the important stuff first

Some search engines, such as Lycos, give precedence to text near the top of Web pages. If some of your must-see content is located far down on a page, move it up, put it on a separate page, or make sure the appropriate keywords from the text are in your <META> keyword tag.

Set out Web page "buoys"

While repeating keywords may no longer score you points with search engines, setting adrift multiple versions of Web pages still works. This can be particularly useful if your site features a mix of interests. Let's say you are a Web designer with expertise in both Java and Photoshop. Search engines will give you more exposure to both audiences if you create one page with the title and <META> tags heavily slanted toward programming and another page with tags slanted toward graphic design skills.

Register via submission sites and services

Sure, you could register with search engines and directories one by one, but there's a better way. At free sites such as Submit-It and Add-It, you fill out one form with your Web site's title, URL, keywords, and other pertinent information. The sites guide you through the submission process for the most popular search engines and Web directories, filling in the appropriate fields in each form. All you have to do is click Submit buttons to send your information.

If you're not satisfied registering with the dozen or so sites that account for 99 percent of the searches on the Web, you can use fee-based submission sites and companies to register sites with hundreds of specialized search engines and directories. These services, some of which are high-powered versions of free submission sites, submit your site even to niche search engines such as ChurchSurf, TextileWeb, and Curioscape.

Register via Web promotion software

Web promotion software is similar to online submission sites: the program asks you for the information required by a slew of search engines and directories, then goes online to submit that information. As with fee-based submission Web sites, paying for this software gets your site listed in many more search sites than a free submission site would. Most of the applications offer free updates, so you can keep up when new search engines go online and existing search engines change their formats.

Register multiple pages from your site

Registering the URLs of more than one page from your site is a good idea. Search engines vary in the depth to which they'll catalog your pages. Some, such as HotBot, are thorough, following all available links and cataloging each site in its entirety. Other search engines, such as AltaVista, go down only two or three levels of links before stopping. If your site has important pages that are more than a couple levels down, register them separately.

Editor's Picks