The problem
We cannot be certain what causes a search engine to
index a site because search engines don’t publish the
workings of their ‘spiders’ (the software systems that
traverse the web and index pages on the web). However:
-
pages generated from a database, in such a way that a
“?”, or a complex character string is placed in page
URIs, tend to be given low weightings by search engines, or ignored altogether;
-
pages with significant content which are deep within the structure
of a site, may fail to be indexed at all.
Each of the search engines has one or more spiders that
have a distinctive ‘finger print’ entry in the site web
logs. There is a list of these on Search Engine World. A way to test these points on a specific site is to examine the log-files for the site.
Two remedies
Have at least some static HTML pages high up within the
site’s structure, or on a separate server, but linked to key
dynamically generated pages, if you also use the latter.
Get at least one other indexed site to link to a static area of your site. (There is more about inward linking below.)
Here are some tips which have proved successful on other sites (as well as this one).
Use of Title Tag. Ensure that each page title contains a
short piece of text which is specific to the content of the page,
and built from the sorts of terms which are likely to be used in a
search. This should not contain the name of the web site, nor,
necessarily, use the same text as the page title within the body of the page. Example. The title of this page contains the text "How to get a website highly ranked".
Use of Meta-tags. While it is good practice to populate the
standard meta-tags within the page header, the more popular search
engines now ignore these tags and rely instead on the full text
held within the page. For this reason it is important to
ensure that there is target text in the page body that describes
the page-contents using words you think a searcher might use when
searching.
Standard HTML. The spiders used by the search engines need
to be able to understand the target pages in order to parse them
appropriately. For this reason alone it is good practice to ensure
that all pages are made up of standard-conformant code, for example
HTML 4.01, or XHTML 1.0, for which there are excellent explanations, and validation tools on, or linked to from, the validation area of the W3C site. (This is separate from the usability
benefits which users will get if the site is standards-compliant.)
Paragraph placement. There is some evidence that words held
within the first paragraph and the last paragraph on any page are
given a higher weighting than words in other paragraphs on a page.
Inward links. There is strong evidence that the
greater the number of links on other sites that reference your
site, the higher your page will be ranked. For this reason
it is a good idea to ‘assist’ as many advocates as
possible to reference your site. For example, organisations
which have members or affiliates, may wish to provide HTML code snippets
to enable members to signify their membership by linking to your
site.
Registration on internet directory sites. Google
has a relationship with the Open
Directory Project. Directories like DMOZ are moderated by people, which
increases the likelihood that listed sites contain quality
content. For this reason, all other things being equal, a listing in DMOZ seems to result in a high ranking in Google, and possibly other search
engines.
Outward Links. It seems that the inclusion of outward links to
high quality sites will define your site as a portal, and will tend to raise its ranking. This runs counter to the commonly held view that outward links should not be included on a site because this looses you visitors, who supposedly follow links
away from your site, never to return.
Repeat of key terms. Some claim that the
repetition of key search terms in the text of a page will increase its ranking. However, some search
engines are thought to reduce a site’s ranking, or ignore it
altogether, if this practice is adopted.
News-feeds. It is unlikely that providing a news-feed from
your site will have a direct impact upon your ranking. But if
your news-feed is picked up by other content aggregators with links
pointing to your site, this will ensure that your site gets scanned
more regularly, and it will increase the number of inbound links to
your site. http://www.webreference.com/authoring/languages/xml/rss/intro/ provides
a good technical introduction, while http://www.webreference.com/services/news/
gives a set of examples.
If you have found this page from my web site, or with
a search tool, and want to receive your own mailing directly from now on,
you can sign
up for a subscription.
