Friday, August 10, 2007

HTML Meta Tags - Robots Meta name

clipped from www.i18nguy.com
ROBOTS
GOOGLEBOT
<META NAME="GOOGLEBOT" CONTENT="NOARCHIVE">

<META NAME="ROBOTS" CONTENT="ALL">

<META NAME="ROBOTS" CONTENT="INDEX,NOFOLLOW">

<META NAME="ROBOTS" CONTENT="NOINDEX,FOLLOW">

<META NAME="ROBOTS" CONTENT="NONE">

CONTENT="ALL | NONE | NOINDEX | INDEX| NOFOLLOW | FOLLOW | NOARCHIVE"

default = empty = "ALL"

"NONE" = "NOINDEX, NOFOLLOW"

The CONTENT field is a comma separated list:

INDEX: search engine robots should include this page.

FOLLOW: robots should follow links from this page to other pages.

NOINDEX: links can be explored, although the page is not indexed.

NOFOLLOW: the page can be indexed, but no links are explored.

NONE: robots can ignore the page.

NOARCHIVE: Google uses this to prevent archiving of the page.
See http://www.google.com/bot.html

In addition to the ROBOTS META Command above,
Google supports a GOOGLEBOT command. With it, you can tell
Google that you do not want the page archived, but allow other search engines to

0 comments: