Jack <jank200 (AT) poczta (DOT) fm> typed:
in a meta-tag named robots it means "dear crawler, please don't put this
single page into your database but follow links on this page so that you
might find more pages of which you may take some to put them into your
database". It's basically a possibility to let authors manage robot-behavior
without having access to the /robots.txt (which, of course, wouldn't let you
decide for something to be crawled for links but not indexed and something
to be indexed but not crawled for links).