Does search engine always re-crawl the web sites that were indexed before?

Does search engine always re-crawl the web sites that were indexed
before? Looks like it will, because search engine result always produce
different ranks.

Let's say I google on keyword 'software development', the rankings of
the search engine result can be different from tomorrow's result. Well,
it makes sense because web crawler is keep doing searching and updating
the indexing in schedule.

Let's say http://www.mydomain.com is found on page 2 in search result;
is it possible that tomorrow will suddenly drop to page 10, or not
found? I have done some searches in newsgroup postings, and I realize
some people's web sites have been dropped or lower the ranks. How to
make sure the web site always are searchable by the web crawler then? I
guess there is no guarantee, and it depends on which database servers
to hit also.

Search engine is really an interesting and advanced topic. If you have
any good search engine or crawling resources, please advise.

thanks!!

On 29-Nov-2005, "Joe" <strutsng (AT) gmail (DOT) com> wrote:

Let's say http://www.mydomain.com is found on page 2 in search result;
is it possible that tomorrow will suddenly drop to page 10, or not
found? I have done some searches in newsgroup postings, and I realize
some people's web sites have been dropped or lower the ranks.

Sites compete for ranking, so if another competeing site is
updated, the ranking of your own and the competing site may change.
The internet is dynamic, continually changing.
Server logs tell you a lot. Like how often you are being crawled,
what keywords were used to find you, where your visitors are
coming from.
As far as I can judge Google bots tend to crawl the top ranking
sites more frequently than those at rank 180 (or whatever
you get the idea) in the results.

90% of what people actually know about Google, as opposed to pure
speculation, comes from this paper that its founders wrote as grad
students: http://www-db.stanford.edu/~backrub/google.html . That, in
my opinion is one of the better resources to find out more about search
engines.