| Spiders will enver crawl a full site in one day, instead they will crawl two or three pages per day or, leaving a gap inbetween each crawl. This is done until the whole site has been crawled and indexed. This self regimented gap stops the server from becoming over burdened with information. |
SEO Resources 
