The Internet Archive discovers and captures web pages through many different web crawls.
At any given time several distinct crawls are running, some for months, and some every day or longer.
View the web archive through the Wayback Machine.
Web wide crawl with initial seedlist and crawler configuration from October 2010
TIMESTAMPS
The Wayback Machine - https://web.archive.org/web/20110106202448/http://www.searchengineoptimising.com/glossary/glossary-of-computer-and-internet-terms/bot
Bots are usually used online and they are software programs that are automated and perform specific functions when ordered to. For example they can monitor online chat rooms and can throw people who do not abide by the rules of the chat room out.